From erlang@REDACTED Fri Jun 1 09:03:02 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 1 Jun 2012 09:03:02 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <4FC6938A.6090601@mansionfamily.plus.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> <4FC6938A.6090601@mansionfamily.plus.com> Message-ID: On Wed, May 30, 2012 at 11:39 PM, james wrote: >> If they aren't written down, they WILL be forgotten >> sooner or later. > > > I would also like to suggest that the act of writing down a design forces a > degree of rigour that is hard to achieve otherwise. ?I have several times > thought that I had considered a design from 'every angle', only to find that > when I *write it down*, some of the angles were imperial and some were > metric, and some were just smoke and mirrors. I agree 100% - when I get *really stuck* - I write out in clear English what I want my program to do. I often start programming without a clear idea of what my program is to do - I "think" I have a clear idea but as the program evolves I find that the original idea was unclear. Writing down in another language (ie English, instead of Erlang or C or whatever) forces a cognitive shift in my brain - I suspect that actually *different* parts of the brain are involved. I can also feel the ideas moving around in my head - feel is too strong a word here. There is also another strange phenomena - until a program is completed, the problem is not really solved. But as soon as the program is completed in its entirety, a strange thing happens - then, and not sooner, I realise that a better solution was possible. Why is this? - I suspect a different part of the brain is involved. Some part of the brain says "move on problem completed" and as soon as that happens the "move on" part of the brain realises that the solution you have just arrived at was flawed - so throw it all away and start again. Knuth, he of the wise ways, says this rewriting process should be repeated seven times. In organisations this causes problems. Just about when the system is to be delivered you fix the final bug and then realize that it was all wrong and needs a total rewrite, and you start the rewrite the day before delivery ... I have never met a project manager on the planet who understands this (Project mangers are from Venus, Programmers are from Mars) I also believe in working in a distraction-free environment (no phone, email, twitters etc) you have to listen very carefully to catch the small fleeting thoughts in your brain. When I solve sudoku I get instant flashes where I see where numbers are to be placed - but these are fleeting and easy to miss. I suspect my right-brain instantly sees a solution and tries to tell my left-brain, but since I wasn't listening I missed it. Programming is actually a form of "applied thinking" where you can actually test the results of the thought process. Since all the real work takes place inside the brain, a brain-friendly environment is essential. I once worked in an open-plan office, after a while I noticed my (programmer) productivity dropped to zero and that all my programs were written at home. I could wax on but i'm supposed to be writing a book, but have been distracted by the Erlang mailing list ... Cheers /Joe > > When something is written down, the handwaving stops and the engineering > starts. > From erlang@REDACTED Fri Jun 1 09:22:27 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 1 Jun 2012 09:22:27 +0200 Subject: [erlang-questions] unanswered beginner questions in the list Message-ID: Hi, We had an interesting discussion at the EUC. One relatively new user observed that several new converts to Erlang had asked questions in this list and not gotten a reply. I took a quick look at the list statistics and indeed this it true. What usually happens is - I read the list every day and see a lot of questions that I know how to answer - but I don't immediately answer. The reason for this is that a) I haven't time and b) I hope somebody else will answer and then I won't need to. I think it would be a good idea if those of us who might qualify as being "knowledgeable persons" could take the trouble to occasionally answer questions from apparent beginners that have not been answered. After a threshold of say 3 days - then at least *somebody* could attempt an answer so that beginner to Erlang don't feel unloved when they reach to this list. Of course, some extremely knowledgable and well-known people ask extremely difficult questions here - and zero replies should probably be taken as a sign that "nobody knows the answer" We could even have volunteers among those who posses the knowledge to answer all unanswered beginners questions. Cheers /Joe From watson.timothy@REDACTED Fri Jun 1 09:27:34 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 01 Jun 2012 08:27:34 +0100 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: Message-ID: <4FC86EE6.3050509@gmail.com> On 01/06/2012 08:22, Joe Armstrong wrote: > I think it would be a good idea if those of us who might qualify as being > "knowledgeable persons" could take the trouble to occasionally answer > questions from apparent beginners that have not been answered. > > After a threshold of say 3 days - then at least *somebody* could attempt > an answer so that beginner to Erlang don't feel unloved when they > reach to this list. It would be easier to do this if there was a some kind of separate feed for mail that hasn't been answered in the last 3 days or whatever. Otherwise you loose a lot of cycles actually keeping track of who has asked what, and I suspect that is the time killer for a lot of people, rather than not having time to answer basic questions. I have *no idea* about managing mailing list software, so I'm not even sure if that idea is feasible. > Of course, some extremely knowledgable and well-known people ask > extremely difficult > questions here - and zero replies should probably > be taken as a sign that "nobody knows the answer" > > We could even have volunteers among those who posses the knowledge to answer > all unanswered beginners questions. > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Fri Jun 1 09:31:13 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 1 Jun 2012 16:31:13 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> <4FC6938A.6090601@mansionfamily.plus.com> Message-ID: Joe writes: "... you fix the final bug and then realize that it was all wrong and needs a total rewrite, and you start the rewrite the day before delivery ...." A friend of mine, former coder who went on to manage coders, once met me for a beer and moaned that he'd spent the day "fighting a sudden outbreak of Truth and Beauty." ;-) This is actually an advantage of generating as much of your implementation as possible from your documentation (as ROK suggests above): it gets you more of a Separation of Concerns between where you want your Truth (the spec) and where you need the Beauty (the code.) With ths point in mind, I started http://beam.referata.com/wiki/Main_Page If Semantic Mediawiki isn't powerful enough to generate at least the *makings* of a VM implementation from a rigorous (but still reasonably human-readable spec), I don't know what is. The above main page is, of course, wrong. In almost every possible way. But anybody can help out in fixing it .... -michael turner On Fri, Jun 1, 2012 at 4:03 PM, Joe Armstrong wrote: > On Wed, May 30, 2012 at 11:39 PM, james wrote: >>> If they aren't written down, they WILL be forgotten >>> sooner or later. >> >> >> I would also like to suggest that the act of writing down a design forces a >> degree of rigour that is hard to achieve otherwise. ?I have several times >> thought that I had considered a design from 'every angle', only to find that >> when I *write it down*, some of the angles were imperial and some were >> metric, and some were just smoke and mirrors. > > I agree 100% - when I get *really stuck* - I write out in clear > English what I want my program to do. > > I often start programming without a clear idea of what my program is to do - I > "think" I have a clear idea but as the program evolves I find that the > original idea was > unclear. Writing down in another language (ie English, instead of > Erlang or C or whatever) forces > a cognitive shift in my brain - I suspect that actually *different* > parts of the brain are involved. > I can also feel the ideas moving around in my head - feel is too > strong a word here. > > There is also another strange phenomena - until a program is > completed, the problem > is not really solved. But as soon as the program is completed in its > entirety, a strange thing > happens - then, and not sooner, I realise that a better solution was possible. > > Why is this? - I suspect a different part of the brain is involved. > Some part of the brain > says "move on problem completed" and as soon as that happens the "move > on" part of the brain > realises that the solution you have just arrived at was flawed - so > throw it all away and start again. > > Knuth, he of the wise ways, ?says this rewriting process should be > repeated seven times. > > In organisations this causes problems. Just about when the system is > to be delivered > you fix the final bug and then realize that it was > all wrong and needs a total rewrite, and you start the rewrite the day > before delivery ... > I have never met a project manager on the planet who understands this > (Project mangers are from Venus, > Programmers are from Mars) > > I also believe in working in a distraction-free environment (no phone, > email, twitters etc) you have to > listen very carefully to catch the small fleeting thoughts in your > brain. When I solve sudoku I > get instant flashes where I see where numbers are to be placed - but > these are fleeting and easy to miss. > I suspect my right-brain instantly sees a solution and tries to tell > my left-brain, but since I wasn't listening I missed it. > > Programming is actually a form of "applied thinking" where you can > actually test the results of the > thought process. Since all the real work takes place inside the brain, > a brain-friendly environment is > essential. I once worked in an open-plan office, after a while I > noticed my (programmer) productivity > dropped to zero and that all my programs were written at home. > > I could wax on but i'm supposed to be writing a book, but have been > distracted by the Erlang mailing > list ... > > Cheers > > /Joe > > > >> >> When something is written down, the handwaving stops and the engineering >> starts. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Fri Jun 1 09:36:12 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 1 Jun 2012 09:36:12 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <4FC86EE6.3050509@gmail.com> References: <4FC86EE6.3050509@gmail.com> Message-ID: On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: > On 01/06/2012 08:22, Joe Armstrong wrote: >> >> I think it would be a good idea if those of us who might qualify as being >> "knowledgeable persons" could take the trouble to occasionally answer >> ?questions from apparent beginners that have not been answered. >> >> After a threshold of say 3 days - then at least *somebody* could attempt >> an answer so that beginner to Erlang don't feel unloved when they >> reach to this list. > > > It would be easier to do this if there was a some kind of separate feed for > mail that hasn't been answered in the last 3 days or whatever. Otherwise you > loose a lot of cycles actually keeping track of who has asked what, and I > suspect that is the time killer for a lot of people, rather than not having > time to answer basic questions. > > I have *no idea* about managing mailing list software, so I'm not even sure > if that idea is feasible. Pretty easy to write an erlang program to do this and run it once a day with crontab The mails can be obtained programmatically from http://erlang.org/pipermail/erlang-questions/ The rest is 'left as an exercise to the reader' /Joe >> >> Of course, some extremely knowledgable and well-known people ask >> extremely difficult >> questions here - and zero replies should probably >> be taken as a sign that "nobody knows the answer" >> >> We could even have volunteers among those who posses the knowledge to >> answer >> all unanswered beginners questions. >> >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ivan@REDACTED Fri Jun 1 09:58:39 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 1 Jun 2012 08:58:39 +0100 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: Message-ID: <134D4B77-B317-4B30-8530-F07BEF30A077@llaisdy.com> > The reason for this is that a) I haven't time and b) I hope somebody > else will answer > and then I won't need to. I think the psychologists call this "deferred responsibility", even if the "somebody else" is your future self. Waiting three days is part of the problem. How about: you are only allowed to reply to exciting expert-level posts after you have responded to at least one less exciting beginner's question. Beginners' questions can often be answered by other beginners of course (so it's not just the experts who can try harder). That might mean some dodgy answers but it might also mean some common misconceptions are brought into the open and resolved. I gave a dodgy answer myself recently. Best wishes Ivan Sent from my iPhone On 1 Jun 2012, at 08:22, Joe Armstrong wrote: > Hi, > > We had an interesting discussion at the EUC. > > One relatively new user observed that several new converts to Erlang > had asked questions in this > list and not gotten a reply. > > I took a quick look at the list statistics and indeed this it true. > > What usually happens is - I read the list every day and see a lot of > questions that I know > how to answer - but I don't immediately answer. > > The reason for this is that a) I haven't time and b) I hope somebody > else will answer > and then I won't need to. > > I think it would be a good idea if those of us who might qualify as being > "knowledgeable persons" could take the trouble to occasionally answer > questions from apparent beginners that have not been answered. > > After a threshold of say 3 days - then at least *somebody* could attempt > an answer so that beginner to Erlang don't feel unloved when they > reach to this list. > > Of course, some extremely knowledgable and well-known people ask > extremely difficult > questions here - and zero replies should probably > be taken as a sign that "nobody knows the answer" > > We could even have volunteers among those who posses the knowledge to answer > all unanswered beginners questions. > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Fri Jun 1 10:01:44 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 01 Jun 2012 09:01:44 +0100 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <4FC876E8.8040008@gmail.com> On 01/06/2012 08:36, Joe Armstrong wrote: > On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >> I have *no idea* about managing mailing list software, so I'm not even sure >> if that idea is feasible. > Pretty easy to write an erlang program to do this and run it once a > day with crontab > > The mails can be obtained programmatically from > > http://erlang.org/pipermail/erlang-questions/ > > The rest is 'left as an exercise to the reader' Ivan is right. I had been giving myself 'deferred responsibility' for writing this simple program.... Time to pull up socks and answer more questions I guess. Thanks for giving us all a nudge Joe. From rickard@REDACTED Fri Jun 1 10:05:45 2012 From: rickard@REDACTED (Rickard Green) Date: Fri, 1 Jun 2012 10:05:45 +0200 Subject: [erlang-questions] About Erlang SMP scheduler In-Reply-To: References: <0F4D9489-7755-4FD2-8831-E4BEE2A8C8F1@gmail.com> <2FF9F129-C96C-48C2-BFE9-223F591D624B@gmail.com> <0773DEE9-E2B7-440A-814E-33C55A472A7C@gmail.com> Message-ID: <4FC877D9.7020004@erlang.org> On 05/31/2012 01:29 PM, Emilio De Camargo Francesquini wrote: > Hello, > >> check_balance() sets up >> migration paths and migration limits that are used in order to balance >> the load between schedulers. Without balancing, processes with the >> same assigned priority can get very different effective priorities. > > I have some questions relating to that. > > First, after the decision to rebalance has been made, what are the > criteria to select which processes from each queue will be migrated? > The last ones in the queue, the first, at random, ...? > Each priority is balanced separately (while considering the effects different priorities have on each other). Run-queue lengths on a specific priority is the most important factor. Within each priority all processes are considered equal. We migrate away processes from an overloaded run queue at (sort of) both ends. New processes will be redirected instead of enqueued and underloaded schedulers will pick from the front of the queue. > Second, how are these migration paths and limits defined? Does the > definition of the migration paths take into consideration the > architecture (SMP, multisocket SMP, NUMA, ...) of the machine? If we > consider that the lifespan of the majority of the process is quite > short, and if we are running on a NUMA machine for example, migrating > processes at random to another numa-node might hurt the performance > more than letting the execution queues unbalanced. > The primary purpose of the load balancing is to preserve the soft real time properties. That is, a load balancing decision that hurt performance isn't necessarily unwanted. We do, of course, not want to unnecessarily hurt performance. A performance improvement is, of course, preferred. The CPU topology is currently taken into account in a situation where schedulers run out of work. In this case we try to move work towards schedulers with low ids. Scheduler bind type will determine the effect of this compaction of work (for more info see cpu topology and scheduler bind type in the documentation erl). We have on the todo-list to take the CPU topology into account also in other load situations. Most likely some kind of hierarchical balancing strategy based on the CPU topology. However, at the time being there are other things ahead of this on the todo-list, but at some point this will be introduced. Regards, Rickard > Thanks! > > Best Regards > > Emilio Francesquini > > 2012/5/8 Rickard Green: >> The work stealing is there to quickly distribute work between >> schedulers, but it doesn't balance the load. check_balance() sets up >> migration paths and migration limits that are used in order to balance >> the load between schedulers. Without balancing, processes with the >> same assigned priority can get very different effective priorities. >> >> Example: With 4 schedulers and 400 cpu bound processes executing the >> same code, each scheduler will eventually end up with 100 processes to >> manage. That is, all processes will have the same access to the cpu. >> If you disable check_balance(), you may end up with three schedulers >> only having one process each, and one scheduler with 397 processes. >> These 397 processes will effectively have a much lower priority than >> the other three processes. >> >> Regards, >> Rickard Green, Erlang/OTP, Ericsson AB >> >> 2012/5/2 Siyao Zheng(???): >>> Hi, >>> >>> Yes, that's true. But the "try_steal_task()" afterwards is also controlled >>> by ERTS_SMP macro. It doesn't make any sense to talk about workload balance >>> outside multi-processor environment. I'm just wondering why "check_balance" >>> is needed while there exists work stealing(try_steal_task). >>> >>> Cheers >>> >>> On May 2, 2012, at 12:04 PM, xu yu wrote: >>> >>> Hi, >>> >>> "check_balance" controlled by "ifdef ERTS_SMP", so... >>> >>> >>> 2012/4/27 "Siyao Zheng(???)" >>>> >>>> Hi, >>>> >>>> schedule(), in which check_balance() is only called. >>>> >>>> In schedule(), when check_balance_reds reaches zero, scheduler decides to >>>> "check_balance". But later in schedule(), when scheduler finds run queue >>>> being empty, it will try to steal task from other schedulers (by calling >>>> try_steal_task()), which is also a load balance mechanism. I'm just >>>> wondering what the purpose of check_balance(). >>>> >>>> Cheers >>>> Siyao >>>> >>>> On Apr 27, 2012, at 9:40 PM, Zabrane Mickael wrote: >>>> >>>>> Hi Siyao, >>>>> >>>>> Which "check_balance" you've commented out? >>>>> >>>>> Here is what I found so far: >>>>> >>>>> $ find /otp_src_R15B01 -type f | xargs grep check_balance | grep -v >>>>> matches >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: int >>>>> forced_check_balance; >>>>> >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:check_balance(ErtsRunQueue >>>>> *c_rq) >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:# error >>>>> check_balance() assumes ERTS_MAX_PROCESS< (1<< 27) >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> c_rq->check_balance_reds = INT_MAX; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> c_rq->check_balance_reds = INT_MAX; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: * >>>>> check_balance() is never called in more threads >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: forced = >>>>> balance_info.forced_check_balance; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> balance_info.forced_check_balance = 0; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> c_rq->check_balance_reds = INT_MAX; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> rq->check_balance_reds = INT_MAX; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> balance_info.forced_check_balance = 0; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> (RQ)->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; \ >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> balance_info.forced_check_balance = 1; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> ERTS_RUNQ_IX(0)->check_balance_reds = 0; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: if >>>>> (rq->check_balance_reds<= 0) >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>>>> check_balance(rq); >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: int >>>>> check_balance_reds; >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >>>>> (RQ)->check_balance_reds -= (REDS); \ >>>>> /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >>>>> (RQ)->check_balance_reds -= (REDS); >>>>> [...] >>>>> >>>>> For OTP Team: is this dangerous ? >>>>> >>>>> Regards, >>>>> Zabrane >>>>> >>>>> >>>>> On Apr 27, 2012, at 1:23 PM, Siyao Zheng(???) wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> In SMP version of Erlang, every scheduler periodically call >>>>>> "check_balance()" to check load balance among all schedulers, then it might >>>>>> migrate some processes to balance load, or shut down some schedulers with >>>>>> low load. Does anyone know why scheduler should do this? check_balance() is >>>>>> quite a big one, and it has to lock every run queue when it checks the run >>>>>> queue. I think it's quite a big cost here. The work stealing at each >>>>>> schedule step afterwards actually does work load balance very well. Actually >>>>>> after I comment out the check_balance() step and run BigBang and Hackbench >>>>>> benchmarks, they are really faster! >>>>>> >>>>>> So, I wonder what is the purpose of check_balance() step here, if it is >>>>>> related to scalability on many core processors or something, and if there is >>>>>> any other benchmark I can run to prove its usefulness. >>>>>> >>>>>> Cheers! >>>>>> >>>>>> Siyao >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Rickard Green, Erlang/OTP, Ericsson AB. From watson.timothy@REDACTED Fri Jun 1 10:09:19 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 01 Jun 2012 09:09:19 +0100 Subject: [erlang-questions] Default install prefix In-Reply-To: <4FBFFA18.4080700@gmail.com> References: <4FBFFA18.4080700@gmail.com> Message-ID: <4FC878AF.5030002@gmail.com> On 25/05/2012 22:31, Forrest Aldrich wrote: > My configure flags are pretty straightforward: > > ./configure --prefix=/usr --enable-m64-build --without-javac > --with-odbc=/usr > > WHY does erlang insist on doing this for the installation? > > /usr/lib/erlang/bin/./erl > > No other program is going to find this in a reasonable $PATH setting. > > The ./configure --prefix=/usr should assume /usr as the root and bin/ > underneath that, just like any other program. Am I missing something > here. Sounds reasonable to me. On the other hand.... cd r13src && ./configure --prefix=/opt/erlang/versions/R13B01 --etc && make && make install cd r14src && ./configure --prefix=/opt/erlang/versions/R14B01 --etc && make && make install ...etc ln -s /opt/erlang/versions/R14B01 /opt/erlang/current export PATH="/opt/erlang/current/bin:$PATH" export ERL_TOP="opt/erlang/current" That way you can swap 'em around easily. You can also use https://github.com/spawngrid/kerl for this, or https://github.com/hyperthunk/evm (which is incredibly stupid, but therefore easy not to get confused by). -------------- next part -------------- An HTML attachment was scrubbed... URL: From omarkj@REDACTED Fri Jun 1 10:20:59 2012 From: omarkj@REDACTED (=?UTF-8?Q?=C3=93mar_Kjartan_Yasin?=) Date: Fri, 1 Jun 2012 08:20:59 +0000 Subject: [erlang-questions] Spawnfest 2012! Message-ID: Hi everybody, We're happy to announce that second Spawnfest is scheduled for 7th and 8th of July 2012. Our committee is currently working on drumming up interest and sponsors/prizes. We've already secured some of the big name (and you could say the biggest name!) in the Erlang community as judges. We're very excited about this opportunity to show the world what Erlang/OTP is capable of! The contest is not limiting entrants to web applications, in fact, we'll be having nominees in different categories. You can register your team at http://spawnfest.com/ Feel free to post any question on the mailing list, we will be happy to address those. If you're interested in becoming a sponsor, please email me at omarkj at spawnfest.com; it could be a tremendous opportunity to expose your business. From bgustavsson@REDACTED Fri Jun 1 10:28:12 2012 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 1 Jun 2012 10:28:12 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 1:55 PM, Jos? Valim wrote: > I thought Erlang would optimize the ++ operator when the left side is known > at compile time. > > For example, if the compiler sees the following outside of a function > signature: > > ? ? "Foo" ++ Bar > > It could rewrite it as: > > ? ? [$F, $o, $o | assert_list(Bar)] > The '++' operator does not verify that the second argument is a list. Therefore, the compiler will rewrite the first expression to simply: [$F,$o,$o|Bar] > However, I ran some benchmarks and it seems the optimization does not happen > (on R15B). > With a local dummy implementation of assert_list(Bar), I got that the first > format is 50% slower than the second one. Have you made sure that you run each test in a newly spawned process? Do you run the test for long enough time? If you have set up your benchmark environment properly, and your second example is still faster, I can only assume that the code sits better in cache. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jose.valim@REDACTED Fri Jun 1 10:46:28 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Fri, 1 Jun 2012 10:46:28 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: References: Message-ID: > > The '++' operator does not verify that the second argument is a list. > Therefore, the compiler will rewrite the first expression to simply: > > [$F,$o,$o|Bar] > Yes, thanks. For some weird reason I thought Erlang restricted the right side to be a list in such cases. > Have you made sure that you run each test in a newly spawned process? Do you run the test for long enough time? > Running in a newly spawned process did the trick, the results are now consistent. I was already running it 1000 times, here is the code (without using spawn): https://www.refheap.com/paste/2954 I assume having a "clean" heap (reducing the chance of garbage collection) is the reason why it is a good idea to run benchmarks in a newly spawned process? Thanks Bj?rn! -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Jun 1 11:47:01 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Jun 2012 11:47:01 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <20120601094701.GA21944@erix.ericsson.se> On Fri, Jun 01, 2012 at 09:36:12AM +0200, Joe Armstrong wrote: > On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: > > On 01/06/2012 08:22, Joe Armstrong wrote: > >> > >> I think it would be a good idea if those of us who might qualify as being > >> "knowledgeable persons" could take the trouble to occasionally answer > >> ?questions from apparent beginners that have not been answered. > >> > >> After a threshold of say 3 days - then at least *somebody* could attempt > >> an answer so that beginner to Erlang don't feel unloved when they > >> reach to this list. > > > > > > It would be easier to do this if there was a some kind of separate feed for > > mail that hasn't been answered in the last 3 days or whatever. Otherwise you > > loose a lot of cycles actually keeping track of who has asked what, and I > > suspect that is the time killer for a lot of people, rather than not having > > time to answer basic questions. > > > > I have *no idea* about managing mailing list software, so I'm not even sure > > if that idea is feasible. > Normally this is solved by the noob bumping the post after a while... > Pretty easy to write an erlang program to do this and run it once a > day with crontab > > The mails can be obtained programmatically from > > http://erlang.org/pipermail/erlang-questions/ > > The rest is 'left as an exercise to the reader' > > /Joe > > >> > >> Of course, some extremely knowledgable and well-known people ask > >> extremely difficult > >> questions here - and zero replies should probably > >> be taken as a sign that "nobody knows the answer" > >> > >> We could even have volunteers among those who posses the knowledge to > >> answer > >> all unanswered beginners questions. > >> > >> > >> Cheers > >> > >> /Joe > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Fri Jun 1 11:47:05 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 01 Jun 2012 11:47:05 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: Message-ID: <4FC88F99.8000000@ninenines.eu> I often recommend to people I talk to to go on IRC (#erlang or #erlounge on freenode) to ask seemingly beginners questions, and if nobody can answer them there to try on the mailing list. Many regulars hang out there and answer questions as they come. On 06/01/2012 09:22 AM, Joe Armstrong wrote: > Hi, > > We had an interesting discussion at the EUC. > > One relatively new user observed that several new converts to Erlang > had asked questions in this > list and not gotten a reply. > > I took a quick look at the list statistics and indeed this it true. > > What usually happens is - I read the list every day and see a lot of > questions that I know > how to answer - but I don't immediately answer. > > The reason for this is that a) I haven't time and b) I hope somebody > else will answer > and then I won't need to. > > I think it would be a good idea if those of us who might qualify as being > "knowledgeable persons" could take the trouble to occasionally answer > questions from apparent beginners that have not been answered. > > After a threshold of say 3 days - then at least *somebody* could attempt > an answer so that beginner to Erlang don't feel unloved when they > reach to this list. > > Of course, some extremely knowledgable and well-known people ask > extremely difficult > questions here - and zero replies should probably > be taken as a sign that "nobody knows the answer" > > We could even have volunteers among those who posses the knowledge to answer > all unanswered beginners questions. > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From watson.timothy@REDACTED Fri Jun 1 11:48:01 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 01 Jun 2012 10:48:01 +0100 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <20120601094701.GA21944@erix.ericsson.se> References: <4FC86EE6.3050509@gmail.com> <20120601094701.GA21944@erix.ericsson.se> Message-ID: <4FC88FD1.1080901@gmail.com> On 01/06/2012 10:47, Raimo Niskanen wrote: > On Fri, Jun 01, 2012 at 09:36:12AM +0200, Joe Armstrong wrote: >> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >>> On 01/06/2012 08:22, Joe Armstrong wrote: >>>> I think it would be a good idea if those of us who might qualify as being >>>> "knowledgeable persons" could take the trouble to occasionally answer >>>> questions from apparent beginners that have not been answered. >>>> >>>> After a threshold of say 3 days - then at least *somebody* could attempt >>>> an answer so that beginner to Erlang don't feel unloved when they >>>> reach to this list. >>> >>> It would be easier to do this if there was a some kind of separate feed for >>> mail that hasn't been answered in the last 3 days or whatever. Otherwise you >>> loose a lot of cycles actually keeping track of who has asked what, and I >>> suspect that is the time killer for a lot of people, rather than not having >>> time to answer basic questions. >>> >>> I have *no idea* about managing mailing list software, so I'm not even sure >>> if that idea is feasible. > Normally this is solved by the noob bumping the post after a while... lol that's true enough. :) From wallentin.dahlberg@REDACTED Fri Jun 1 12:07:33 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Fri, 1 Jun 2012 12:07:33 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: References: Message-ID: <460492439358073642@unknownmsgid> 1 jun 2012 kl. 10:46 skrev "Jos? Valim" : The '++' operator does not verify that the second argument is a list. > Therefore, the compiler will rewrite the first expression to simply: > > [$F,$o,$o|Bar] > Yes, thanks. For some weird reason I thought Erlang restricted the right side to be a list in such cases. > Have you made sure that you run each test in a newly spawned process? Do you run the test for long enough time? > Running in a newly spawned process did the trick, the results are now consistent. I was already running it 1000 times, here is the code (without using spawn): https://www.refheap.com/paste/2954 I assume having a "clean" heap (reducing the chance of garbage collection) is the reason why it is a good idea to run benchmarks in a newly spawned process? If you want to vary one parameter in a test, make sure all other parameters stays the same otherwise you have no idea what you are measuring! Code loading, heaps, position of mars .. Everything has to be taken into account. Thanks Bj?rn! _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav@REDACTED Fri Jun 1 15:09:09 2012 From: gustav@REDACTED (Gustav Simonsson) Date: Fri, 1 Jun 2012 15:09:09 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: <460492439358073642@unknownmsgid> References: <460492439358073642@unknownmsgid> Message-ID: <4FC8BEF5.3000006@erlang.org> On 2012-06-01 12:07, Bj?rn-Egil Dahlberg wrote: > > 1 jun 2012 kl. 10:46 skrev "Jos? Valim" >: > >> The '++' operator does not verify that the second argument is a list. >> Therefore, the compiler will rewrite the first expression to simply: >> >> [$F,$o,$o|Bar] >> >> >> Yes, thanks. For some weird reason I thought Erlang restricted the >> right side to be a list in such cases. >> >> Have you made sure that you run each test in a newly spawned >> process? >> >> Do you run the test for long enough time? >> >> >> Running in a newly spawned process did the trick, the results are now >> consistent. >> I was already running it 1000 times, here is the code (without using >> spawn): >> >> https://www.refheap.com/paste/2954 >> >> I assume having a "clean" heap (reducing the chance of garbage >> collection) is the reason why it is a good idea to run benchmarks in >> a newly spawned process? > > If you want to vary one parameter in a test, make sure all other > parameters stays the same otherwise you have no idea what you are > measuring! > > Code loading, heaps, position of mars .. Everything has to be taken > into account. The effect of Mars magnetic field on that of Earth, even if Earth's aphelion and the perihelion of Mars were to coincide and the interplanetary distance were to be at its theoretical minimum, is negligible compared to the normal variations in Earth's magnetic field. Therefore it's very unlikely that the relative position of Mars would affect the probability of a memory bitflip occurring due to cosmic radiation and thus affecting the benchmark. Regards, Gustav Simonsson Erlang/OTP team > > >> >> Thanks Bj?rn! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Fri Jun 1 15:22:16 2012 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Fri, 1 Jun 2012 15:22:16 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: <4FC8BEF5.3000006@erlang.org> References: <460492439358073642@unknownmsgid> <4FC8BEF5.3000006@erlang.org> Message-ID: <4FC8C208.7080200@erlang.org> On 2012-06-01 15:09, Gustav Simonsson wrote: > On 2012-06-01 12:07, Bj?rn-Egil Dahlberg wrote: >> >> 1 jun 2012 kl. 10:46 skrev "Jos? Valim" > >: >> >>> The '++' operator does not verify that the second argument is a >>> list. >>> Therefore, the compiler will rewrite the first expression to simply: >>> >>> [$F,$o,$o|Bar] >>> >>> >>> Yes, thanks. For some weird reason I thought Erlang restricted the >>> right side to be a list in such cases. >>> >>> Have you made sure that you run each test in a newly spawned >>> process? >>> >>> Do you run the test for long enough time? >>> >>> >>> Running in a newly spawned process did the trick, the results are >>> now consistent. >>> I was already running it 1000 times, here is the code (without using >>> spawn): >>> >>> https://www.refheap.com/paste/2954 >>> >>> I assume having a "clean" heap (reducing the chance of garbage >>> collection) is the reason why it is a good idea to run benchmarks in >>> a newly spawned process? >> >> If you want to vary one parameter in a test, make sure all other >> parameters stays the same otherwise you have no idea what you are >> measuring! >> >> Code loading, heaps, position of mars .. Everything has to be taken >> into account. > > The effect of Mars magnetic field on that of Earth, even if Earth's > aphelion and the perihelion of Mars were to coincide and the > interplanetary distance were to be at its theoretical minimum, is > negligible compared to the normal variations in Earth's magnetic field. > > Therefore it's very unlikely that the relative position of Mars would > affect the probability of a memory bitflip occurring due to cosmic > radiation and thus affecting the benchmark. My point, in case you missed it, is that you can't disregard something out of hand. You have to be sure. In physics you might be familiar to similar topics from dimensional analysis, i.e. how to come up with a model and how to disregard useless parameters. All I'm saying is that you should be sure of what you are measuring. =) > > Regards, > Gustav Simonsson > Erlang/OTP team > >> >> >>> >>> Thanks Bj?rn! >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Fri Jun 1 16:03:43 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Fri, 1 Jun 2012 16:03:43 +0200 Subject: [erlang-questions] covertool for eunit without CT? Is that possible? Message-ID: Hi, I want code coverage data to be visible on Jenkins CI using cobertura plugin. This part of HOWTO[1] is unclear: """ Configure cover to export data. Sample cover.spec for Common Test: {incl_app, app0, details}. {export, "all.coverdata"}. """ Of course, after ./rebar eunit, no relevant XML files are created (neither all.coverdata is). How do I configure covertool without CT? Thanks, Motiejus Jak?tys [1]: https://github.com/idubrov/covertool From ulf@REDACTED Fri Jun 1 17:21:27 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 1 Jun 2012 17:21:27 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: Message-ID: <72D3E159-07B2-454D-9E95-BF7463116CCF@feuerlabs.com> On 1 Jun 2012, at 09:22, Joe Armstrong wrote: > What usually happens is - I read the list every day and see a lot of > questions that I know > how to answer - but I don't immediately answer. > > The reason for this is that a) I haven't time and b) I hope somebody > else will answer > and then I won't need to. I also tend to defer answering to a lot of questions, for the above reasons, but also because I think answering questions is a great way for people to grow into the community, and I don't want to deny anyone that opportunity. :) In my experience, this works pretty well, though. A few questions may fall through the cracks, but most of the time, I think people try to answer questions quickly and in a helpful tone. And as Raimo said, the best way to get an answer is to resend, if a few days have passed - most likely, the question is then no longer on anyone's radar, but a polite reminder will usually do the trick. BR, Ulf Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From robert.virding@REDACTED Fri Jun 1 18:34:58 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 01 Jun 2012 17:34:58 +0100 (BST) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: Message-ID: Or to make it more concise: When you don't understand something you can't write the documentation, while when you understand you see no need to write the documentation. So you never write the documentation. Robert ----- Original Message ----- > Excellent! This should be codified into Armstrong's Law of > Technology > Obfuscation! > > > On Mon, May 7, 2012 at 4:47 AM, Joe Armstrong > wrote: > > I think it works like this: > > > > ? ?1) first you don't understand how the X works (X=Beam, JVM, X11, > > ... you name it) > > ? ?2) You struggle - and think - ?google and have a hot bath > > ? ?3) Eureka - bath flows over > > ? ?4) Now you can understand it - and you can also remember why you > > could not understand it > > ? ?5) Now it's easy you understand it > > ? ?6) You see no reason to document it since it's obvious > > > > Round about 4) there is a small window of opportunity to explain to > > other people how it works. > > Once you get to 6) it's very difficult to remember what it felt > > like > > at point 2) and consequently difficult > > to write decent documentation. > > > > /Joe > > > > > > On Mon, May 7, 2012 at 11:27 AM, Michael Turner > > wrote: > >> "Actually, I don't think such docs are all _that_ crucial -- who > >> really needs to know, except a small number of VM implementors?" > >> > >> Aren't Erlang's chances of greater mindshare improved by making it > >> easier to become a VM implementor? I doubt very much that Java > >> would > >> be where it is today had it not been for clear VM specification. > >> That's not to say that Erlang should follow in all of Java's > >> footsteps, even if it could. But I have to say I was a boggled to > >> learn that you can't find out what the VM opcodes mean without > >> reading > >> the source (and maybe not even then, if the source contains bugs > >> vis-a-vis some idealized machine model.) > >> > >> -michael turner > >> > >> > >> > >> On Mon, May 7, 2012 at 5:46 PM, Thomas Lindgren > >> wrote: > >>> > >>> > >>> > >>>>________________________________ > >>>> From: Jonathan Coveney > >>>>To: erlang-questions@REDACTED > >>>>Sent: Monday, May 7, 2012 8:39 AM > >>>>Subject: [erlang-questions] Is there a good source for > >>>>documentation on BEAM? > >>>> > >>>> > >>>>This question seems to come up now and again, and it's surprising > >>>>to me that a crucial part of the documentation isn't better > >>>>documented. Is there a reason that it is the case? Is the reason > >>>>that there is no VM spec to give the devs the flexibility to > >>>>change the intermediate layer without having to worry about > >>>>backwards compatibility to the degree that Java does? > >>> > >>> > >>> Actually, I don't think such docs are all _that_ crucial -- who > >>> really needs to know, except a small number of VM implementors? > >>> (And they should read the source to get at all the goodies.) But > >>> perhaps someone on the list might be moved to do a tutorial > >>> presentation on an Erlang Factory or something? > >>> > >>> (By the way, I too assume not doing it is to avoid getting bogged > >>> down into minutiae.) > >>> > >>> If you want to learn more about some of the intellectual roots, > >>> try these: > >>> http://wambook.sourceforge.net/ > >>> > >>> http://dl.acm.org/citation.cfm?id=188051 > >>> > >>> > >>> Best regards, > >>> Thomas > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmercer@REDACTED Fri Jun 1 18:44:42 2012 From: dmercer@REDACTED (David Mercer) Date: Fri, 1 Jun 2012 11:44:42 -0500 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <72D3E159-07B2-454D-9E95-BF7463116CCF@feuerlabs.com> References: <72D3E159-07B2-454D-9E95-BF7463116CCF@feuerlabs.com> Message-ID: <00e101cd4015$d8e3a1a0$8aaae4e0$@com> On Friday, June 01, 2012, Ulf Wiger wrote: > On 1 Jun 2012, at 09:22, Joe Armstrong wrote: > > > What usually happens is - I read the list every day and see a lot of > > questions that I know > > how to answer - but I don't immediately answer. > > > > The reason for this is that a) I haven't time and b) I hope somebody > > else will answer > > and then I won't need to. > > I also tend to defer answering to a lot of questions, for the above > reasons, > but also because I think answering questions is a great way for people > to grow into the community, and I don't want to deny anyone that > opportunity. :) And see I'm sitting around thinking I could answer that, but Ulf would probably give a better, more complete answer, so I'll leave it to him. :-) > In my experience, this works pretty well, though. A few questions may > fall > through the cracks, but most of the time, I think people try to answer > questions quickly and in a helpful tone. > > And as Raimo said, the best way to get an answer is to resend, if a > few days have passed - most likely, the question is then no longer on > anyone's radar, but a polite reminder will usually do the trick. That sounds a bit whiney, though. My way of bumping a question is to add some information to it, such as, "Since I asked this question 3 days ago, I have tried X, Y, and Z, and none of those worked. Anyone got any other suggestions?" Cheers, DBM From andre@REDACTED Fri Jun 1 19:32:49 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Fri, 1 Jun 2012 19:32:49 +0200 Subject: [erlang-questions] covertool for eunit without CT? Is that possible? In-Reply-To: References: Message-ID: Hi Motiejus I did this a while ago and unfortunately I have no access for that CI-server anymore so not sure if my information are (still) valid. I have installed the covertool rebar plugin as described in https://github.com/idubrov/covertool. Then you must configure the plugin in your rebar.config file. So: """ plugins, [rebar_covertool]}. % Enable covertool rebar plugin {covertool_eunit, "eunit.coverage.xml"}. % Output report file name {covertool_ct, {"ct.coverdata", "ct.coverage.xml"}}. % Source file name, output report file name """ I guess you are referring to line 3 where you configure the covertool for common test, however if you are not using common test at all you can omit that line. I think that is it. Cheers Andre On 1 June 2012 16:03, Motiejus Jak?tys wrote: > Hi, > I want code coverage data to be visible on Jenkins CI using cobertura > plugin. This part of HOWTO[1] is unclear: > """ > Configure cover to export data. Sample cover.spec for Common Test: > {incl_app, app0, details}. > {export, "all.coverdata"}. > """ > > Of course, after ./rebar eunit, no relevant XML files are created > (neither all.coverdata is). > > How do I configure covertool without CT? > > Thanks, > Motiejus Jak?tys > > [1]: https://github.com/idubrov/covertool > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pmundkur.erlang@REDACTED Fri Jun 1 21:23:08 2012 From: pmundkur.erlang@REDACTED (Prashanth Mundkur) Date: Fri, 1 Jun 2012 12:23:08 -0700 Subject: [erlang-questions] Erlang and OCaml In-Reply-To: References: Message-ID: <20120601192308.GA3407@damage.nokiapaloalto.com> Peter Mechlenborg wrote: > Hi all > > Do any of you have experience with combining Erlang and OCaml? > > I think this could be a great match, Erlang for coordination and fault > tolerance and OCaml for number crunching, but I have only found > http://code.google.com/p/erlocaml/ searching, and this project seems > dead...? A different style of combining Erlang and OCaml is supported by Disco[1], a map-reduce style framework for clustered computing. The cluster management and job scheduling is done by Erlang, and the actual data processing can be done in OCaml[2]. [1] http://discoproject.org/ [2] https://github.com/discoproject/odisco --prashanth From thomas.elsgaard@REDACTED Fri Jun 1 21:53:09 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Fri, 1 Jun 2012 21:53:09 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: Message-ID: > Hi, > > We had an interesting discussion at the EUC. > > One relatively new user observed that several new converts to Erlang > had asked questions in this > list and not gotten a reply. > > I took a quick look at the list statistics and indeed this it true. > > What usually happens is - I read the list every day and see a lot of > questions that I know > how to answer - but I don't immediately answer. > > The reason for this is that a) I haven't time and b) I hope somebody > else will answer > and then I won't need to. > > I think it would be a good idea if those of us who might qualify as being > "knowledgeable persons" could take the trouble to occasionally answer > questions from apparent beginners that have not been answered. > > Hi Joe and everybody else.. I will just add, that the extremely helpfull community around erlang was one of the main reasons for why i have sticked around since i participated in my first EUC in 2005. The erlang mail list and the community around it is just as important as the language itself, so keep up the great work! Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Jun 1 23:25:52 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 1 Jun 2012 17:25:52 -0400 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: That depends. As a personal rule, I see 3 general levels of documentation: 1. Beginner guide/tutorial 2. Reference Manual 3. Architecture 1. The first type is the one that scratches the surface, gives a few usage examples, and generally helps newcomers and users. It explains why you'd use the library/module/thing, and directs in how to do so. It holds the user's hands so they can quickly solve whatever problem they have. The raw version of this is either reading test cases or Open Source code that uses the product. 2. The reference manual is a listing made to help those who already know the item. It details everything but gives few explanations. It can be used to deepen one's knowledge, but won't do any hand holding, or very little. EDocs and the general javadoc-style stuff fit this well. The raw version of it is 'read the source, Luke'. It works for experienced users, and is of nearly no use to others. 3. Architecture is the doc that explains why the app was built the way it is, a view of how it works from 10,000 feet. It shows how to understand the app were you to dive in its source. This kind of doc explains the rationale of some choices made, and is especially helpful to developers or contributors to your code, so that they do not undo future plans, respect trade off decisions you made, and know where to dive in to extend things without causing headaches to anyone. I know of no raw version of this; it's experience and in-depth knowledge of the product's life. Writing this doc is often vital to raise the quality and relevance of contributions received. ---- In my opinion, you don't have great documentation until you covered the 3 aspects. You can have poor, decent, or good docs without all of them, but greatness requires to cover all bases. A self-proclaimed complete book, in-depth tutorial, or course, should touch all 3, for example. Understanding your code well and making it self-explanatory sadly rarely helps with more than one out of 3 points. On Jun 1, 2012 12:35 PM, "Robert Virding" < robert.virding@REDACTED> wrote: -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Sat Jun 2 01:47:22 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 02 Jun 2012 01:47:22 +0200 Subject: [erlang-questions] Understanding recursion In-Reply-To: References: Message-ID: <4FC9548A.3080901@eonblast.com> I second the premise. But the 0-case is missing. You basically proposed: grok(understood) -> ok. grok(Prereq) -> grok(Prereq). Which is a dead loop for Prereq /= understood, as the jokes it inspired demonstrate. There is hope though, I was allowed to hear the sense of wonder first hand when a CS student discovered that on machine code level recursions really become iterations. I am pretty sure the newest generation of programmers have recursion hardwired now, for skipping all the low level stuff and starting out with functional languages (not kidding). Henning On 5/19/11 2:58 PM, Joe Armstrong wrote: > I think how we teach recursion is wrong. > > Understanding recursion is really easy, > all you have to do is understand recursion. > > /Joe > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From arun11299@REDACTED Sat Jun 2 07:18:44 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Sat, 2 Jun 2012 10:48:44 +0530 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: References: Message-ID: A Remider. :) (As per Joe's suggestion on unanswered questions) -Arun On Thu, May 31, 2012 at 10:34 PM, Arun Muralidharan wrote: > Hi Folks! > Is there any way to detect if the packet sent by issuing the statement > "gen_udp:send" is really sent or got dropped because of buffer overflow in > the kernel . > I am using UDP socket with one of the option set as active once and both > send and receive buffer set to 2MB. > > > Thanks, > -A > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paperless@REDACTED Sat Jun 2 07:29:17 2012 From: paperless@REDACTED (Tim McNamara) Date: Sat, 2 Jun 2012 17:29:17 +1200 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: References: Message-ID: Others can correct me. AFAIK UDP is 'fire and forget'. You don't get the opportunity to check whether the message was received, or even if there was someone there to receive it. On 2 June 2012 17:18, Arun Muralidharan wrote: > A Remider. :) (As per Joe's suggestion on unanswered questions) > -Arun > > > On Thu, May 31, 2012 at 10:34 PM, Arun Muralidharan > wrote: >> >> Hi Folks! >> Is there any way to detect if the packet sent by issuing the statement >> "gen_udp:send" is really sent or got dropped because of buffer overflow in >> the kernel . >> I am using UDP socket with one of the option set as active once and both >> send and receive buffer set to 2MB. >> >> >> Thanks, >> -A > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From torben.lehoff@REDACTED Sat Jun 2 09:39:10 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Sat, 02 Jun 2012 09:39:10 +0200 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: References: Message-ID: <4FC9C31E.60209@gmail.com> Tim, you are correct about the 'fire and forget', but that is only for the part outside your own machine. The original poster wanted to know if the message really got sent or got dropped in his own machine. Which actually poses a question in my head: what does it matter where the message was lost? The only reason I can think of right now is to control load on your own kernel and slow down if you are pushing your machine too hard. Perhaps the original poster can clarify? Cheers, Torben On 02/06/2012 07:29, Tim McNamara wrote: > Others can correct me. > > AFAIK UDP is 'fire and forget'. You don't get the opportunity to check > whether the message was received, or even if there was someone there > to receive it. > > > On 2 June 2012 17:18, Arun Muralidharan wrote: >> A Remider. :) (As per Joe's suggestion on unanswered questions) >> -Arun >> >> >> On Thu, May 31, 2012 at 10:34 PM, Arun Muralidharan >> wrote: >>> Hi Folks! >>> Is there any way to detect if the packet sent by issuing the statement >>> "gen_udp:send" is really sent or got dropped because of buffer overflow in >>> the kernel . >>> I am using UDP socket with one of the option set as active once and both >>> send and receive buffer set to 2MB. >>> >>> >>> Thanks, >>> -A >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From arun11299@REDACTED Sat Jun 2 10:47:20 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Sat, 2 Jun 2012 14:17:20 +0530 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: <4FC9C31E.60209@gmail.com> References: <4FC9C31E.60209@gmail.com> Message-ID: Yes Torben, you are correct about the question. But how would I check if my application is pushing the machine too hard. As UDP is "send and forget" as said by Tim, so does the performance of UDP send really comes down to the performance of kernel/machine not the application ? Thanks -A On Sat, Jun 2, 2012 at 1:09 PM, Torben Hoffmann wrote: > Tim, you are correct about the 'fire and forget', but that is only for the > part outside your own machine. > > The original poster wanted to know if the message really got sent or got > dropped in his own machine. > Which actually poses a question in my head: what does it matter where the > message was lost? > The only reason I can think of right now is to control load on your own > kernel and slow down if you are pushing your machine too hard. > Perhaps the original poster can clarify? > > Cheers, > Torben > > > On 02/06/2012 07:29, Tim McNamara wrote: > >> Others can correct me. >> >> AFAIK UDP is 'fire and forget'. You don't get the opportunity to check >> whether the message was received, or even if there was someone there >> to receive it. >> >> >> On 2 June 2012 17:18, Arun Muralidharan> >> wrote: >> >>> A Remider. :) (As per Joe's suggestion on unanswered questions) >>> -Arun >>> >>> >>> On Thu, May 31, 2012 at 10:34 PM, Arun Muralidharan>> com > >>> wrote: >>> >>>> Hi Folks! >>>> Is there any way to detect if the packet sent by issuing the statement >>>> "gen_udp:send" is really sent or got dropped because of buffer overflow >>>> in >>>> the kernel . >>>> I am using UDP socket with one of the option set as active once and both >>>> send and receive buffer set to 2MB. >>>> >>>> >>>> Thanks, >>>> -A >>>> >>> >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >>> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > -- > http://www.linkedin.com/in/**torbenhoffmann > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sat Jun 2 11:14:35 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 2 Jun 2012 12:14:35 +0300 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: References: <4FC9C31E.60209@gmail.com> Message-ID: <5911B8EB-B8BE-43F0-97D7-F226BA52F707@gmail.com> Hi, There are no cross platform & erlangish way to detect it. On linux platform kernel maintains various counters including drop ratio at /proc//net/udp I do not see a reason why your application should look into that metric. Since, udp is not reliable by nature only e2e flow control make sense. If you are just worried about a local instance then I would propose to use bandwidth as a flow control metric. In this respect a inet:getstat(Socket) is enough http://erldocs.com/R14B02/kernel/inet.html?i=10&search=inet:get#getstat/2 - Dmitry On Jun 2, 2012, at 11:47 AM, Arun Muralidharan wrote: > Yes Torben, you are correct about the question. But how would I check if my application is pushing the machine too hard. As UDP is "send and forget" as said by Tim, so does the performance of UDP send really comes down to the performance of kernel/machine not the application ? > > Thanks > -A > > > On Sat, Jun 2, 2012 at 1:09 PM, Torben Hoffmann wrote: > Tim, you are correct about the 'fire and forget', but that is only for the part outside your own machine. > > The original poster wanted to know if the message really got sent or got dropped in his own machine. > Which actually poses a question in my head: what does it matter where the message was lost? > The only reason I can think of right now is to control load on your own kernel and slow down if you are pushing your machine too hard. > Perhaps the original poster can clarify? > > Cheers, > Torben > > > On 02/06/2012 07:29, Tim McNamara wrote: > Others can correct me. > > AFAIK UDP is 'fire and forget'. You don't get the opportunity to check > whether the message was received, or even if there was someone there > to receive it. > > > On 2 June 2012 17:18, Arun Muralidharan wrote: > A Remider. :) (As per Joe's suggestion on unanswered questions) > -Arun > > > On Thu, May 31, 2012 at 10:34 PM, Arun Muralidharan > wrote: > Hi Folks! > Is there any way to detect if the packet sent by issuing the statement > "gen_udp:send" is really sent or got dropped because of buffer overflow in > the kernel . > I am using UDP socket with one of the option set as active once and both > send and receive buffer set to 2MB. > > > Thanks, > -A > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- > http://www.linkedin.com/in/torbenhoffmann > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wojtek@REDACTED Sat Jun 2 12:24:09 2012 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Sat, 02 Jun 2012 12:24:09 +0200 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: <4FC9C31E.60209@gmail.com> References: <4FC9C31E.60209@gmail.com> Message-ID: <4FC9E9C9.6020206@power.com.pl> On 2012-06-02 09:39, Torben Hoffmann wrote: > Tim, you are correct about the 'fire and forget', but that is only for > the part outside your own machine. Really? I always thought that in 'fire and forget' you can forget exactly because the missile is (almost) guaranteed to reach its target. Silly me. From mattevans123@REDACTED Sat Jun 2 19:24:50 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 2 Jun 2012 13:24:50 -0400 Subject: [erlang-questions] A visual tour of Erlang Message-ID: Sorry if this is a repost. This was posted on Reddit. Although it probably doesn't have any "real" uses (yet) I think it's a very nice way to model an Erlang VM and your applications. It got praise from our Java developers when I ran it on our Erlang application at work. Cudos to Kresten Krab for writing this application. https://www.youtube.com/watch?v=lHoWfeNuAN8 https://github.com/krestenkrab/erlubi -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Sat Jun 2 21:02:10 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 02 Jun 2012 21:02:10 +0200 Subject: [erlang-questions] Trace-Driven Development Message-ID: <4FCA6332.5000109@eonblast.com> Should there be any interest I would like to discuss tracing and if it may be more valuable for the actor model than unit tests. A couple of days ago, during Jesper Louis Andersen's very enlightening debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether showing visual traces /first/ may be a good approach to /teaching/ Erlang, book-side. I would like to propose again that trace-driven development may possibly be /the/ way to teach andalso /program/ Erlang. In the instance, we were talking about visual traces, like these: http://www.erlang.org/doc/apps/et/coffee_order.png I mean it in the slightly unrealistic way that most people probably appreciate test-driven programming as a great idea but seldom execute it as dogmatically as preached. But I realize that what I /actually/ do when programming a new app, or when looking for truly nasty bugs, is almost always pretty 'trace-driven'. Writing traces into the program has become a routine step for me when coding medium complicated stuff that uses supervisors, monitors and multiple process that come to live at various times throughout the lifetime of an Erlang (OTP) application -- and most of all, during the /start up/. And I have always found surprises, which process comes to live when. I think Jesper related a similar story how they found tons of bugs that no-one had realized were there when using tracing (not sure any more.) I believe that the main aggravating factor is that you don't construct an OTP application from scratch that often. You'll have to look things up again every next time. My guess is that this is true for the vast majority of Erlang programmers. Of course, for the ultimate expert, it may be hard to grasp what I am even talking about because it's all so 'obvious'. But what I really find myself currently doing, before writing tests, is writing traces. I also haven't found out how to plan out and write meaningful stateful tests up front, before I even made it to architect the application. My concrete order is: writing first, get to compile, put in traces, try to run, debug. I am so sure meanwhile that I will need the traces that I don't wait anymore until I have gotten lost in searching for a bug and not even knowing what happens before the crash and what parts of the program may have silently died and what other parts may have waited for which other part. These things will be obvious for expert Erlang programmers who have set up applications time and again. But few people using Erlang for productivity reasons will /ever/ come into that position. And for finding bugs: that's exactly the moment when reality doesn't follow your fantasy and a reality check in the form of a trace is one of the first tools to turn to. Is it not? So trace-driven development sounds like it could be a useful recommendation. And if it is, a new best practice of waterfalling your application into existence could maybe emerge, similar to what Joe describes in his book (if I remember well) how he almost always starts out. I learned a lot in Jesper's tutorial. For example that because Erlang and OTP keep growing, there are now three distinct mechanisms, with respective modules and function calls, to trace and debug. And despite me using traces a lot, I hadn't used any of them, for the wrong reasons it turned out. (And for productivity, a simple io:format can beat getting tied up into using more powerful but less simple minded options.) So I am asking this out of honest ignorance: Instead of tracing, is there a way to write meaningful, concise tests that are meant or at least good for checking the sequence of process communications? In other words, are there better alternatives to follow traces with the naked eye? I do not mean a heavy construct, or an also-possible use, but a test-package made for that and/or a useful practice that someone is in fact applying? And it need not be the visual traces as the image linked to above. A ping-pong output in the terminal may do (if not skewered by delays or re-sequences or contentions in io:format.) For trace-driven development, what concrete procedure with which concrete calls and parameters to erl can be a best practice for starting out? I am sure it' s 'obvious' for many, but at least for me, not. What could be the hello world case for a trace-driven approach? And finally, how do /you/ approach setting up a new app, finding a bug -- how much tracing does everyone use in these things? And is anyone also using io:format instead of the OTP goodness, if so, for a good reason? Best, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Sat Jun 2 21:20:53 2012 From: lukas@REDACTED (Lukas Larsson) Date: Sat, 2 Jun 2012 21:20:53 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: Message-ID: If you find that useful you might want to checkout https://github.com/psyeugenic/fgraph as well. On Sat, Jun 2, 2012 at 7:24 PM, Matthew Evans wrote: > Sorry if this is a repost. This was posted on Reddit. Although it probably > doesn't have any "real" uses (yet) I think it's a very nice way to model an > Erlang VM and your applications. > > It got praise from our Java developers when I ran it on our Erlang > application at work. > > Cudos to Kresten Krab for writing this application. > > https://www.youtube.com/watch?v=lHoWfeNuAN8 > > https://github.com/krestenkrab/erlubi > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From desired.mta@REDACTED Sat Jun 2 21:29:47 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Sat, 2 Jun 2012 21:29:47 +0200 Subject: [erlang-questions] covertool for eunit without CT? Is that possible? In-Reply-To: References: Message-ID: On Fri, Jun 1, 2012 at 8:22 PM, Andr? Graf wrote: > Have you cover enabled? > {cover_enabled, true}. Hi Andr?, Yes, I did. I set up a very tiny sample project for this: https://github.com/Motiejus/covertest Testing on Erlang R15B and rebar binary is copied from your project. Thanks for your help, Motiejus > On 1 June 2012 20:08, Motiejus Jak?tys wrote: >> On Fri, Jun 1, 2012 at 7:32 PM, Andr? Graf wrote: >>> Hi Motiejus >>> >>> I did this a while ago and unfortunately I have no access for that >>> CI-server anymore so not sure if my information are (still) valid. I >>> have installed the covertool rebar plugin as described in >>> https://github.com/idubrov/covertool. Then you must configure the >>> plugin in your rebar.config file. >>> So: >>> """ >>> plugins, [rebar_covertool]}. % Enable covertool rebar plugin >>> {covertool_eunit, "eunit.coverage.xml"}. % Output report file name >>> {covertool_ct, {"ct.coverdata", "ct.coverage.xml"}}. % Source file >>> name, output report file name >>> """ >>> I guess you are referring to line 3 where you configure the covertool >>> for common test, however if you are not using common test at all you >>> can omit that line. >>> I think that is it. >> >> Hi, >> thanks for the answer. Tried removing that line as well. However, >> eunit.coverage.xml does not appear anywhere after running ./rebar >> eunit >> >> Any more ideas? -- Motiejus Jak?tys From jose.valim@REDACTED Sat Jun 2 22:38:57 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Sat, 2 Jun 2012 22:38:57 +0200 Subject: [erlang-questions] Broken User Guide links in the docs Message-ID: Browsing through Erlang docs I have noticed that many User's Guide links are broken. For example: http://www.erlang.org/doc/apps/observer/index.html Clicking on the User's Guide leads to a blank page giving a false impression the page does not exist: http://www.erlang.org/doc/apps/observer/users_guide.html However, the page *does* exist and is located at: http://www.erlang.org/doc/apps/observer/observer_ug.html This also happens with other applications, for example: http://www.erlang.org/doc/apps/percept/percept_ug.html If someone provides some guidance, I can try to provide a patch to fix the broken links. On another note, I have noticed that there isn't a page (or at least I could not find one) that lists all OTP applications and provides a quick summary about them. I believe such pages would be interesting because very often I discover new tools in OTP (today was etop) and having a listing may make the discovery of such tools easier. If there is an interest, I can work on such page myself and submit a patch. Finally, I am frequently amazed by Erlang docs quality, so I would like to thank those who worked/work on maintaing it. :) * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Jun 2 23:00:27 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Sat, 2 Jun 2012 23:00:27 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: Message-ID: 2012/6/2 Lukas Larsson > If you find that useful you might want to checkout > https://github.com/psyeugenic/fgraph as well. > After that reminder I felt I had to write a README. *commit, push* There, I fixed it. > > On Sat, Jun 2, 2012 at 7:24 PM, Matthew Evans > wrote: > > Sorry if this is a repost. This was posted on Reddit. Although it > probably > > doesn't have any "real" uses (yet) I think it's a very nice way to model > an > > Erlang VM and your applications. > > > > It got praise from our Java developers when I ran it on our Erlang > > application at work. > > > > Cudos to Kresten Krab for writing this application. > > > > https://www.youtube.com/watch?v=lHoWfeNuAN8 > > > > https://github.com/krestenkrab/erlubi > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sat Jun 2 23:04:53 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Sat, 2 Jun 2012 23:04:53 +0200 Subject: [erlang-questions] Broken User Guide links in the docs In-Reply-To: References: Message-ID: Sorry, after sending the e-mail I realized that the link is not actually broken. Clicking the User's Guide link actually changes the menu on the bottom-left corner: http://www.erlang.org/doc/apps/observer/users_guide.html Since I had my browser half-opened, I could not see that it changed at all. That said, maybe we could change those blank pages to actually show the index or available sections in the guide? * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Sat, Jun 2, 2012 at 10:38 PM, Jos? Valim wrote: > Browsing through Erlang docs I have noticed that many User's Guide links > are broken. For example: > > http://www.erlang.org/doc/apps/observer/index.html > > Clicking on the User's Guide leads to a blank page giving a false > impression the page does not exist: > > http://www.erlang.org/doc/apps/observer/users_guide.html > > However, the page *does* exist and is located at: > > http://www.erlang.org/doc/apps/observer/observer_ug.html > > This also happens with other applications, for example: > > http://www.erlang.org/doc/apps/percept/percept_ug.html > > If someone provides some guidance, I can try to provide a patch to fix the > broken links. > > On another note, I have noticed that there isn't a page (or at least I > could not find one) that lists all OTP applications and provides a quick > summary about them. I believe such pages would be interesting because very > often I discover new tools in OTP (today was etop) > and having a listing may make the discovery of such tools easier. If there > is an interest, I can work on such page myself and submit a patch. > > Finally, I am frequently amazed by Erlang docs quality, so I would like to > thank those who worked/work on maintaing it. :) > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From atmb4u@REDACTED Sun Jun 3 06:41:42 2012 From: atmb4u@REDACTED (Anoop Thomas Mathew) Date: Sun, 3 Jun 2012 10:11:42 +0530 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: Message-ID: Hi All, That was a cool video demonstration of erlang processes. Nice work. Kudos to Kresten Krab. It aroused a question in me that, isn't erlang creating way too many processes, and why is that so? If someone can spread some light into this topic, and give a comprehensive explanation about HOW and WHY, that would be great. Thanks, Anoop Thomas Mathew atm ___ Life is short, Live it hard. On 3 June 2012 02:30, Bj?rn-Egil Dahlberg wrote: > > > 2012/6/2 Lukas Larsson > >> If you find that useful you might want to checkout >> https://github.com/psyeugenic/fgraph as well. >> > > After that reminder I felt I had to write a README. > > *commit, push* > > There, I fixed it. > >> >> On Sat, Jun 2, 2012 at 7:24 PM, Matthew Evans >> wrote: >> > Sorry if this is a repost. This was posted on Reddit. Although it >> probably >> > doesn't have any "real" uses (yet) I think it's a very nice way to >> model an >> > Erlang VM and your applications. >> > >> > It got praise from our Java developers when I ran it on our Erlang >> > application at work. >> > >> > Cudos to Kresten Krab for writing this application. >> > >> > https://www.youtube.com/watch?v=lHoWfeNuAN8 >> > >> > https://github.com/krestenkrab/erlubi >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Sun Jun 3 07:39:58 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Sun, 3 Jun 2012 07:39:58 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: Message-ID: <64E3EC38-2C96-4073-8672-D941649EF18E@gmail.com> Hi Anoop, Erlang has its own VM, so all those processes are very lightweight compared to OS processes. The basic unit of computation in Erlang is a process with its own memory space and you can only communicate with a process by sending it a message. This set-up is necessary if you want to build a fault tolerant system - the last ingredience is the ability to link and monitor processes. Two linked processes will die if either of them dies. A monitor will be notified if the process it monitors die. These simple mechanisms is what allows Erlang to work so well. Hope this clarifies things a bit for you. Cheers, Torben Sent from my iPhone On 03/06/2012, at 06.41, Anoop Thomas Mathew wrote: > Hi All, > > That was a cool video demonstration of erlang processes. Nice work. Kudos to Kresten Krab. > It aroused a question in me that, isn't erlang creating way too many processes, and why is that so? > If someone can spread some light into this topic, and give a comprehensive explanation about HOW and WHY, that would be great. > > Thanks, > Anoop Thomas Mathew > > atm > ___ > Life is short, Live it hard. > > > > > On 3 June 2012 02:30, Bj?rn-Egil Dahlberg wrote: > > > 2012/6/2 Lukas Larsson > If you find that useful you might want to checkout > https://github.com/psyeugenic/fgraph as well. > > After that reminder I felt I had to write a README. > > *commit, push* > > There, I fixed it. > > On Sat, Jun 2, 2012 at 7:24 PM, Matthew Evans wrote: > > Sorry if this is a repost. This was posted on Reddit. Although it probably > > doesn't have any "real" uses (yet) I think it's a very nice way to model an > > Erlang VM and your applications. > > > > It got praise from our Java developers when I ran it on our Erlang > > application at work. > > > > Cudos to Kresten Krab for writing this application. > > > > https://www.youtube.com/watch?v=lHoWfeNuAN8 > > > > https://github.com/krestenkrab/erlubi > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From atmb4u@REDACTED Sun Jun 3 09:02:48 2012 From: atmb4u@REDACTED (Anoop Thomas Mathew) Date: Sun, 3 Jun 2012 12:32:48 +0530 Subject: [erlang-questions] Fundamentals of Erlang - Newbie Introductions Message-ID: Hi, Where can a newbie like me find information of such basic questions. Not a tutorial or like(http://learnyousomeerlang.com/), but about the underlying concepts of Erlang. Thanks. Anoop Thomas Mathew atm ___ Life is short, Live it hard. On 3 June 2012 11:09, Torben Hoffmann wrote: > Hi Anoop, > > Erlang has its own VM, so all those processes are very lightweight > compared to OS processes. > > The basic unit of computation in Erlang is a process with its own memory > space and you can only communicate with a process by sending it a message. > > This set-up is necessary if you want to build a fault tolerant system - > the last ingredience is the ability to link and monitor processes. Two > linked processes will die if either of them dies. A monitor will be > notified if the process it monitors die. > > These simple mechanisms is what allows Erlang to work so well. > > Hope this clarifies things a bit for you. > > Cheers, > Torben > > Thanks a lot! That was well enough to clear my doubt. > Sent from my iPhone > > On 03/06/2012, at 06.41, Anoop Thomas Mathew wrote: > > Hi All, > > That was a cool video demonstration of erlang processes. Nice work. Kudos > to Kresten Krab. > It aroused a question in me that, isn't erlang creating way too many > processes, and why is that so? > If someone can spread some light into this topic, and give a > comprehensive explanation about HOW and WHY, that would be great. > > Thanks, > Anoop Thomas Mathew > > atm > ___ > Life is short, Live it hard. > > > > > On 3 June 2012 02:30, Bj?rn-Egil Dahlberg wrote: > >> >> >> 2012/6/2 Lukas Larsson >> >>> If you find that useful you might want to checkout >>> https://github.com/psyeugenic/fgraph as well. >>> >> >> After that reminder I felt I had to write a README. >> >> *commit, push* >> >> There, I fixed it. >> >>> >>> On Sat, Jun 2, 2012 at 7:24 PM, Matthew Evans >>> wrote: >>> > Sorry if this is a repost. This was posted on Reddit. Although it >>> probably >>> > doesn't have any "real" uses (yet) I think it's a very nice way to >>> model an >>> > Erlang VM and your applications. >>> > >>> > It got praise from our Java developers when I ran it on our Erlang >>> > application at work. >>> > >>> > Cudos to Kresten Krab for writing this application. >>> > >>> > https://www.youtube.com/watch?v=lHoWfeNuAN8 >>> > >>> > https://github.com/krestenkrab/erlubi >>> > >>> > >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun Jun 3 11:11:55 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 3 Jun 2012 11:11:55 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: Message-ID: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> On 3 Jun 2012, at 06:41, Anoop Thomas Mathew wrote: > It aroused a question in me that, isn't erlang creating way too many processes, and why is that so? There is such a thing as too many processes - and too few. The key thing about a process is that it is a single thread of control. If you try to handle multiple, interdependent and interleaving tasks in one single thread of control, bad things tend to happen, just as it tends to do when one human being tries to handle too many things at the same time. But many threads of control need managing/coordination. Just as human organizations become unwieldy when there are more people than the problem calls for, too many processes in a concurrency- oriented program tends to become a problem in itself. The trick, then, is to figure out how many processes the problem calls for, and to use that many - no more, no less. Drawing inspiration from how people solve coordination problems, including how people deal with uncertainty and inconsistencies, can be a great help when programming Erlang. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sun Jun 3 12:17:36 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 3 Jun 2012 11:17:36 +0100 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> Message-ID: <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> On 3 Jun 2012, at 10:11, Ulf Wiger wrote: > Drawing inspiration from how people solve coordination problems, > including how people deal with uncertainty and inconsistencies, > can be a great help when programming Erlang. Very good suggestion. What do they know of Erlang, who only Erlang know? (with apologies to Kipling) Ivan From matti.oinas@REDACTED Sun Jun 3 13:04:23 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Sun, 03 Jun 2012 14:04:23 +0300 Subject: [erlang-questions] Fundamentals of Erlang - Newbie Introductions In-Reply-To: References: Message-ID: <4FCB44B7.30006@gmail.com> On 06/03/2012 10:02 AM, Anoop Thomas Mathew wrote: > Hi, > > Where can a newbie like me find information of such basic questions. > Not a tutorial or like(http://learnyousomeerlang.com/), but about the > underlying concepts of Erlang. > > Thanks. > Anoop Thomas Mathew > > atm Hi, Google is your friend here. I started learning Erlang by reading Joe's book and after that I read the Erlang and OTP in action. Every time I faced a new term I wasn't really familiar with I googled it. Many times the route leads you to wikipedia and there is usually links pointing you to the source of the information. Sometimes in google the keywords don't provide results good enough and you don't know enough from the subject so you could add more relevant keywords. In this situation I usually use the google ability to filter search results by filetype. Just add filetype:pdf to your search and you get back only results in PDF and these are most of the time some thesis and scientific articles about the subject. Adding thesis or article keyword to your search also helps most of the times. That way I was able to find almost all the information I ever wanted to know about some subject. Matti Oinas -------------- next part -------------- An HTML attachment was scrubbed... URL: From nycholas@REDACTED Sun Jun 3 17:05:47 2012 From: nycholas@REDACTED (Nycholas de Oliveira e Oliveira) Date: Sun, 3 Jun 2012 12:05:47 -0300 Subject: [erlang-questions] Fundamentals of Erlang - Newbie Introductions In-Reply-To: References: Message-ID: Hi, On Sun, Jun 3, 2012 at 4:02 AM, Anoop Thomas Mathew wrote: > Where can a newbie like me find information of such basic questions. > Not a tutorial or like(http://learnyousomeerlang.com/), but about the > underlying concepts of Erlang. > I like these: http://www.tryerlang.org/ http://trigonakis.com/blog/series/introduction-to-erlang/ http://www.erlang.org/doc/ I read these, the Joe's book and I'm finishing the http://learnyousomeerlang.com/, I believe that will soon be doing some projects in Erlang. Along the way I put some code in http://code.google.com/p/nycholas/source/browse/code/erlang/. Have fun, ;-). -- So long and good luck, Nycholas de Oliveira e Oliveira. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g9414002.pccu.edu.tw@REDACTED Sun Jun 3 17:14:23 2012 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 3 Jun 2012 23:14:23 +0800 Subject: [erlang-questions] Understanding recursion In-Reply-To: References: Message-ID: No. There shell be a rule like "All receivers of the mail replies to all." :D On Thu, May 19, 2011 at 9:22 PM, Paul Barry wrote: > So... if I click on "Reply All", then this email message will itself > recur, right? ;-) > > On 19 May 2011 13:58, Joe Armstrong wrote: > > I think how we teach recursion is wrong. > > > > Understanding recursion is really easy, > > all you have to do is understand recursion. > > > > /Joe > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sun Jun 3 19:48:32 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 03 Jun 2012 18:48:32 +0100 (BST) Subject: [erlang-questions] clarify: init_per_suite/0 in Common Test In-Reply-To: Message-ID: <26a1afd3-0c36-4107-b20c-710a97e91639@knuth> There is a suite/0 for function, described here: http://www.erlang.org/doc/man/common_test.html#Module:suite-0 http://www.erlang.org/doc/apps/common_test/write_test_chapter.html#id69736 which is an info function for the whole test suite. It is the suite version of the /0 function. Robert ----- Original Message ----- > Hi, > > I suppose Bengt's asking if you can use an info function also with > init_per_suite. Answer is yes, and it's documented in the user's > guide, chapter 4.14: > > http://www.erlang.org/doc/apps/common_test/write_test_chapter.html#id71329 > > We should add info also in the reference manual that what goes for > the > testcase info function, "Testcase()", is valid for config functions > init/end_per_suite/group too. > > Thanks! > > Best regards, > Peter > > Ericsson AB, Erlang/OTP > > > 2012/5/31, Abdul Fattah Mahran : > > Hi Bengt, > > It is documented > > http://www.erlang.org/doc/man/common_test.html#Module:init_per_suite-1 > > > > Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | > > {skip_and_save,Reason,SaveConfig} > > > > Types: > > Config = NewConfig = SaveConfig = [{Key,Value}] > > Key = atom() > > Value = term() > > Reason = term() > > > > OPTIONAL > > > > This configuration function is called as the first function in the > > suite. > > It typically contains initializations which are common for all test > > cases > > in the suite, and which shall only be done once. The Config > > parameter is > > the configuration data which can be modified here. Whatever is > > returned > > from this function is given as Config to all configuration > > functions and > > test cases in the suite. If {skip,Reason} is returned, all test > > cases in > > the suite will be skipped and Reason printed in the overview log > > for the > > suite. > > > > For information on save_config and skip_and_save, please see > > Dependencies > > between Test Cases and > > Suitesin > > the User's Guide. > > > > > > Thanks, > > > > best regards, > > > > > > > > > > On Thu, May 31, 2012 at 8:41 AM, Bengt Kleberg > > wrote: > > > >> Greetings, > >> > >> There is no Module:init_per_suite/0 documented for Common Test > >> test > >> suites. Should it be? > >> > >> > >> bengt > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > -- > > Thanks > > Best Regards, > > Abd El-Fattah Mahran > > http://www.linkedin.com/in/abdoomahran > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Sun Jun 3 19:52:21 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 03 Jun 2012 18:52:21 +0100 (BST) Subject: [erlang-questions] StringIO? In-Reply-To: Message-ID: An alternative would be to write a string_io_server which could do read/write requests against a string and use it as file descriptor in i/o calls. It is well defined and not that difficult. Robert ----- Original Message ----- > Thank you Bob, > that looks like it's exactly what I need. > Thank you, > r. > On Wed, May 30, 2012 at 5:25 PM, Bob Ippolito < bob@REDACTED > > wrote: > > It's undocumented, but there's the ram_file module: > > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl > > > -bob > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Sun Jun 3 20:01:08 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Sun, 03 Jun 2012 19:01:08 +0100 Subject: [erlang-questions] Understanding recursion In-Reply-To: References: Message-ID: <4FCBA664.6090306@erlang-solutions.com> But once you've received three copies of this thread, stop replying. On 03/06/2012 16:14, ??? (Yau-Hsien Huang) wrote: > No. There shell be a rule like "All receivers of the mail replies to > all." :D > > On Thu, May 19, 2011 at 9:22 PM, Paul Barry > > wrote: > > So... if I click on "Reply All", then this email message will itself > recur, right? ;-) > > On 19 May 2011 13:58, Joe Armstrong > wrote: > > I think how we teach recursion is wrong. > > > > Understanding recursion is really easy, > > all you have to do is understand recursion. > > > > /Joe > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie, e: > paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, > Ireland. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > > Best Regards. > > --- Y-H. H. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Sun Jun 3 23:17:33 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Sun, 3 Jun 2012 23:17:33 +0200 Subject: [erlang-questions] covertool for eunit without CT? Is that possible? In-Reply-To: References: Message-ID: On Sat, Jun 2, 2012 at 9:29 PM, Motiejus Jak?tys wrote: > On Fri, Jun 1, 2012 at 8:22 PM, Andr? Graf wrote: >> Have you cover enabled? >> {cover_enabled, true}. > > Hi Andr?, > > Yes, I did. I set up a very tiny sample project for this: > https://github.com/Motiejus/covertest > > Testing on Erlang R15B and rebar binary is copied from your project. Figured it out, it's rebar fault. If all tests pass, covertool works as expected. However, if any of tests fail while running ./rebar eunit, it halts the VM on the first opportunity and hence does not call rebar_covertool:eunit/2. I created a trivial patch for rebar and now working on a test suite which is less trivial. Will submit a patch to rebar in next couple of days. Best, Motiejus Jak?tys From marcel.meyer@REDACTED Sun Jun 3 23:38:43 2012 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Sun, 3 Jun 2012 17:38:43 -0400 Subject: [erlang-questions] Erlang and the TDS protocol Message-ID: Dear list, I am curious why there isn't a native Erlang TDS protocol implementation. Even /otp/lib wraps a C ODBC implementation using a port. Would this not be simpler and more concise using bit syntax, since the TCP datagram unpacks in 1 line of code (more for readability, of course)? It just seems like extra work: C wraps the msg, port wraps C, ODBC app wraps port etc. There is a lot of 'wrapping' going on. Any input would be much appreciated. Kind regards, Marcel -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Mon Jun 4 00:11:02 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Jun 2012 00:11:02 +0200 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> Message-ID: On Wed, May 30, 2012 at 4:56 PM, Tim Watson wrote: > > On 30 May 2012, at 15:40, Tuncer Ayaz wrote: > > > > - No need for storing (meta)data in a real vcs: > > Ok so Eric and I were trying to avoid the need for hosting, making > this completely free and community oriented. You want to go down > some hosting route, that's fair enough, but who's going to provide > the infrastructure? It worries me that this will put people off > using the system and we'll end up getting nowhere. > > Apart from that I'm fine with the idea though. > > - We should consider leveraging existing ftp mirror networks like > > distros and texlive do. For both the files and index. > > How will this work in practice? We'd have to figure out how the files can be distributed on existing mirror networks. It wouldn't be the first open source project doing that. > > - The website can be made central, but should be avoided if possible. > > What does this mean? This is sounding more and more like a system > that someone has to build and maintain. I'm *absolutely* in favour > of doing that, but someone is going to stump up the money for it. > > - Hayoo like search would be nice to have in a 2nd step. > > > > Again, this is *great* idea, but someone, somewhere, covers the cost > of hackage/hayoo - the same applies here. If done right, the website will be just a fancy interface for the index and/or archives fetched. Anyone should be able to host it, but having a single well-known address is nice to have. That hypothetical website would be a nice to have 2nd step on top of the foundation. It doesn't have to exist from the beginning. From watson.timothy@REDACTED Mon Jun 4 00:32:27 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 3 Jun 2012 23:32:27 +0100 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> Message-ID: On 3 Jun 2012, at 23:11, Tuncer Ayaz wrote: > >>> - We should consider leveraging existing ftp mirror networks like >>> distros and texlive do. For both the files and index. >> >> How will this work in practice? > > We'd have to figure out how the files can be distributed on existing > mirror networks. It wouldn't be the first open source project doing > that. > Hi - sorry for not commenting about this sooner. I agree that it's a solvable problem, as long as someone 'sponsors' the storage somewhere, as well as dealing with the mirroring. Given some kind of mirror network backed by, say, FTP - how are you going to deal with authentication and authorisation? More specifically, when I decide that I want to publish my stuff, how're you proposing that the underlying source determines that I am (1) who I claim to be and (2) have the right to publish/upload this 'stuff'. I am fully aware that various solutions exist to this problem, I'm just wondering how you envisage this being handled in a way that minimises administrative overhead - consider that mine and Eric's initial suggestion about this removes this overhead altogether, as only a repository owner (or authorised committer) can contribute patches and therefore if you trust the account then you trust the content. So how do we do this, and what overhead is there, if any? Admittedly, creating your own .deb packages, signing them and then making your repository accessible over the web isn't rocket science. How about the mirroring thing? Also, the does the index design for these solutions cater for the fact that you possibly have numerous origins publishing the same package/version? >>> - The website can be made central, but should be avoided if possible. >> >> What does this mean? This is sounding more and more like a system >> that someone has to build and maintain. I'm *absolutely* in favour >> of doing that, but someone is going to stump up the money for it. > > > >>> - Hayoo like search would be nice to have in a 2nd step. >>> >> >> Again, this is *great* idea, but someone, somewhere, covers the cost >> of hackage/hayoo - the same applies here. > > If done right, the website will be just a fancy interface for the > index and/or archives fetched. Anyone should be able to host it, but > having a single well-known address is nice to have. That hypothetical > website would be a nice to have 2nd step on top of the foundation. > It doesn't have to exist from the beginning. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Mon Jun 4 00:41:10 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 04 Jun 2012 00:41:10 +0200 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> Message-ID: <4FCBE806.5090705@eonblast.com> On 6/4/12 12:11 AM, Tuncer Ayaz wrote: > I'm*absolutely* in favour > > of doing that, but someone is going to stump up the money for it. How much traffic would you expect, very roughly, 'optimistically' == upper limit? Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Mon Jun 4 01:25:45 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Jun 2012 01:25:45 +0200 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> Message-ID: On Mon, Jun 4, 2012 at 12:32 AM, Tim Watson wrote: > Hi - sorry for not commenting about this sooner. I agree that it's a > solvable problem, as long as someone 'sponsors' the storage > somewhere, as well as dealing with the mirroring. Given some kind of > mirror network backed by, say, FTP - how are you going to deal with > authentication and authorisation? More specifically, when I decide > that I want to publish my stuff, how're you proposing that the > underlying source determines that I am (1) who I claim to be and (2) > have the right to publish/upload this 'stuff'. I am fully aware that > various solutions exist to this problem, I'm just wondering how you > envisage this being handled in a way that minimises administrative > overhead - consider that mine and Eric's initial suggestion about > this removes this overhead altogether, as only a repository owner > (or authorised committer) can contribute patches and therefore if > you trust the account then you trust the content. Which sounds somewhat like how distro maintainers work. > So how do we do this, and what overhead is there, if any? > Admittedly, creating your own .deb packages, signing them and then > making your repository accessible over the web isn't rocket science. > How about the mirroring thing? Also, the does the index design for > these solutions cater for the fact that you possibly have numerous > origins publishing the same package/version? Valid concerns which we'd have to deal with and an indicator why it might be hard to use an existing solution without adapting it. I think we're better off planning it right from the beginning and coming up with a design that makes it simple to distribute on mirror networks. From watson.timothy@REDACTED Mon Jun 4 01:27:07 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 4 Jun 2012 00:27:07 +0100 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> Message-ID: On 4 Jun 2012, at 00:25, Tuncer Ayaz wrote: > On Mon, Jun 4, 2012 at 12:32 AM, Tim Watson wrote: >> Hi - sorry for not commenting about this sooner. I agree that it's a >> solvable problem, as long as someone 'sponsors' the storage >> somewhere, as well as dealing with the mirroring. Given some kind of >> mirror network backed by, say, FTP - how are you going to deal with >> authentication and authorisation? More specifically, when I decide >> that I want to publish my stuff, how're you proposing that the >> underlying source determines that I am (1) who I claim to be and (2) >> have the right to publish/upload this 'stuff'. I am fully aware that >> various solutions exist to this problem, I'm just wondering how you >> envisage this being handled in a way that minimises administrative >> overhead - consider that mine and Eric's initial suggestion about >> this removes this overhead altogether, as only a repository owner >> (or authorised committer) can contribute patches and therefore if >> you trust the account then you trust the content. > > Which sounds somewhat like how distro maintainers work. > >> So how do we do this, and what overhead is there, if any? >> Admittedly, creating your own .deb packages, signing them and then >> making your repository accessible over the web isn't rocket science. >> How about the mirroring thing? Also, the does the index design for >> these solutions cater for the fact that you possibly have numerous >> origins publishing the same package/version? > > Valid concerns which we'd have to deal with and an indicator why it > might be hard to use an existing solution without adapting it. > Indeed. > I think we're better off planning it right from the beginning and > coming up with a design that makes it simple to distribute on mirror > networks. I completely agree. From michael.eugene.turner@REDACTED Mon Jun 4 06:52:37 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 4 Jun 2012 13:52:37 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCA6332.5000109@eonblast.com> References: <4FCA6332.5000109@eonblast.com> Message-ID: I'm a little embarrassed to be doing a "reply all" on this message, because I'm (still) such a stop-start Erlang newbie. What compensates for the mortification, however, is passages like the following, which suggest I'm hardly the only one who should be embarrassed: "Erlang tracing is a seething pile of pain that involves reasonably complex knowledge of clever ports, tracing return formats, and specialized tracing MatchSpecs (which are really their own special kind of hell). The tracing mechanism is very powerful indeed, but it can be hard to grasp." Obviously, that kind of statement has no place in the official documentation of a professional product. Oh, except that's precisely where I found it: http://www.erlang.org/doc/apps/et/et_intro.html#id62156 Now, if the formal documentation hosted by Ericsson seems determined to frighten me away, I'll oblige Ericsson and run off. That "hell" paragraph immediately followed two paragraphs that I couldn't make much sense of, so it pretty much validated my confusion and dismay at that point, while warning me that it was only going to get worse. [I wrote 5 more lines of rant here, then deleted them in the interests of being diplomatic.] I started using seq_trace. It has its own documentation problems, of course. For example: since seq_trace is an implementation of Lamport clocks, you should *say* somewhere (like, in the first paragraph, maybe?) that it's an implementation of Lamport clocks. Don't make it sound like your own invention. That's dishonest. And don't make people infer it. That's wasting people's time. Just say it. It only takes a few seconds to type "seq_trace implements Lamport clocks." Still, I found seq_trace relatively simple and usable, and I'm now doing unit testing on a module as I develop it further, based on collecting and filtering seq_trace results with a small amount of code I wrote myself. At some point, I expect to finally put the horse in front of the cart and do *trace*-driven development. But before then, I should make a decision: do I keep building ever more sophisticated match filtering on top of seq_trace, undoubtedly reinventing wheel after wheel, or do I bite the bullet and plunge into what Jayson Vantuyl describes as "hell"? It's a discouraging choice. -michael turner On Sun, Jun 3, 2012 at 4:02 AM, Henning Diedrich wrote: > Should there be any interest I would like to discuss tracing and if it may > be more valuable for the actor model than unit tests. > > A couple of days ago, during Jesper Louis Andersen's very enlightening > debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether > showing visual traces first may be a good approach to teaching Erlang, > book-side. > > I would like to propose again that trace-driven development may possibly be > the way to teach andalso program Erlang. > > In the instance, we were talking about visual traces, like these: > > http://www.erlang.org/doc/apps/et/coffee_order.png > > I mean it in the slightly unrealistic way that most people probably > appreciate test-driven programming as a great idea but seldom execute it as > dogmatically as preached. > > But I realize that what I actually do when programming a new app, or when > looking for truly nasty bugs, is almost always pretty 'trace-driven'. > Writing traces into the program has become a routine step for me when coding > medium complicated stuff that uses supervisors, monitors and multiple > process that come to live at various times throughout the lifetime of an > Erlang (OTP) application -- and most of all, during the start up. And I have > always found surprises, which process comes to live when. I think Jesper > related a similar story how they found tons of bugs that no-one had realized > were there when using tracing (not sure any more.) > > I believe that the main aggravating factor is that you don't construct an > OTP application from scratch that often. You'll have to look things up again > every next time. My guess is that this is true for the vast majority of > Erlang programmers. Of course, for the ultimate expert, it may be hard to > grasp what I am even talking about because it's all so 'obvious'. > > But what I really find myself currently doing, before writing tests, is > writing traces. I also haven't found out how to plan out and write > meaningful stateful tests up front, before I even made it to architect the > application. > > My concrete order is: writing first, get to compile, put in traces, try to > run, debug. > > I am so sure meanwhile that I will need the traces that I don't wait anymore > until I have gotten lost in searching for a bug and not even knowing what > happens before the crash and what parts of the program may have silently > died and what other parts may have waited for which other part. > > These things will be obvious for expert Erlang programmers who have set up > applications time and again. But few people using Erlang for productivity > reasons will ever come into that position. And for finding bugs: that's > exactly the moment when reality doesn't follow your fantasy and a reality > check in the form of a trace is one of the first tools to turn to. Is it > not? > > So trace-driven development sounds like it could be a useful recommendation. > And if it is, a new best practice of waterfalling your application into > existence could maybe emerge, similar to what Joe describes in his book (if > I remember well) how he almost always starts out. > > I learned a lot in Jesper's tutorial. For example that because Erlang and > OTP keep growing, there are now three distinct mechanisms, with respective > modules and function calls, to trace and debug. And despite me using traces > a lot, I hadn't used any of them, for the wrong reasons it turned out. (And > for productivity, a simple io:format can beat getting tied up into using > more powerful but less simple minded options.) > > So I am asking this out of honest ignorance: > > Instead of tracing, is there a way to write meaningful, concise tests that > are meant or at least good for checking the sequence of process > communications? In other words, are there better alternatives to follow > traces with the naked eye? I do not mean a heavy construct, or an > also-possible use, but a test-package made for that and/or a useful practice > that someone is in fact applying? > > And it need not be the visual traces as the image linked to above. A > ping-pong output in the terminal may do (if not skewered by delays or > re-sequences or contentions in io:format.) > > For trace-driven development, what concrete procedure with which concrete > calls and parameters to erl can be a best practice for starting out? I am > sure it' s 'obvious' for many, but at least for me, not. What could be the > hello world case for a trace-driven approach? > > And finally, how do you approach setting up a new app, finding a bug -- how > much tracing does everyone use in these things? And is anyone also using > io:format instead of the OTP goodness, if so, for a good reason? > > Best, > Henning > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bengt.kleberg@REDACTED Mon Jun 4 06:52:46 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 4 Jun 2012 06:52:46 +0200 Subject: [erlang-questions] StringIO? In-Reply-To: References: Message-ID: <1338785566.5664.1.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, Just to make it perfectly clear, when I said I had a string_io module, this is what it does. bengt On Sun, 2012-06-03 at 19:52 +0200, Robert Virding wrote: > An alternative would be to write a string_io_server which could do > read/write requests against a string and use it as file descriptor in > i/o calls. It is well defined and not that difficult. > > Robert > > > ______________________________________________________________________ > Thank you Bob, > > > that looks like it's exactly what I need. > > > Thank you, > > > r. > > On Wed, May 30, 2012 at 5:25 PM, Bob Ippolito > wrote: > > > It's undocumented, but there's the ram_file module: > > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl > > -bob > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From maruthavanan_s@REDACTED Mon Jun 4 06:58:11 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Mon, 4 Jun 2012 00:58:11 -0400 Subject: [erlang-questions] Predefined Registered Processes Message-ID: Hi, I was wondering do we have any predefined list of registered processes listed down? Because I did a small mistake by chance. I registered my database supervisor as odbc_sup, after which erlang did not allow application:start(odbc) throwing the below exception {error,{{already_started,<0.62.0>}, {odbc_app,start,[normal,[]]}}}=INFO REPORT==== 4-Jun-2012::10:23:24 === application: odbc exited: {{already_started,<0.62.0>},{odbc_app,start,[normal,[]]}} type: temporary Later I found that odbc library of erlang tries to register its process but failing as my database supervisor has already started with same name. after renaming it worked. So thought if we have such a place where it is listed down, it would be helpful to some like me; may be the list is big but still wanted to check it up. Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Jun 4 07:21:54 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 03 Jun 2012 22:21:54 -0700 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <4FCC45F2.2040204@gmail.com> I think your link is slightly wrong: http://www.erlang.org/doc/apps/et/et_tutorial.html#id58003 On 06/03/2012 09:52 PM, Michael Turner wrote: > I'm a little embarrassed to be doing a "reply all" on this message, > because I'm (still) such a stop-start Erlang newbie. What compensates > for the mortification, however, is passages like the following, which > suggest I'm hardly the only one who should be embarrassed: > > "Erlang tracing is a seething pile of pain that involves reasonably > complex knowledge of clever ports, tracing return formats, and > specialized tracing MatchSpecs (which are really their own special > kind of hell). The tracing mechanism is very powerful indeed, but it > can be hard to grasp." > > Obviously, that kind of statement has no place in the official > documentation of a professional product. Oh, except that's precisely > where I found it: > > http://www.erlang.org/doc/apps/et/et_intro.html#id62156 > > Now, if the formal documentation hosted by Ericsson seems determined > to frighten me away, I'll oblige Ericsson and run off. That "hell" > paragraph immediately followed two paragraphs that I couldn't make > much sense of, so it pretty much validated my confusion and dismay at > that point, while warning me that it was only going to get worse. > > [I wrote 5 more lines of rant here, then deleted them in the interests > of being diplomatic.] > > I started using seq_trace. It has its own documentation problems, of > course. For example: since seq_trace is an implementation of Lamport > clocks, you should *say* somewhere (like, in the first paragraph, > maybe?) that it's an implementation of Lamport clocks. Don't make it > sound like your own invention. That's dishonest. And don't make people > infer it. That's wasting people's time. Just say it. It only takes a > few seconds to type "seq_trace implements Lamport clocks." > > Still, I found seq_trace relatively simple and usable, and I'm now > doing unit testing on a module as I develop it further, based on > collecting and filtering seq_trace results with a small amount of code > I wrote myself. At some point, I expect to finally put the horse in > front of the cart and do *trace*-driven development. But before then, > I should make a decision: do I keep building ever more sophisticated > match filtering on top of seq_trace, undoubtedly reinventing wheel > after wheel, or do I bite the bullet and plunge into what Jayson > Vantuyl describes as "hell"? > > It's a discouraging choice. > > -michael turner > > On Sun, Jun 3, 2012 at 4:02 AM, Henning Diedrich wrote: >> Should there be any interest I would like to discuss tracing and if it may >> be more valuable for the actor model than unit tests. >> >> A couple of days ago, during Jesper Louis Andersen's very enlightening >> debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether >> showing visual traces first may be a good approach to teaching Erlang, >> book-side. >> >> I would like to propose again that trace-driven development may possibly be >> the way to teach andalso program Erlang. >> >> In the instance, we were talking about visual traces, like these: >> >> http://www.erlang.org/doc/apps/et/coffee_order.png >> >> I mean it in the slightly unrealistic way that most people probably >> appreciate test-driven programming as a great idea but seldom execute it as >> dogmatically as preached. >> >> But I realize that what I actually do when programming a new app, or when >> looking for truly nasty bugs, is almost always pretty 'trace-driven'. >> Writing traces into the program has become a routine step for me when coding >> medium complicated stuff that uses supervisors, monitors and multiple >> process that come to live at various times throughout the lifetime of an >> Erlang (OTP) application -- and most of all, during the start up. And I have >> always found surprises, which process comes to live when. I think Jesper >> related a similar story how they found tons of bugs that no-one had realized >> were there when using tracing (not sure any more.) >> >> I believe that the main aggravating factor is that you don't construct an >> OTP application from scratch that often. You'll have to look things up again >> every next time. My guess is that this is true for the vast majority of >> Erlang programmers. Of course, for the ultimate expert, it may be hard to >> grasp what I am even talking about because it's all so 'obvious'. >> >> But what I really find myself currently doing, before writing tests, is >> writing traces. I also haven't found out how to plan out and write >> meaningful stateful tests up front, before I even made it to architect the >> application. >> >> My concrete order is: writing first, get to compile, put in traces, try to >> run, debug. >> >> I am so sure meanwhile that I will need the traces that I don't wait anymore >> until I have gotten lost in searching for a bug and not even knowing what >> happens before the crash and what parts of the program may have silently >> died and what other parts may have waited for which other part. >> >> These things will be obvious for expert Erlang programmers who have set up >> applications time and again. But few people using Erlang for productivity >> reasons will ever come into that position. And for finding bugs: that's >> exactly the moment when reality doesn't follow your fantasy and a reality >> check in the form of a trace is one of the first tools to turn to. Is it >> not? >> >> So trace-driven development sounds like it could be a useful recommendation. >> And if it is, a new best practice of waterfalling your application into >> existence could maybe emerge, similar to what Joe describes in his book (if >> I remember well) how he almost always starts out. >> >> I learned a lot in Jesper's tutorial. For example that because Erlang and >> OTP keep growing, there are now three distinct mechanisms, with respective >> modules and function calls, to trace and debug. And despite me using traces >> a lot, I hadn't used any of them, for the wrong reasons it turned out. (And >> for productivity, a simple io:format can beat getting tied up into using >> more powerful but less simple minded options.) >> >> So I am asking this out of honest ignorance: >> >> Instead of tracing, is there a way to write meaningful, concise tests that >> are meant or at least good for checking the sequence of process >> communications? In other words, are there better alternatives to follow >> traces with the naked eye? I do not mean a heavy construct, or an >> also-possible use, but a test-package made for that and/or a useful practice >> that someone is in fact applying? >> >> And it need not be the visual traces as the image linked to above. A >> ping-pong output in the terminal may do (if not skewered by delays or >> re-sequences or contentions in io:format.) >> >> For trace-driven development, what concrete procedure with which concrete >> calls and parameters to erl can be a best practice for starting out? I am >> sure it' s 'obvious' for many, but at least for me, not. What could be the >> hello world case for a trace-driven approach? >> >> And finally, how do you approach setting up a new app, finding a bug -- how >> much tracing does everyone use in these things? And is anyone also using >> io:format instead of the OTP goodness, if so, for a good reason? >> >> Best, >> Henning >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From michael.eugene.turner@REDACTED Mon Jun 4 08:12:46 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 4 Jun 2012 15:12:46 +0900 Subject: [erlang-questions] Predefined Registered Processes In-Reply-To: References: Message-ID: It's hard to predict what *could* get registered after start-up - are there guarantees anywhere? But I've found a way to get by, for now. Since I'm using the process registry in a kind of hacky way at the moment, and need to know what I *haven't* registered myself, I initially get the list of what's currently registered using a call to registered/0 http://www.erlang.org/doc/reference_manual/processes.html#id82208 shortly after starting the Erlang shell. I'm not sure how a standalone app would figure out when a call to registered/0 would return the complete list -- whatever "complete" means in this case. Does anyone know? It seems like you'd have to get complete initialization of everything you use that might register a process, and maybe the only way to be sure is to keep checking against a list of names to see if the process has been registered yet. And if you already have that list .... I also qualify the process names I use, prefixing each with a period ('.'), to help avoid any future clashes. This means putting single quotes around each process-name atom, but I can live with that. It's dangerous in the long run, I hope to move to a more sensible scheme soon, but it's fine for now. -michael turner On Mon, Jun 4, 2012 at 1:58 PM, Maruthavanan Subbarayan wrote: > Hi, > > I was wondering do we have any predefined list of registered processes > listed down? > > Because I did a small mistake by chance. > > I registered my database supervisor as odbc_sup, after which erlang did not > allow application:start(odbc) throwing the below exception > > {error,{{already_started,<0.62.0>}, > ? ? ? ? {odbc_app,start,[normal,[]]}}} > =INFO REPORT==== 4-Jun-2012::10:23:24 === > ? ? application: odbc > ? ? exited: {{already_started,<0.62.0>},{odbc_app,start,[normal,[]]}} > ? ? type: temporary > > Later I found that odbc library of erlang tries to register its process but > failing as my database supervisor has already started with same name. after > renaming it worked. > > So thought if we have such a place where it is listed down, it would be > helpful to some like me; may be the list is big but still wanted to check it > up. > > Thanks, > Marutha > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Jun 4 08:52:46 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 4 Jun 2012 08:52:46 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> 4 jun 2012 kl. 06:52 skrev Michael Turner : > I'm a little embarrassed to be doing a "reply all" on this message, > because I'm (still) such a stop-start Erlang newbie. What compensates > for the mortification, however, is passages like the following, which > suggest I'm hardly the only one who should be embarrassed: > > "Erlang tracing is a seething pile of pain that involves reasonably > complex knowledge of clever ports, tracing return formats, and > specialized tracing MatchSpecs (which are really their own special > kind of hell). The tracing mechanism is very powerful indeed, but it > can be hard to grasp." > > Obviously, that kind of statement has no place in the official > documentation of a professional product. You are absolutely right about that. > I started using seq_trace. It has its own documentation problems, of > course. For example: since seq_trace is an implementation of Lamport > clocks, you should *say* somewhere (like, in the first paragraph, > maybe?) that it's an implementation of Lamport clocks. Don't make it > sound like your own invention. Actually, I believe it is more correct to say that it's based on the "forlopp trace" thst exists in Ericsson's AXE switches (using the proprietary language PLEX). Whether forlopp trace was based on Lamport clocks, I couldn't say. I don't know when it was introduced in the AXE (Lamport published his paper in 1978, the same year that the first digital AXE was taken into service), and to what extent it was informed by Lamport's work. Anyway, not noting in the docs that sequence trace mimicks AXE's forlopp trace seems forgiveable, since very few people would know what that is. :) See e.g. http://es.scribd.com/mobile/doc/83784121, starting at page 109. BR, Ulf W From torben.lehoff@REDACTED Mon Jun 4 09:33:42 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 04 Jun 2012 09:33:42 +0200 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: <4FC9E9C9.6020206@power.com.pl> References: <4FC9C31E.60209@gmail.com> <4FC9E9C9.6020206@power.com.pl> Message-ID: <4FCC64D6.2010307@gmail.com> You cannot make any assumptions about how often the missile hits. What you know for sure is that you will not be notified about the success or failure. From the Wikipedia entry on UDP: > UDP uses a simple transmission model without implicit handshaking > dialogues for providing reliability, ordering, or data integrity. > Thus, UDP provides an unreliable service and datagrams may arrive out > of order, appear duplicated, or go missing without notice. UDP assumes > that error checking and correction is either not necessary or > performed in the application, avoiding the overhead of such processing > at the network interface level. So you have to think about how to deal with this. UDP is similar to message passing in Erlang, you cannot be sure that a message is received, but unlike UDP, you can be sure that there are no duplicates and that messages arrive in order (but you cannot be guaranteed that the receiver processes them in order). Cheers, __ /orben On 02/06/2012 12:24, Wojtek Narczy?ski wrote: > On 2012-06-02 09:39, Torben Hoffmann wrote: >> Tim, you are correct about the 'fire and forget', but that is only >> for the part outside your own machine. > > Really? > > I always thought that in 'fire and forget' you can forget exactly > because the missile is (almost) guaranteed to reach its target. > > Silly me. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From avinash@REDACTED Mon Jun 4 09:37:08 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Mon, 04 Jun 2012 13:07:08 +0530 Subject: [erlang-questions] Timestamp in HH:MM:SS.mmm Message-ID: In order to bind a value of SQL-type TIMESTAMP that has granularity till milliseconds to an attribute of an object, which library call is suitable? I checked erlang:localtime/0 and erlang:now/0; former's granularity is till seconds and I am not sure about later's. Thanks From desired.mta@REDACTED Mon Jun 4 09:43:19 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 4 Jun 2012 09:43:19 +0200 Subject: [erlang-questions] Predefined Registered Processes In-Reply-To: References: Message-ID: On Mon, Jun 4, 2012 at 6:58 AM, Maruthavanan Subbarayan wrote: > Hi, > > Because I did a small mistake by chance. Does not really answer your question, but if the program gets bigger and more complex, and more advanced process register is required, I suggest having a look at pg2 or gproc. I use former and it's pretty cool. Also, *no* collisions. You can register a tuple {myodbc, supervisor} and you're safe. -- Motiejus Jak?tys From desired.mta@REDACTED Mon Jun 4 09:47:58 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 4 Jun 2012 09:47:58 +0200 Subject: [erlang-questions] Timestamp in HH:MM:SS.mmm In-Reply-To: References: Message-ID: On Mon, Jun 4, 2012 at 9:37 AM, Avinash Dhumane wrote: > In order to bind a value of SQL-type TIMESTAMP that has granularity till > milliseconds to an attribute of an object, which library call is suitable? I > checked erlang:localtime/0 and erlang:now/0; former's granularity is till > seconds and I am not sure about later's. now() -> timestamp() Types: timestamp() = {MegaSecs, Secs, MicroSecs} MegaSecs = Secs = MicroSecs = integer() >= 0 Micro seconds for erlang:now(). -- Motiejus Jak?tys From erlang@REDACTED Mon Jun 4 09:49:01 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 4 Jun 2012 09:49:01 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCA6332.5000109@eonblast.com> References: <4FCA6332.5000109@eonblast.com> Message-ID: On Sat, Jun 2, 2012 at 9:02 PM, Henning Diedrich wrote: > Should there be any interest I would like to discuss tracing and if it may > be more valuable for the actor model than unit tests. > > A couple of days ago, during Jesper Louis Andersen's very enlightening > debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether > showing visual traces first may be a good approach to teaching Erlang, > book-side. > > I would like to propose again that trace-driven development may possibly be > the way to teach andalso program Erlang. > > In the instance, we were talking about visual traces, like these: > > http://www.erlang.org/doc/apps/et/coffee_order.png These are great. If I have a complex protocol to understand I often draw these (on paper) but then throw them away after the code has been written (the throwing them away bit is wrong I know) The problem is "published code" that ends up in books git repositories does not contain all the design notes used to create the code - not is there a clue as to how the code was created. (I'm thinking about ways to bundle "research" and code toghether so that *both* get published) Message sequence charts (MSCs) are parts of SDL and UML and there are many tools (which I don't use) to make them - a quick google turned up this http://en.wikipedia.org/wiki/MscGen which is food for thought. Using the trace BIFs it should be easy (famous last words) to make a checker that checks if the observed messages correspond to allowed flows in the MSC This strikes me as a abstraction-lift over unit tests ... > > I mean it in the slightly unrealistic way that most people probably > appreciate test-driven programming as a great idea but seldom execute it as > dogmatically as preached. > > But I realize that what I actually do when programming a new app, or when > looking for truly nasty bugs, is almost always pretty 'trace-driven'. > Writing traces into the program has become a routine step for me when coding > medium complicated stuff that uses supervisors, monitors and multiple > process that come to live at various times throughout the lifetime of an > Erlang (OTP) application -- and most of all, during the start up. And I have > always found surprises, which process comes to live when. I think Jesper > related a similar story how they found tons of bugs that no-one had realized > were there when using tracing (not sure any more.) > > I believe that the main aggravating factor is that you don't construct an > OTP application from scratch that often. You'll have to look things up again > every next time. My guess is that this is true for the vast majority of > Erlang programmers. Of course, for the ultimate expert, it may be hard to > grasp what I am even talking about because it's all so 'obvious'. > I for one have to look things up a lot of the time - the difference is that I know exactly where to look. > But what I really find myself currently doing, before writing tests, is > writing traces. I also haven't found out how to plan out and write > meaningful stateful tests up front, before I even made it to architect the > application. That's what I'd do - I'd write the traces first. Possibly http://www.mcternan.me.uk/mscgen/ could be integrated with this work flow I'm not sure about testing the state. I just want to see what is in the messages > > My concrete order is: writing first, get to compile, put in traces, try to > run, debug. > > I am so sure meanwhile that I will need the traces that I don't wait anymore > until I have gotten lost in searching for a bug and not even knowing what > happens before the crash and what parts of the program may have silently > died and what other parts may have waited for which other part. > > These things will be obvious for expert Erlang programmers who have set up > applications time and again. But few people using Erlang for productivity > reasons will ever come into that position. And for finding bugs: that's > exactly the moment when reality doesn't follow your fantasy and a reality > check in the form of a trace is one of the first tools to turn to. Is it > not? > > So trace-driven development sounds like it could be a useful recommendation. > And if it is, a new best practice of waterfalling your application into > existence could maybe emerge, similar to what Joe describes in his book (if > I remember well) how he almost always starts out. > > I learned a lot in Jesper's tutorial. For example that because Erlang and > OTP keep growing, there are now three distinct mechanisms, with respective > modules and function calls, to trace and debug. And despite me using traces > a lot, I hadn't used any of them, for the wrong reasons it turned out. (And > for productivity, a simple io:format can beat getting tied up into using > more powerful but less simple minded options.) > > So I am asking this out of honest ignorance: > > Instead of tracing, is there a way to write meaningful, concise tests that > are meant or at least good for checking the sequence of process > communications? In other words, are there better alternatives to follow > traces with the naked eye? I do not mean a heavy construct, or an > also-possible use, but a test-package made for that and/or a useful practice > that someone is in fact applying? > > And it need not be the visual traces as the image linked to above. A > ping-pong output in the terminal may do (if not skewered by delays or > re-sequences or contentions in io:format.) > > For trace-driven development, what concrete procedure with which concrete > calls and parameters to erl can be a best practice for starting out? I am > sure it' s 'obvious' for many, but at least for me, not. What could be the > hello world case for a trace-driven approach? Good question. You'd have to make a mini-language to describe message sequences something like TCL expect [{send,client,server,hello}, %% client sends a hello message to server {receive,client,server,ack} %% cleint receives an ack from server ] For such a spec one could generate a PNG and test code :-) (There is a library for making PNGs well hidden in the erlang distribution code:which(egd) will find it for you) (aside: there are lots of good things in the erlang distribution which nobody knows about like egd.erl -- one of lifes mysteries is why these are not documented or promoted - I suspect lack of time and "not core businesss") > > And finally, how do you approach setting up a new app, finding a bug -- how > much tracing does everyone use in these things? And is anyone also using > io:format instead of the OTP goodness, if so, for a good reason? Absolutely - I use io:format all the time. I put the io:formats next to send and receive statements. The good reason was "I didn't know about the trace bifs" (modified truth) - I knew about them but hadn't realized how good they were until I attended Jespers (excellent) tutorial. I'd relegated traces to an obscure appendix in my book (Appendix E.3) where nobody could find it. In the next edition (Which I'm working on *now*) I will introduce tracing far far earlier Cheers /Joe > > Best, > Henning > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From michael.eugene.turner@REDACTED Mon Jun 4 09:52:08 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 4 Jun 2012 16:52:08 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> Message-ID: "(Lamport published his paper in 1978, the same year that the first digital AXE was taken into service), ...." Leaving publication delays aside: as with so many ideas that get named after a single person, Lamport clocks (in the sense of the counter-pairs used in seq_trace) are at least partially credited to the earlier efforts of others, by Lamport himself, in the original paper, and here: http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#time-clocks In particular, he credits the authors of an RFC submitted in 1975: http://merlot.tools.ietf.org/pdf/rfc677.pdf > See e.g. http://es.scribd.com/mobile/doc/83784121, starting at page 109. Yes, I've been pointed to this claim before in a previous "actually Ericsson invented that" discussion. Looking at the section closely, however, I don't see how either Forlopp Identity Registers or Forlopp Identity itself correspond to Lamport clocks. All I see is something that vaguely corresponds to seq_trace's "contamination" algorithm, together with an "error intensity count" that doesn't seem to have anything to do with Lamport clocks. Regardless of who first invented this technique, I think my point stands: the seq_trace documentation should call what it does by its proper name. Hardly anybody knows what forlopp means, whereas Lamport's paper won the "2000 PODC Influential Paper Award (later renamed the Edsger W. Dijkstra Prize in Distributed Computing)," and the "ACM SIGOPS Hall of Fame Award in 2007." So there's a classic distributed-systems paper that gives the technique its name, yet somehow, even though Kenneth Lundin agreed, all the way back in 2007, that seq_trace implements Lamport clocks http://erlang.org/pipermail/erlang-questions/2007-May/026827.html this fact somehow hasn't made it into the documentation. Everybody has to rediscover it. This is not just a waste of their time, there's also the opportunity cost for Erlang/OTP (and Ericsson) in people *not* discovering it because it's not named as such at www.erlang.org. Tracing tools building on seq_trace, exploiting the formal properties that were elucidated by Lamport and elaborated on by others, might considerably improve on what's available now. But as far as I can tell, seq_trace isn't the foundation for *anything* in OTP. -michael turner On Mon, Jun 4, 2012 at 3:52 PM, Ulf Wiger wrote: > 4 jun 2012 kl. 06:52 skrev Michael Turner : > >> I'm a little embarrassed to be doing a "reply all" on this message, >> because I'm (still) such a stop-start Erlang newbie. What compensates >> for the mortification, however, is passages like the following, which >> suggest I'm hardly the only one who should be embarrassed: >> >> "Erlang tracing is a seething pile of pain that involves reasonably >> complex knowledge of clever ports, tracing return formats, and >> specialized tracing MatchSpecs (which are really their own special >> kind of hell). The tracing mechanism is very powerful indeed, but it >> can be hard to grasp." >> >> Obviously, that kind of statement has no place in the official >> documentation of a professional product. > > You are absolutely right about that. > >> I started using seq_trace. It has its own documentation problems, of >> course. For example: since seq_trace is an implementation of Lamport >> clocks, you should *say* somewhere (like, in the first paragraph, >> maybe?) that it's an implementation of Lamport clocks. Don't make it >> sound like your own invention. > > Actually, I believe it is more correct to say that it's based on the "forlopp trace" thst exists in Ericsson's AXE switches (using the proprietary language PLEX). Whether forlopp trace was based on Lamport clocks, I couldn't say. I don't know when it was introduced in the AXE (Lamport published his paper in 1978, the same year that the first digital AXE was taken into service), and to what extent it was informed by Lamport's work. > > Anyway, not noting in the docs that sequence trace mimicks AXE's forlopp trace seems forgiveable, since very few people would know what that is. :) > > See e.g. http://es.scribd.com/mobile/doc/83784121, starting at page 109. > > BR, > Ulf W From raimo+erlang-questions@REDACTED Mon Jun 4 10:01:07 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 4 Jun 2012 10:01:07 +0200 Subject: [erlang-questions] Return status of gen_udp send In-Reply-To: <4FCC64D6.2010307@gmail.com> References: <4FC9C31E.60209@gmail.com> <4FC9E9C9.6020206@power.com.pl> <4FCC64D6.2010307@gmail.com> Message-ID: <20120604080107.GA15412@erix.ericsson.se> On Mon, Jun 04, 2012 at 09:33:42AM +0200, Torben Hoffmann wrote: > You cannot make any assumptions about how often the missile hits. > What you know for sure is that you will not be notified about the > success or failure. > > From the Wikipedia entry on UDP: > >UDP uses a simple transmission model without implicit handshaking > >dialogues for providing reliability, ordering, or data integrity. > >Thus, UDP provides an unreliable service and datagrams may arrive > >out of order, appear duplicated, or go missing without notice. UDP > >assumes that error checking and correction is either not necessary > >or performed in the application, avoiding the overhead of such > >processing at the network interface level. > So you have to think about how to deal with this. > > UDP is similar to message passing in Erlang, you cannot be sure that > a message is received, but unlike UDP, you can be sure that there It is a bit more reliable than that. If you send 3 messages and the first and third arrives you can be sure the second also did arrive in between them. So messages are not lost unless the receiver or the connection fails. And it is possible to ensure that the receiver or the connection can not fail silently using e.g erlang:monitor. > are no duplicates and that messages arrive in order (but you cannot > be guaranteed that the receiver processes them in order). > > Cheers, > __ > /orben > > > > On 02/06/2012 12:24, Wojtek Narczy?ski wrote: > >On 2012-06-02 09:39, Torben Hoffmann wrote: > >>Tim, you are correct about the 'fire and forget', but that is > >>only for the part outside your own machine. > > > >Really? > > > >I always thought that in 'fire and forget' you can forget exactly > >because the missile is (almost) guaranteed to reach its target. > > > >Silly me. > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > -- > http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From torben.lehoff@REDACTED Mon Jun 4 10:01:33 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 04 Jun 2012 10:01:33 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <4FCC6B5D.4040701@gmail.com> On 04/06/2012 09:49, Joe Armstrong wrote: > > > Message sequence charts (MSCs) are parts of SDL and UML and there are many tools > (which I don't use) to make them - a quick google turned up this > http://en.wikipedia.org/wiki/MscGen which is food for thought. If you want to create MSCs I recommend using other tools than MscGen. I have tried MscGen and I think that is not so good as PlantUML - http://plantuml.sourceforge.net/ Warning: I have not tried the latest version of MscGen, so my knowledge might be outdated. Cheers, __ /orben -- http://www.linkedin.com/in/torbenhoffmann From carlsson.richard@REDACTED Mon Jun 4 10:10:20 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 04 Jun 2012 10:10:20 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <4FCC6D6C.3030103@gmail.com> IMO, Mats Cronqvist's excellent "redbug" interface to the tracing mechanism ought to be productified (or at least documented...) and included in the standard OTP distribution. That would make Erlang's tracing mechanisms something that a beginner could use from day one, rather than being considered an advanced technique. Today it lives in obscurity as part of his "eper" tools: https://github.com/massemanet/eper /Richard PS. As I'm the one who actually meets him every day at work, I could take it upon myself to pester him until it gets properly documented, but I think OTP should start taking a look at redbug and give their opinion on what might be needed for including it in the distribution. From gleber.p@REDACTED Mon Jun 4 10:29:39 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 4 Jun 2012 10:29:39 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCC6D6C.3030103@gmail.com> References: <4FCA6332.5000109@eonblast.com> <4FCC6D6C.3030103@gmail.com> Message-ID: On Mon, Jun 4, 2012 at 10:10 AM, Richard Carlsson wrote: > IMO, Mats Cronqvist's excellent "redbug" interface to the tracing mechanism > ought to be productified (or at least documented...) and included in the > standard OTP distribution. +10000 From robert.virding@REDACTED Mon Jun 4 10:49:55 2012 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 04 Jun 2012 09:49:55 +0100 (BST) Subject: [erlang-questions] StringIO? In-Reply-To: <1338785566.5664.1.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: <0abdb979-9267-4dc2-a20d-1829dcb494ea@knuth> Do you have a link to it? Robert ----- Original Message ----- > Greetings, > > Just to make it perfectly clear, when I said I had a string_io > module, > this is what it does. > > > bengt > > On Sun, 2012-06-03 at 19:52 +0200, Robert Virding wrote: > > An alternative would be to write a string_io_server which could do > > read/write requests against a string and use it as file descriptor > > in > > i/o calls. It is well defined and not that difficult. > > > > Robert > > > > > > ______________________________________________________________________ > > Thank you Bob, > > > > > > that looks like it's exactly what I need. > > > > > > Thank you, > > > > > > r. > > > > On Wed, May 30, 2012 at 5:25 PM, Bob Ippolito > > > > wrote: > > > > > > It's undocumented, but there's the ram_file module: > > > > > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl > > > > -bob > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Jun 4 12:07:16 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 4 Jun 2012 12:07:16 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> Message-ID: <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> On 4 Jun 2012, at 09:52, Michael Turner wrote: > So there's a classic distributed-systems paper that gives the > technique its name, yet somehow, even though Kenneth Lundin agreed, > all the way back in 2007, that seq_trace implements Lamport clocks > > http://erlang.org/pipermail/erlang-questions/2007-May/026827.html > > this fact somehow hasn't made it into the documentation. Everybody has > to rediscover it. This is not just a waste of their time, there's also > the opportunity cost for Erlang/OTP (and Ericsson) in people *not* > discovering it because it's not named as such at www.erlang.org. First of all, I didn't dispute that seq_trace implements Lamport clocks, only that the requirements for seq_trace had different origins. In a similar fashion, one might claim that Erlang was inspired by Tony Hoare's CSP, but that would be rewriting history, even though the pieces mostly fit. There was a lot of work done on concurrency algorithms in the '70s and '80s; the telecoms industry (and Ericsson) started designing software-controlled phone switches in the '60s. It's very hard to untangle after the fact who inspired whom (but it's certainly a fascinating excercise - for one thing, Bjarne and co at one point visited Niklaus Wirth and came to the conclusion that many of the things they had been working on were manifest in his Modula-2? smart people working on the same type of problems will sometimes independently arrive at very similar conclusions.) My main reason for responding was that you accused the OTP team of being "dishonest" in not mentioning where their ideas came from. I maintain that they are not; only that many of the inputs that *actually* informed the implementation were either confidential or proprietary enough to be of no interest to the people reading the manuals. A question is of course how many people are helped by the seq_trace documentation mentioning the relation to Lamport clocks. Some might, but others couldn't care less; they just want to know how to use the functionality. We have had examples of OTP man pages in the past that have gone into great detail about algorithm choices, resulting only in terrifying anyone who came there just to learn how to use the API. This is of course the big challenge when writing product manuals. Many things that are of academic interest must be left out of the manual if it ends up confusing the reader. In this case, I'm pretty sure it could be worked in without harming readability, but it is of course perfectly possible to use seq_trace without understanding, or even being aware of the existence of, Lamport clocks. The OTP team is known to accept patches to the documentation, so please feel free to contribute to a more helpful way to describe the tracing support. I'm sure it would be universally appreciated. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From bengt.kleberg@REDACTED Mon Jun 4 12:34:58 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 4 Jun 2012 12:34:58 +0200 Subject: [erlang-questions] StringIO? In-Reply-To: <0abdb979-9267-4dc2-a20d-1829dcb494ea@knuth> References: <0abdb979-9267-4dc2-a20d-1829dcb494ea@knuth> Message-ID: <1338806098.5664.12.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> The module is old and predates github, etc. So it is not available on the web. On the other hand it is only a module (plus a test module if you want that too) so I can easily email it to any interested parties. bengt On Mon, 2012-06-04 at 10:49 +0200, Robert Virding wrote: > Do you have a link to it? > > Robert > > ----- Original Message ----- > > Greetings, > > > > Just to make it perfectly clear, when I said I had a string_io > > module, > > this is what it does. > > > > > > bengt > > > > On Sun, 2012-06-03 at 19:52 +0200, Robert Virding wrote: > > > An alternative would be to write a string_io_server which could do > > > read/write requests against a string and use it as file descriptor > > > in > > > i/o calls. It is well defined and not that difficult. > > > > > > Robert > > > > > > > > > ______________________________________________________________________ > > > Thank you Bob, > > > > > > > > > that looks like it's exactly what I need. > > > > > > > > > Thank you, > > > > > > > > > r. > > > > > > On Wed, May 30, 2012 at 5:25 PM, Bob Ippolito > > > > > > wrote: > > > > > > > > > It's undocumented, but there's the ram_file module: > > > > > > > > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl > > > > > > -bob > > > > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From gustav@REDACTED Mon Jun 4 13:22:54 2012 From: gustav@REDACTED (Gustav Simonsson) Date: Mon, 4 Jun 2012 13:22:54 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> Message-ID: <4FCC9A8E.10706@erlang.org> On 2012-06-04 12:07, Ulf Wiger wrote: > On 4 Jun 2012, at 09:52, Michael Turner wrote: > >> So there's a classic distributed-systems paper that gives the >> technique its name, yet somehow, even though Kenneth Lundin agreed, >> all the way back in 2007, that seq_trace implements Lamport clocks >> >> http://erlang.org/pipermail/erlang-questions/2007-May/026827.html >> >> this fact somehow hasn't made it into the documentation. Everybody has >> to rediscover it. This is not just a waste of their time, there's also >> the opportunity cost for Erlang/OTP (and Ericsson) in people *not* >> discovering it because it's not named as such at www.erlang.org. > First of all, I didn't dispute that seq_trace implements Lamport clocks, > only that the requirements for seq_trace had different origins. In a > similar fashion, one might claim that Erlang was inspired by Tony > Hoare's CSP, but that would be rewriting history, even though the pieces > mostly fit. There was a lot of work done on concurrency algorithms in the > '70s and '80s; the telecoms industry (and Ericsson) started designing > software-controlled phone switches in the '60s. It's very hard to untangle > after the fact who inspired whom (but it's certainly a fascinating excercise > - for one thing, Bjarne and co at one point visited Niklaus Wirth and came > to the conclusion that many of the things they had been working on were > manifest in his Modula-2? smart people working on the same type of > problems will sometimes independently arrive at very similar conclusions.) > > My main reason for responding was that you accused the OTP team of being > "dishonest" in not mentioning where their ideas came from. I maintain that > they are not; only that many of the inputs that *actually* informed the > implementation were either confidential or proprietary enough to be of > no interest to the people reading the manuals. > > A question is of course how many people are helped by the seq_trace > documentation mentioning the relation to Lamport clocks. Some might, > but others couldn't care less; they just want to know how to use the > functionality. We have had examples of OTP man pages in the past that > have gone into great detail about algorithm choices, resulting only in > terrifying anyone who came there just to learn how to use the API. > > This is of course the big challenge when writing product manuals. > Many things that are of academic interest must be left out of the > manual if it ends up confusing the reader. In this case, I'm pretty sure > it could be worked in without harming readability, but it is of course > perfectly possible to use seq_trace without understanding, or even > being aware of the existence of, Lamport clocks. > > The OTP team is known to accept patches to the documentation, > so please feel free to contribute to a more helpful way to describe > the tracing support. I'm sure it would be universally appreciated. +1 Especially under the "Advanced examples" section there is less need for brevity since users clicking on that section likely are prepared for multiple, possibly long, examples of using the Event Tracer. // Gustav Simonsson > > BR, > Ulf W > > Ulf Wiger, Co-founder& Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Mon Jun 4 14:40:33 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 4 Jun 2012 14:40:33 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCC6B5D.4040701@gmail.com> References: <4FCA6332.5000109@eonblast.com> <4FCC6B5D.4040701@gmail.com> Message-ID: On Mon, Jun 4, 2012 at 10:01 AM, Torben Hoffmann wrote: > > > On 04/06/2012 09:49, Joe Armstrong wrote: >> >> >> >> >> Message sequence charts (MSCs) are parts of SDL and UML and there are many >> tools >> (which I don't use) ?to make them - a quick google turned up this >> http://en.wikipedia.org/wiki/MscGen which is food for thought. > > If you want to create MSCs I recommend using other tools than MscGen. > I have tried MscGen and I think that is not so good as PlantUML - > http://plantuml.sourceforge.net/ > I just tried PlantUML - very nice - worked out of the box :-) /Joe > Warning: I have not tried the latest version of MscGen, so my knowledge > might be outdated. > > Cheers, > __ > /orben > > -- > http://www.linkedin.com/in/torbenhoffmann > From ulf@REDACTED Mon Jun 4 14:53:07 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 4 Jun 2012 14:53:07 +0200 Subject: [erlang-questions] deriving boot scripts from a running system Message-ID: I made a small addition to my setup [1] application, which allows you to 'save' a release based on a running system. This is partly a logical consequence of the reload_app/1 function, which allows you to upgrade individual applications easily. For it to be complete, some more interaction with the release_handler would be needed (mainly, support for release_handler:make_permanent(), which I haven't tested yet). A wiki page illustrating the function can be found here: https://github.com/uwiger/setup/wiki/Example:-generating-boot-scripts-from-a-running-system Comments and complementary ideas are welcome. BR, Ulf W [1] http://github.com/uwiger/setup Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From eric@REDACTED Mon Jun 4 15:24:32 2012 From: eric@REDACTED (Eric Moritz) Date: Mon, 4 Jun 2012 09:24:32 -0400 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <4FCC6B5D.4040701@gmail.com> Message-ID: Does anyone have any resources (blog post, talk slides, etc) explaining Trace Driven Development? I have seen quite a few references to it lately but my Google search efforts come up short when I attempt to find an explanation of it. Thanks, Eric Moritz. On Mon, Jun 4, 2012 at 8:40 AM, Joe Armstrong wrote: > On Mon, Jun 4, 2012 at 10:01 AM, Torben Hoffmann > wrote: > > > > > > On 04/06/2012 09:49, Joe Armstrong wrote: > >> > >> > >> > >> > >> Message sequence charts (MSCs) are parts of SDL and UML and there are > many > >> tools > >> (which I don't use) to make them - a quick google turned up this > >> http://en.wikipedia.org/wiki/MscGen which is food for thought. > > > > If you want to create MSCs I recommend using other tools than MscGen. > > I have tried MscGen and I think that is not so good as PlantUML - > > http://plantuml.sourceforge.net/ > > > > I just tried PlantUML - very nice - worked out of the box :-) > > /Joe > > > > Warning: I have not tried the latest version of MscGen, so my knowledge > > might be outdated. > > > > Cheers, > > __ > > /orben > > > > -- > > http://www.linkedin.com/in/torbenhoffmann > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Mon Jun 4 16:06:57 2012 From: eric@REDACTED (Eric Moritz) Date: Mon, 4 Jun 2012 10:06:57 -0400 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <4FCC6B5D.4040701@gmail.com> Message-ID: Disregard my previous question. I read the head of the thread and realized that trace driven development is something that Henning Diedrich coined. Eric. On Mon, Jun 4, 2012 at 9:24 AM, Eric Moritz wrote: > Does anyone have any resources (blog post, talk slides, etc) explaining > Trace Driven Development? I have seen quite a few references to it lately > but my Google search efforts come up short when I attempt to find > an explanation of it. > > Thanks, > Eric Moritz. > > On Mon, Jun 4, 2012 at 8:40 AM, Joe Armstrong wrote: > >> On Mon, Jun 4, 2012 at 10:01 AM, Torben Hoffmann >> wrote: >> > >> > >> > On 04/06/2012 09:49, Joe Armstrong wrote: >> >> >> >> >> >> >> >> >> >> Message sequence charts (MSCs) are parts of SDL and UML and there are >> many >> >> tools >> >> (which I don't use) to make them - a quick google turned up this >> >> http://en.wikipedia.org/wiki/MscGen which is food for thought. >> > >> > If you want to create MSCs I recommend using other tools than MscGen. >> > I have tried MscGen and I think that is not so good as PlantUML - >> > http://plantuml.sourceforge.net/ >> > >> >> I just tried PlantUML - very nice - worked out of the box :-) >> >> /Joe >> >> >> > Warning: I have not tried the latest version of MscGen, so my knowledge >> > might be outdated. >> > >> > Cheers, >> > __ >> > /orben >> > >> > -- >> > http://www.linkedin.com/in/torbenhoffmann >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Mon Jun 4 16:42:13 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Jun 2012 16:42:13 +0200 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: <4FCBE806.5090705@eonblast.com> References: <4FC3F807.3040701@gmail.com> <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> <4FCBE806.5090705@eonblast.com> Message-ID: On Mon, Jun 4, 2012 at 12:41 AM, Henning Diedrich wrote: > How much traffic would you expect, very roughly, > 'optimistically' == upper limit? I don't know. Are you saying that you'd like to host a mirror? From michael.eugene.turner@REDACTED Mon Jun 4 17:10:45 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 5 Jun 2012 00:10:45 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> Message-ID: On Mon, Jun 4, 2012 at 7:07 PM, Ulf Wiger wrote: >> this fact somehow hasn't made it into the documentation. Everybody has >> to rediscover it. This is not just a waste of their time, there's also >> the opportunity cost for Erlang/OTP (and Ericsson) in people *not* >> discovering it because it's not named as such at www.erlang.org. > > First of all, I didn't dispute that seq_trace implements Lamport clocks, And I never said you disputed it. If there's a dispute, it's whether AXE did. I can't see that. > only that the requirements for seq_trace had different origins. That would be a more credible claim if the counters in seq_trace weren't computed in a manner *identical* to the usual descriptions of them found in the literature starting over 30 years ago. > ... smart people working on the same type of > problems will sometimes independently arrive at very similar conclusions.) The copyright on the manual starts with the year 2001. Over two decades after Lamport famously wrote, seq_trace implements exactly what's typically described, and the AXE document -- which doesn't supply nearly enough detail to verify independent invention -- is the best you can offer as evidence of independent invention? > My main reason for responding was that you accused the OTP team of being > "dishonest" in not mentioning where their ideas came from. "Ideas" is *your* plural, not mine. But look at the facts: there was an obvious thing to call it -- obvious to anyone who's studied concurrent programming, and I think you all have. One thing it's never called in the literature is "sequential tracing." As for dishonesty: If somebody had this algorithm described to them, was assigned to write the documentation, and did so in a hurry, without checking with their informant about origins, that makes them more sloppy than dishonest. But not taking the trouble to even *briefly* credit the foundational work by a famous computer scientist, in the official documentation, in 2007, after the question arose on the list and was answered in the affirmative? That's getting *really* sloppy. > ... I maintain that > they are not; only that many of the inputs that *actually* informed the > implementation were either confidential or proprietary enough to be of > no interest to the people reading the manuals. This is an algorithm made famous among people concerned with concurrent programming in the late 70s and in seq_trace it's implemented identically to the canonical implementation. And I don't have evidence of any implementation in Erlang/OTP before 2001. An idea this widespread and remarked upon -- and by the year 2000, celebrated as a landmark paper -- but somehow people missed that they had reinvented a wheel? I don't think so. At best, they missed how sloppy their documentation was. And is. > A question is of course how many people are helped by the seq_trace > documentation mentioning the relation to Lamport clocks. Some might, > but others couldn't care less; they just want to know how to use the > functionality. OK, so now the defense is "who cares who invented it?" There is reason to care. When there's decades worth of literature referring to its uses, *everybody* should care. Anything else is an encouragement to reinvent wheels. > We have had examples of OTP man pages in the past that > have gone into great detail about algorithm choices, resulting only in > terrifying anyone who came there just to learn how to use the API. What a strawman argument. I asked: how long does it take to add "seq_trace implements Lamport clocks"? Nowhere have I suggested that Ericsson's documentation on seq_trace be a comprehensive treatment of the subject. Don't put words in my mouth. I only ask for this: Credit where it's due, and give people a starting point for further investigation (or for the more savvy among those doing concurrent programming, to orient them quickly to what seq_trace does.) In the same four words. > This is of course the big challenge when writing product manuals. Writing a 4-word sentence giving credit where it's due is no big writing challenge. And in this case, it's an ethical imperative. > Many things that are of academic interest must be left out of the > manual if it ends up confusing the reader. In your last response, you said you weren't even sure whether seq_trace implemented Lamport clocks. A glance by anyone familiar with them can verify that it does. Despite your previous level of ignorance about Lamport clocks, it now sounds like you know for a fact that they are only of academic interest. Where is THAT certainty coming from? A comprehensive literature survey done in a few hours? > ... In this case, I'm pretty sure > it could be worked in without harming readability, but it is of course > perfectly possible to use seq_trace without understanding, or even > being aware of the existence of, Lamport clocks. As it happens, so far, I haven't used the Lamport's relative timestamp feature. But I'm coming up to the point where the fact that they help describe a partial ordering of events is an essential part of my testing. I *will* be looking to the literature to see what's been done on this subject. Who wouldn't, given that people have over 30 years of experience with it now, and must have written about it? > The OTP team is known to accept patches to the documentation, > so please feel free to contribute to a more helpful way to describe > the tracing support. I'm sure it would be universally appreciated. The one time I tried to get a patch into anything in Erlang (something that was causing a build under FreeBSD to segfault), it didn't make it into the next release -- even though other people helped get it into shape for submission. Pardon me for being a little leery of the process. I hereby suggest, for whomever it's easier: add the line "seq_trace implements Lamport clocks." (Alternatively, "Lamport timestamps.") Kenneth Lundin already admitted it back in 2007, on this list. It's long past time to get it in there. And I suggest that you NOT take the opportunity to add any supposed "history" about how AXE implemented Lamport clocks until you can demonstrate that it did. From what I can see in the documentation you pointed to, AXE did not. -michael turner From robert.virding@REDACTED Mon Jun 4 18:00:28 2012 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 04 Jun 2012 17:00:28 +0100 (BST) Subject: [erlang-questions] Trace-Driven Development In-Reply-To: Message-ID: <017901a6-e1c0-49aa-9c8f-12ee7f0096e7@knuth> I really think there are two issues here: what they are, or what they are commonly known as; and the generating idea behind them. I can't comment about seq_trace as I have absolutely no idea how they actually got the idea to do it like it is, whether from Lamport or from "forlopp trace" in AXE. And how you should describe them is another matter. IF the idea came "forlopp trace" then you should say that and then maybe add a comment that it is similar to Lamport clocks. Another example is when people say that Erlang implements the actor model. Well perhaps, but when we designed Erlang it was definitely not the actor model which influenced us as we had not heard about the actor model at that time. Even though it had been invented much earlier. We worked from our ideas of how such a system could/should be implemented. With maybe a little CSP in it, at least that is where the '!' came from, and a long defunct '?'. The actor model first came when people told us we had implemented it. So if we don't mention the actor model we are not being dishonest, but truthful. Robert ----- Original Message ----- > On Mon, Jun 4, 2012 at 7:07 PM, Ulf Wiger wrote: > > >> this fact somehow hasn't made it into the documentation. Everybody > >> has > >> to rediscover it. This is not just a waste of their time, there's > >> also > >> the opportunity cost for Erlang/OTP (and Ericsson) in people *not* > >> discovering it because it's not named as such at www.erlang.org. > > > > First of all, I didn't dispute that seq_trace implements Lamport > > clocks, > > And I never said you disputed it. If there's a dispute, it's whether > AXE did. I can't see that. > > > only that the requirements for seq_trace had different origins. > > That would be a more credible claim if the counters in seq_trace > weren't computed in a manner *identical* to the usual descriptions of > them found in the literature starting over 30 years ago. > > > ... smart people working on the same type of > > problems will sometimes independently arrive at very similar > > conclusions.) > > The copyright on the manual starts with the year 2001. Over two > decades after Lamport famously wrote, seq_trace implements exactly > what's typically described, and the AXE document -- which doesn't > supply nearly enough detail to verify independent invention -- is the > best you can offer as evidence of independent invention? > > > My main reason for responding was that you accused the OTP team of > > being > > "dishonest" in not mentioning where their ideas came from. > > "Ideas" is *your* plural, not mine. But look at the facts: there was > an obvious thing to call it -- obvious to anyone who's studied > concurrent programming, and I think you all have. One thing it's > never > called in the literature is "sequential tracing." > > As for dishonesty: If somebody had this algorithm described to them, > was assigned to write the documentation, and did so in a hurry, > without checking with their informant about origins, that makes them > more sloppy than dishonest. > > But not taking the trouble to even *briefly* credit the foundational > work by a famous computer scientist, in the official documentation, > in > 2007, after the question arose on the list and was answered in the > affirmative? That's getting *really* sloppy. > > > ... I maintain that > > they are not; only that many of the inputs that *actually* informed > > the > > implementation were either confidential or proprietary enough to be > > of > > no interest to the people reading the manuals. > > This is an algorithm made famous among people concerned with > concurrent programming in the late 70s and in seq_trace it's > implemented identically to the canonical implementation. And I don't > have evidence of any implementation in Erlang/OTP before 2001. An > idea > this widespread and remarked upon -- and by the year 2000, celebrated > as a landmark paper -- but somehow people missed that they had > reinvented a wheel? I don't think so. At best, they missed how sloppy > their documentation was. And is. > > > A question is of course how many people are helped by the seq_trace > > documentation mentioning the relation to Lamport clocks. Some > > might, > > but others couldn't care less; they just want to know how to use > > the > > functionality. > > OK, so now the defense is "who cares who invented it?" There is > reason > to care. When there's decades worth of literature referring to its > uses, *everybody* should care. Anything else is an encouragement to > reinvent wheels. > > > We have had examples of OTP man pages in the past that > > have gone into great detail about algorithm choices, resulting only > > in > > terrifying anyone who came there just to learn how to use the API. > > What a strawman argument. I asked: how long does it take to add > "seq_trace implements Lamport clocks"? Nowhere have I suggested that > Ericsson's documentation on seq_trace be a comprehensive treatment of > the subject. Don't put words in my mouth. I only ask for this: Credit > where it's due, and give people a starting point for further > investigation (or for the more savvy among those doing concurrent > programming, to orient them quickly to what seq_trace does.) In the > same four words. > > > This is of course the big challenge when writing product manuals. > > Writing a 4-word sentence giving credit where it's due is no big > writing challenge. And in this case, it's an ethical imperative. > > > Many things that are of academic interest must be left out of the > > manual if it ends up confusing the reader. > > In your last response, you said you weren't even sure whether > seq_trace implemented Lamport clocks. A glance by anyone familiar > with > them can verify that it does. Despite your previous level of > ignorance > about Lamport clocks, it now sounds like you know for a fact that > they > are only of academic interest. Where is THAT certainty coming from? A > comprehensive literature survey done in a few hours? > > > ... In this case, I'm pretty sure > > it could be worked in without harming readability, but it is of > > course > > perfectly possible to use seq_trace without understanding, or even > > being aware of the existence of, Lamport clocks. > > As it happens, so far, I haven't used the Lamport's relative > timestamp > feature. But I'm coming up to the point where the fact that they help > describe a partial ordering of events is an essential part of my > testing. I *will* be looking to the literature to see what's been > done > on this subject. Who wouldn't, given that people have over 30 years > of > experience with it now, and must have written about it? > > > The OTP team is known to accept patches to the documentation, > > so please feel free to contribute to a more helpful way to describe > > the tracing support. I'm sure it would be universally appreciated. > > The one time I tried to get a patch into anything in Erlang > (something > that was causing a build under FreeBSD to segfault), it didn't make > it > into the next release -- even though other people helped get it into > shape for submission. Pardon me for being a little leery of the > process. > > I hereby suggest, for whomever it's easier: add the line "seq_trace > implements Lamport clocks." (Alternatively, "Lamport timestamps.") > Kenneth Lundin already admitted it back in 2007, on this list. It's > long past time to get it in there. > > And I suggest that you NOT take the opportunity to add any supposed > "history" about how AXE implemented Lamport clocks until you can > demonstrate that it did. From what I can see in the documentation you > pointed to, AXE did not. > > -michael turner > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Jun 4 19:56:47 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 4 Jun 2012 19:56:47 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> Message-ID: <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> On 4 Jun 2012, at 17:10, Michael Turner wrote: > > The copyright on the manual starts with the year 2001. Over two > decades after Lamport famously wrote, seq_trace implements exactly > what's typically described, and the AXE document -- which doesn't > supply nearly enough detail to verify independent invention -- is the > best you can offer as evidence of independent invention? Well, FWIW, it was introduced before 2001. I'm guessing 1998 or even 1997 (OTP R4, I believe). Listen, I tried to politely point out to you that the *requirements* for sequence trace in Erlang grew from Ericsson developers' experience using "forlopp tracing" in AXE. Much like Erlang itself was created - as Joe has described it - as an effort to "make something like AXE, only better". This is also why it is called "sequential tracing" (let's call this an educated guess, as I was not part of the naming discussion, although I *was* in the loop when the requirement came up). The people who asked for the feature, and helped finance the development of it, did not ask for Lamport clocks - they asked for something like "forlopp tracing" in AXE. "Forlopp" is a bastardization of the Swedish word "f?rlopp", which means "sequence". In the world of AXE, it was a way to associate events to a "transaction", which could be restarted if something went wrong. Sequence tracing was therefore a perfectly logical thing to call it in Erlang, as it is one of the correct ways to translate "forlopp trace" to real English. There was no deceit, sloppiness or arrogance behind it. I have told you that the details of AXE's implementation are proprietary. I provided one link to a document that *was* publicly available, and that mentions "forlopp tracing". Take that as a form of existence proof of "sequence tracing" in AXE. As for "independent verification", am I to understand that I'm being chided for not revealing secret Ericsson design details in a public forum just so you can independently verify my claims? (However, ordering may not have been a terribly difficult problem to manage in AXE, as it was a single-CPU system, never distributed. When the OTP team implemented similar support, obviously they had to make something that worked in a distributed system). >> A question is of course how many people are helped by the seq_trace >> documentation mentioning the relation to Lamport clocks. Some might, >> but others couldn't care less; they just want to know how to use the >> functionality. > > OK, so now the defense is "who cares who invented it?" There is reason > to care. When there's decades worth of literature referring to its > uses, *everybody* should care. Anything else is an encouragement to > reinvent wheels. I did agree that the reference to Lamport clocks could be added to the documentation, since it is not without interest. The one thing I don't agree with is that the OTP team deserves infamy for not having done so already. When publishing research, it is extremely bad form not to mention prior art. When writing a user guide, one has to consider whether describing details of the implementation actually helps the user, *and* whether those details are something one wants to commit to as part of the interface. I hope you can appreciate the difference: - Kenneth agreeing on a mailing list that Lamport clocks are used is a service to the community, but not a commitment to stay with this design choice indefinitely (not that I think there are that many other good ways to do it). - Inserting details about the implementation in the Reference Manual elevates that information to become part of the interface. Another, perhaps more appropriate, place to insert the reference is as a comment in the source code. It isn't there, however. Granted, the interface *as described* pretty much commits to Lamport clocks. It does seem reasonable to mention that, although I'm still not convinced that many will find it that helpful. > As it happens, so far, I haven't used the Lamport's relative timestamp > feature. But I'm coming up to the point where the fact that they help > describe a partial ordering of events is an essential part of my > testing. I *will* be looking to the literature to see what's been done > on this subject. Who wouldn't, given that people have over 30 years of > experience with it now, and must have written about it? Perhaps this thesis can be of interest then? http://www.student.nada.kth.se/~nstahle/staahle_niklas.pdf (Interestingly, the thesis project was run at Ericsson, on real-time 'transaction' or 'sequence' tracing. While it mentions Lamport and a few other techniques, it fails to mention that both the AXE and Erlang - both established Ericsson technologies at the time - already supported this feature. ;-) >> The OTP team is known to accept patches to the documentation, >> so please feel free to contribute to a more helpful way to describe >> the tracing support. I'm sure it would be universally appreciated. > > The one time I tried to get a patch into anything in Erlang (something > that was causing a build under FreeBSD to segfault), it didn't make it > into the next release -- even though other people helped get it into > shape for submission. Pardon me for being a little leery of the > process. Patching can be pretty hairy, but note that patching the documentation is much easier than patching the emulator. ;-) > And I suggest that you NOT take the opportunity to add any supposed > "history" about how AXE implemented Lamport clocks until you can > demonstrate that it did. From what I can see in the documentation you > pointed to, AXE did not. Actually, there is some mention of it in the seq_trace docs already: "A possible output from the system's sequential_tracer (inspired by AXE-10 and MD-110) could look like: 17:<0.30.0> Info {0,1} WITH "**** Trace Started ****" 17:<0.31.0> Received {0,2} FROM <0.30.0> WITH {<0.30.0>,the_message} 17:<0.31.0> Info {2,3} WITH "We are here now" 17:<0.30.0> Received {2,4} FROM <0.31.0> WITH {ack,{received,the_message}}" (MD-110 is another proprietary, PLEX-based, Ericsson system). More Ericsson history than that is hardly needed, but does somewhat strengthen my claim that these systems were the main source of inspiration. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From emeka_1978@REDACTED Mon Jun 4 21:25:16 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Mon, 4 Jun 2012 12:25:16 -0700 (PDT) Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCC6B5D.4040701@gmail.com> References: <4FCA6332.5000109@eonblast.com> <4FCC6B5D.4040701@gmail.com> Message-ID: <2749544f-1d1a-4ecd-be2e-012daaf20520@f30g2000vbz.googlegroups.com> Thank you very much for PlantUML. On Jun 4, 10:01?am, Torben Hoffmann wrote: > On 04/06/2012 09:49, Joe Armstrong wrote:> > > > Message sequence charts (MSCs) are parts of SDL and UML and there are many tools > > (which I don't use) ?to make them - a quick google turned up this > >http://en.wikipedia.org/wiki/MscGenwhich is food for thought. > > If you want to create MSCs I recommend using other tools than MscGen. > I have tried MscGen and I think that is not so good as PlantUML -http://plantuml.sourceforge.net/ > > Warning: I have not tried the latest version of MscGen, so my knowledge > might be outdated. > > Cheers, > __ > /orben > > --http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From erisa85d@REDACTED Mon Jun 4 22:50:07 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Mon, 4 Jun 2012 22:50:07 +0200 Subject: [erlang-questions] Enoent file error message Message-ID: Hello, I am using Tsung, which is a load generator tool written in Erlang to do some performance evaluations. When i start Tsung, a local erlang node is started, which afterwards starts a slave monitoring node in the remote machine where I am performing the tests. I can see that the slave remote node is started properly, and the Tsung Erlang code that servers for monitoring, is load correctly. However when this remote node tries to open an existing file in the remote Suse Linux server, i get the following error: =INFO REPORT==== 4-Jun-2012::11:12:53 === * 127 ts_os_mon_erlang:(5:<7176.63.0>) trying to open file: "/proc/net/dev"* * 128 ** at node os_mon@REDACTED *** * 129 * * 130 =INFO REPORT==== 4-Jun-2012::11:12:53 ===* * 131 ts_os_mon_erlang:(5:<7176.63.0>) file: "/proc/net/dev" not succeeded to open for reason: enoent * * 132 ** at node os_mon@REDACTED *** I thought it was a file permission problem, however when I try the following test, it succeeds: erisa$ *ssh dcr1a erl* Eshell V5.9 (abort with ^G) 1> *file:open("/proc/net/dev",read).* {ok,<0.33.0>} 2> Any idea would be very appreciated! Thanks, Erisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Jun 5 08:01:53 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 5 Jun 2012 15:01:53 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: On Tue, Jun 5, 2012 at 2:56 AM, Ulf Wiger wrote: > Listen, I tried to politely point out to you that the *requirements* > for sequence trace in Erlang grew from Ericsson developers' > experience using "forlopp tracing" in AXE. What you actually wrote: "Anyway, not noting in the docs that sequence trace mimicks AXE's forlopp trace seems forgiveable." But there's nothing in the AXE documentation you pointed me to that shows AXE forlopp trace doing Lamport clocks, which is the relevant point. What you actually wrote: "... many of the inputs that *actually* informed the implementation were either confidential or proprietary enough to be of no interest to the people reading the manuals." Yeah, but there's nothing proprietary or confidential about Lamport clocks, described in a publication that's cited *directly* in publications by about half-a-dozen Erlang contributors in the 1990s. Since that Lamport publication is cited in over 2000 other publications, including one on the internals of Mnesia, it's hard to see how it can be deemed, in advance, "of no interest to the people reading the manuals." *I* read the manual on seq_trace. My reaction? I expressed considerable confusion, here: http://erlang.org/pipermail/erlang-questions/2011-June/059235.html and here https://groups.google.com/group/erlang-programming/browse_thread/thread/3633ef0d84d63bf3/ef39b31f33011f8c That was before it finally dawned on me: "Lamport clocks! Why the hell don't they say so!? Lots has been written about Lamport clocks, and undoubtedly somewhere something has been written more clearly and more usefully than what I'm looking in here. Obviously, if I want to get the most out of seq_trace's trace tokens, reading the seq_trace documentation isn't going to get me much farther. I'll have to go to the open literature." > Much like Erlang itself > was created - as Joe has described it - as an effort to "make something > like AXE, only better". Yeah, the same Joe Armstrong who's co-author of a book about Erlang whose second edition in 1993 cites Leslie Lamport's paper defining Lamport clocks. > This is also why it is called "sequential tracing" (let's call this an > educated guess, as I was not part of the naming discussion, > although I *was* in the loop when the requirement came up). If you want to call everything that seq_trace implements "sequential tracing," OK, fine. But if you've got Lamport clocks in that implementation, and in the API, for God's sake, call them Lamport clocks. Say so in the beginning. Lots of people know what to do with these. Not giving those people a way to get their usual handle on them means you're leaving talent on the table. Telling them early in the documentation would inspire confidence in documentation that's so rife with basic copyediting errors as to inspire very little confidence indeed. > The people who asked for the feature, and helped finance the > development of it, did not ask for Lamport clocks - they asked for > something like "forlopp tracing" in AXE. "Forlopp" is a bastardization > of the Swedish word "f?rlopp", which means "sequence". In the > world of AXE, it was a way to associate events to a "transaction", > which could be restarted if something went wrong. Seq_trace is not "the feature" -- it's a collection of features. It came out of a milieu in which I've identified half-a-dozen people who, if their publications in the 1990s are any indication, know perfectly well what Lamport clocks are. > Sequence tracing was therefore a perfectly logical thing to call > it in Erlang, as it is one of the correct ways to translate "forlopp > trace" to real English. > > There was no deceit, sloppiness or arrogance behind it. I can see that there might be "no deceit," but only by way of sloppiness. And the sloppiness is pretty obvious. As for arrogance, well, when you gesture vaguely in the direction of AXE documentation that contains no evidence of any independent invention of Lamport clocks, in support of the claim "we invented that too", and you do it apparently without actually closely comparing that the relevant sections of that document, the seq_trace documentation, and what Lamport wrote.... OK, I'll be kind, and just call that "yet more sloppiness," not arrogance. > I have told you that the details of AXE's implementation are > proprietary. I provided one link to a document that *was* publicly > available, and that mentions "forlopp tracing". Take that as a > form of existence proof of "sequence tracing" in AXE. I see a tracing facility there. I don't see Lamport clocks. How many times do I have to say this before you'll actually take a look for yourself? > As for "independent verification", am I to understand that I'm being > chided for not revealing secret Ericsson design details in a public > forum just so you can independently verify my claims? There's nothing secret about Lamport clocks. Kenneth Lundin said seq_trace implements Lamport clocks, in 2007. Mnesia uses Lamport clocks, as reported in 1999. The first book about Erlang (2nd ed. 1993) refers to the 1978 paper in which Lamport clocks were defined. Where is the excuse for *not* saying that seq_trace implements Lamport clocks? [snip] > I did agree that the reference to Lamport clocks could be added to > the documentation, since it is not without interest. The one thing > I don't agree with is that the OTP team deserves infamy for not > having done so already. I don't believe the entire team deserves infamy. I DO believe that whoever's responsible for documenting seq_trace has been, at best, very lax. As for any actual motivation to obscure origins, the best I can come up with is one that is still very compelling indeed: having a grotesquely swollen patent pool, for purposes of intimidation and/or deterrence. After all, it's much easier to get a patent attorney to write one of those not-very-inventive software patents if you say, "well, there really IS no prior art on this one, except our own, of course. Which is proprietary." At its extremes, you get disgusted engineering teams pulling pranks like the one James Gosling described: a draft patent describing the electrical power switch. http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=44&f=G&l=50&co1=AND&d=PTXT&s1=%22gosling,+james%22.INNM.&OS=IN/%22gosling,+james%22&RS=IN/%22gosling,+james%22 http://www.theregister.co.uk/2010/08/17/golsing_on_sun_goofy_patent_contestas/ > When publishing research, it is extremely bad form not to mention > prior art. When writing a user guide, one has to consider whether > describing details of the implementation actually helps the user, > *and* whether those details are something one wants to commit > to as part of the interface. > > I hope you can appreciate the difference: Oh, I definitely can. When the documentation isn't very good, a reference in it to published research behind its API can at least encourage confidence. I found the seq_trace documentation pretty confusing. I pressed on anyway, because I didn't like my alternative: other trace facilities that are (literally) described in the company's official documentation as hellish. Then I saw how seq_trace generates trace tokens, which made it sound like some serious thought had been put into seq_trace -- if only I could figure out the implications of the algorithm. After a couple of unanswered questions about this aspect of seq_trace, on this list, I finally did figure it out: I was looking at Lamport clocks. Knowing that seq_trace implements Lamport clocks would have helped me. Maybe lots of people never even get as far as I got with seq_trace before shrugging and deciding it's something like gs was and is -- basically a moribund, abandoned line of approach to tracing in Erlang/OTP, even if the documentation doesn't come right out and say so. (In the case of gs, getting the documentation to come right out and say so required prodding from me, after I discovered I was at a dead end with it.) That's a waste. An easily remedied waste. How long does something have to stay in beta? I'm finding other versions of the seq_trace documentation with copyrights going back to 1991. If, as you say, the requirements come out of the late 90s, that's like FIFTEEN YEARS IN BETA. > - Kenneth agreeing on a mailing list that Lamport clocks are used > ?is a service to the community, but not a commitment to stay with > ?this design choice indefinitely (not that I think there are that many > ?other good ways to do it). You mean if I write to seq_trace as a spec that uses a pretty standard (and in fact famous) way of generating trace tokens, it might change out from under me? May I ask, why? To what? Or would that be inquiring of Ericsson's trade secrets? > - Inserting details about the implementation in the Reference Manual > ?elevates that information to become part of the interface. In the case of seq_trace implementing Lamport clocks, that sounds like an excellent idea to me, given the vintage of the idea and how, in the test of time, that idea has withstood over three decades. > Another, perhaps more appropriate, place to insert the reference is > as a comment in the source code. It isn't there, however. Gosh, yet another place where you save people some comprehension time by just writing four words. I wonder how many more? > Granted, the interface *as described* pretty much commits to Lamport > clocks. It does seem reasonable to mention that, although I'm still > not convinced that many will find it that helpful. Of course not. After all, somebody would have to read all 2000+ papers that cite Lamport's paper, before they could be *really* sure. [snip] > Perhaps this thesis can be of interest then? > > http://www.student.nada.kth.se/~nstahle/staahle_niklas.pdf > > (Interestingly, the thesis project was run at Ericsson, on real-time > 'transaction' or 'sequence' tracing. While it mentions Lamport and > a few other techniques, it fails to mention that both the AXE and > Erlang - both established Ericsson technologies at the time - > already supported this feature. ;-) Yeah, when you don't identify the wheel by name, making it hard to find in a search, people reinvent that wheel. How amazing! I've also seen a professor's assignment online: he starts with simple trace framework in Erlang, and the student is then asked to implement Lamport clocks. No mention of seq_trace. I guess it's because that professor searched on "Lamport clocks" in the Erlang documentation and said, "Hm, that's odd: no Lamport clocks." Nice job of hiding gold. Next step: cast doubt on whether gold is actually all that valuable, when compared to something unstated that you might invent one of these days. >>> The OTP team is known to accept patches to the documentation, >>> so please feel free to contribute to a more helpful way to describe >>> the tracing support. I'm sure it would be universally appreciated. >> >> The one time I tried to get a patch into anything in Erlang (something >> that was causing a build under FreeBSD to segfault), it didn't make it >> into the next release -- even though other people helped get it into >> shape for submission. Pardon me for being a little leery of the >> process. > > Patching can be pretty hairy, but note that patching the documentation > is much easier than patching the emulator. ;-) I'm sure it *can* be hairy, but mine was a single-character change, as I recall, and it only asked that a C auto array have a dimension of 1 rather than zero (a write to that array unsurprising caused the stack to be trashed on the platform I was trying to build on, which made using source-level debugging a non-starter, which made debugging a total pain in the ass.) >> And I suggest that you NOT take the opportunity to add any supposed >> "history" about how AXE implemented Lamport clocks until you can >> demonstrate that it did. From what I can see in the documentation you >> pointed to, AXE did not. > > Actually, there is some mention of it in the seq_trace docs already: [snip] It shows traces "inspired" by AXE, but without the sequence tokens that would suggest that AXE implemented Lamport clocks -- IF it ever did. > More Ericsson history than that is hardly needed, ... I'd say that history is entirely dispensable. I'd even throw out the AXE traces, if you could use the space to instead say something about how useful the trace tokens can be. > ... but does > somewhat strengthen my claim that these systems were the main > source of inspiration. We're not talking about whether AXE was "the main source of inspiration". We're talking about whether the seq_trace documentation (especially given how confusing it is to begin with) should mention that it implements Lamport clocks, since, after all, (a) it does, and (b) lots of people who do concurrent programming know what Lamport clocks are already -- including about half-a-dozen in the Erlang group (at least if what they cite in their publications is any indication.) -michael turner From anders.wei@REDACTED Tue Jun 5 08:04:45 2012 From: anders.wei@REDACTED (Anders Wei) Date: Tue, 5 Jun 2012 14:04:45 +0800 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCA6332.5000109@eonblast.com> References: <4FCA6332.5000109@eonblast.com> Message-ID: I really think it's a great idea. For daily erlang programming, I will always collect call traces and get familiar of the modules that I am not familiar with. Actually I wrote a small tool to generate call flows, but there is a poor muti-process support, so it doesn't require Lamport clock. It just collect the dbg:tracer output and print it out in another way, it's simple but it works fine :) It work like this: [cid:image001.png@REDACTED] From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Henning Diedrich Sent: Sunday, June 03, 2012 3:02 AM To: Erlang Questions Subject: [erlang-questions] Trace-Driven Development Should there be any interest I would like to discuss tracing and if it may be more valuable for the actor model than unit tests. A couple of days ago, during Jesper Louis Andersen's very enlightening debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether showing visual traces first may be a good approach to teaching Erlang, book-side. I would like to propose again that trace-driven development may possibly be the way to teach andalso program Erlang. In the instance, we were talking about visual traces, like these: http://www.erlang.org/doc/apps/et/coffee_order.png I mean it in the slightly unrealistic way that most people probably appreciate test-driven programming as a great idea but seldom execute it as dogmatically as preached. But I realize that what I actually do when programming a new app, or when looking for truly nasty bugs, is almost always pretty 'trace-driven'. Writing traces into the program has become a routine step for me when coding medium complicated stuff that uses supervisors, monitors and multiple process that come to live at various times throughout the lifetime of an Erlang (OTP) application -- and most of all, during the start up. And I have always found surprises, which process comes to live when. I think Jesper related a similar story how they found tons of bugs that no-one had realized were there when using tracing (not sure any more.) I believe that the main aggravating factor is that you don't construct an OTP application from scratch that often. You'll have to look things up again every next time. My guess is that this is true for the vast majority of Erlang programmers. Of course, for the ultimate expert, it may be hard to grasp what I am even talking about because it's all so 'obvious'. But what I really find myself currently doing, before writing tests, is writing traces. I also haven't found out how to plan out and write meaningful stateful tests up front, before I even made it to architect the application. My concrete order is: writing first, get to compile, put in traces, try to run, debug. I am so sure meanwhile that I will need the traces that I don't wait anymore until I have gotten lost in searching for a bug and not even knowing what happens before the crash and what parts of the program may have silently died and what other parts may have waited for which other part. These things will be obvious for expert Erlang programmers who have set up applications time and again. But few people using Erlang for productivity reasons will ever come into that position. And for finding bugs: that's exactly the moment when reality doesn't follow your fantasy and a reality check in the form of a trace is one of the first tools to turn to. Is it not? So trace-driven development sounds like it could be a useful recommendation. And if it is, a new best practice of waterfalling your application into existence could maybe emerge, similar to what Joe describes in his book (if I remember well) how he almost always starts out. I learned a lot in Jesper's tutorial. For example that because Erlang and OTP keep growing, there are now three distinct mechanisms, with respective modules and function calls, to trace and debug. And despite me using traces a lot, I hadn't used any of them, for the wrong reasons it turned out. (And for productivity, a simple io:format can beat getting tied up into using more powerful but less simple minded options.) So I am asking this out of honest ignorance: Instead of tracing, is there a way to write meaningful, concise tests that are meant or at least good for checking the sequence of process communications? In other words, are there better alternatives to follow traces with the naked eye? I do not mean a heavy construct, or an also-possible use, but a test-package made for that and/or a useful practice that someone is in fact applying? And it need not be the visual traces as the image linked to above. A ping-pong output in the terminal may do (if not skewered by delays or re-sequences or contentions in io:format.) For trace-driven development, what concrete procedure with which concrete calls and parameters to erl can be a best practice for starting out? I am sure it' s 'obvious' for many, but at least for me, not. What could be the hello world case for a trace-driven approach? And finally, how do you approach setting up a new app, finding a bug -- how much tracing does everyone use in these things? And is anyone also using io:format instead of the OTP goodness, if so, for a good reason? Best, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 41360 bytes Desc: image001.png URL: From watson.timothy@REDACTED Tue Jun 5 08:39:34 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 5 Jun 2012 07:39:34 +0100 Subject: [erlang-questions] Enoent file error message In-Reply-To: References: Message-ID: <5ECF8C1A-C01A-4AC9-AB9C-97EB99F2E3A4@gmail.com> What account is the remote node running as? Are you sure that user has the appropriate permissions? On 4 Jun 2012, at 21:50, Erisa Dervishi wrote: > Hello, > > I am using Tsung, which is a load generator tool written in Erlang to do some performance evaluations. > When i start Tsung, a local erlang node is started, which afterwards starts a slave monitoring node in the remote machine where I am performing the tests. > > I can see that the slave remote node is started properly, and the Tsung Erlang code that servers for monitoring, is load correctly. > However when this remote node tries to open an existing file in the remote Suse Linux server, i get the following error: > > =INFO REPORT==== 4-Jun-2012::11:12:53 === > 127 ts_os_mon_erlang:(5:<7176.63.0>) trying to open file: "/proc/net/dev" > 128 ** at node os_mon@REDACTED ** > 129 > 130 =INFO REPORT==== 4-Jun-2012::11:12:53 === > 131 ts_os_mon_erlang:(5:<7176.63.0>) file: "/proc/net/dev" not succeeded to open for reason: enoent > 132 ** at node os_mon@REDACTED ** > > I thought it was a file permission problem, however when I try the following test, it succeeds: > > erisa$ ssh dcr1a erl > Eshell V5.9 (abort with ^G) > 1> file:open("/proc/net/dev",read). > {ok,<0.33.0>} > 2> > > Any idea would be very appreciated! > > Thanks, > Erisa > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Jun 5 09:47:02 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 5 Jun 2012 16:47:02 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: Correction: I wrote that the seq_trace documentation "shows traces "inspired" by AXE, but without the sequence tokens that would suggest that AXE implemented Lamport clocks -- IF it ever did." In fact, the seq_trace documentation does feature traces with the sequence tokens as generated by Lamport's algorithm. My point, however, stands: "inspired" is not "virtually identical", and definitely not in this case. In the actual AXE documentation Ulf points to http://www.scribd.com/doc/83784121/AXE-System-Testing-1-Apz-212 the only counter mentioned for tracing is an "error intensity counter." Lamport clocks generate timestamps based on *two* counters, neither of them specific to error reporting. Ulf describes the customer's requirements for seq_trace as "like AXE only better". If I may suggest: Ericsson improved on the AXE traces by adding (perhaps among other things) Lamport clocks. There is ample evidence, in the citations and direct references in at least two open-literature publications by Erlang luminaries, that they were quite aware of Lamport's work. Did they just forget all about it, when writing seq_trace? It's possible, I suppose. Or perhaps an engineer employed by the customer suggested the feature, without mentioning Lamport? Who knows? All I know is: if that's what you're doing, that's what you should call it. And speaking of what to call things: I don't think you should still be calling seq_trace "beta", if (as Ulf says) it originated ca 1997. I'd do the interface differently, but the more important thing to me now is stability and correctness. -michael turner On Tue, Jun 5, 2012 at 3:01 PM, Michael Turner wrote: > On Tue, Jun 5, 2012 at 2:56 AM, Ulf Wiger wrote: > >> Listen, I tried to politely point out to you that the *requirements* >> for sequence trace in Erlang grew from Ericsson developers' >> experience using "forlopp tracing" in AXE. > > What you actually wrote: > > "Anyway, not noting in the docs that sequence trace mimicks AXE's > forlopp trace seems forgiveable." > > But there's nothing in the AXE documentation you pointed me to that > shows AXE forlopp trace doing Lamport clocks, which is the relevant > point. > > What you actually wrote: > > "... many of the inputs that *actually* informed the implementation > were either confidential or proprietary enough to be of no interest to > the people reading the manuals." > > Yeah, but there's nothing proprietary or confidential about Lamport > clocks, described in a publication that's cited *directly* in > publications by about half-a-dozen Erlang contributors in the 1990s. > Since that Lamport publication is cited in over 2000 other > publications, including one on the internals of Mnesia, it's hard to > see how it can be deemed, in advance, "of no interest to the people > reading the manuals." *I* read the manual on seq_trace. My reaction? I > expressed considerable confusion, here: > > ?http://erlang.org/pipermail/erlang-questions/2011-June/059235.html > > and here > > ?https://groups.google.com/group/erlang-programming/browse_thread/thread/3633ef0d84d63bf3/ef39b31f33011f8c > > That was before it finally dawned on me: "Lamport clocks! Why the hell > don't they say so!? Lots has been written about Lamport clocks, and > undoubtedly somewhere something has been written more clearly and more > usefully than what I'm looking in here. Obviously, if I want to get > the most out of seq_trace's trace tokens, reading the seq_trace > documentation isn't going to get me much farther. I'll have to go to > the open literature." > >> Much like Erlang itself >> was created - as Joe has described it - as an effort to "make something >> like AXE, only better". > > Yeah, the same Joe Armstrong who's co-author of a book about Erlang > whose second edition in 1993 cites Leslie Lamport's paper defining > Lamport clocks. > >> This is also why it is called "sequential tracing" (let's call this an >> educated guess, as I was not part of the naming discussion, >> although I *was* in the loop when the requirement came up). > > If you want to call everything that seq_trace implements "sequential > tracing," OK, fine. But if you've got Lamport clocks in that > implementation, and in the API, for God's sake, call them Lamport > clocks. Say so in the beginning. Lots of people know what to do with > these. Not giving those people a way to get their usual handle on them > means you're leaving talent on the table. Telling them early in the > documentation would inspire confidence in documentation that's so rife > with basic copyediting errors as to inspire very little confidence > indeed. > >> The people who asked for the feature, and helped finance the >> development of it, did not ask for Lamport clocks - they asked for >> something like "forlopp tracing" in AXE. "Forlopp" is a bastardization >> of the Swedish word "f?rlopp", which means "sequence". In the >> world of AXE, it was a way to associate events to a "transaction", >> which could be restarted if something went wrong. > > Seq_trace is not "the feature" -- it's a collection of features. It > came out of a milieu in which I've identified half-a-dozen people who, > if their publications in the 1990s are any indication, know perfectly > well what Lamport clocks are. > >> Sequence tracing was therefore a perfectly logical thing to call >> it in Erlang, as it is one of the correct ways to translate "forlopp >> trace" to real English. >> >> There was no deceit, sloppiness or arrogance behind it. > > I can see that there might be "no deceit," but only by way of > sloppiness. And the sloppiness is pretty obvious. > > As for arrogance, well, when you gesture vaguely in the direction of > AXE documentation that contains no evidence of any independent > invention of Lamport clocks, in support of the claim "we invented that > too", and you do it apparently without actually closely comparing that > the relevant sections of that document, the seq_trace documentation, > and what Lamport wrote.... OK, I'll be kind, and just call that "yet > more sloppiness," not arrogance. > >> I have told you that the details of AXE's implementation are >> proprietary. I provided one link to a document that *was* publicly >> available, and that mentions "forlopp tracing". Take that as a >> form of existence proof of "sequence tracing" in AXE. > > I see a tracing facility there. I don't see Lamport clocks. How many > times do I have to say this before you'll actually take a look for > yourself? > >> As for "independent verification", am I to understand that I'm being >> chided for not revealing secret Ericsson design details in a public >> forum just so you can independently verify my claims? > > There's nothing secret about Lamport clocks. Kenneth Lundin said > seq_trace implements Lamport clocks, in 2007. Mnesia uses Lamport > clocks, as reported in 1999. The first book about Erlang (2nd ed. > 1993) refers to the 1978 paper in which Lamport clocks were defined. > Where is the excuse for *not* saying that seq_trace implements Lamport > clocks? > > [snip] > >> I did agree that the reference to Lamport clocks could be added to >> the documentation, since it is not without interest. The one thing >> I don't agree with is that the OTP team deserves infamy for not >> having done so already. > > I don't believe the entire team deserves infamy. I DO believe that > whoever's responsible for documenting seq_trace has been, at best, > very lax. > > As for any actual motivation to obscure origins, the best I can come > up with is one that is still very compelling indeed: having a > grotesquely swollen patent pool, for purposes of intimidation and/or > deterrence. After all, it's much easier to get a patent attorney to > write one of those not-very-inventive software patents if you say, > "well, there really IS no prior art on this one, except our own, of > course. Which is proprietary." At its extremes, you get disgusted > engineering teams pulling pranks like the one James Gosling described: > a draft patent describing the electrical power switch. > > ?http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=44&f=G&l=50&co1=AND&d=PTXT&s1=%22gosling,+james%22.INNM.&OS=IN/%22gosling,+james%22&RS=IN/%22gosling,+james%22 > > ?http://www.theregister.co.uk/2010/08/17/golsing_on_sun_goofy_patent_contestas/ > >> When publishing research, it is extremely bad form not to mention >> prior art. When writing a user guide, one has to consider whether >> describing details of the implementation actually helps the user, >> *and* whether those details are something one wants to commit >> to as part of the interface. >> >> I hope you can appreciate the difference: > > Oh, I definitely can. When the documentation isn't very good, a > reference in it to published research behind its API can at least > encourage confidence. > > I found the seq_trace documentation pretty confusing. I pressed on > anyway, because I didn't like my alternative: other trace facilities > that are (literally) described in the company's official documentation > as hellish. Then I saw how seq_trace generates trace tokens, which > made it sound like some serious thought had been put into seq_trace -- > if only I could figure out the implications of the algorithm. After a > couple of unanswered questions about this aspect of seq_trace, on this > list, I finally did figure it out: I was looking at Lamport clocks. > > Knowing that seq_trace implements Lamport clocks would have helped me. > Maybe lots of people never even get as far as I got with seq_trace > before shrugging and deciding it's something like gs was and is -- > basically a moribund, abandoned line of approach to tracing in > Erlang/OTP, even if the documentation doesn't come right out and say > so. (In the case of gs, getting the documentation to come right out > and say so required prodding from me, after I discovered I was at a > dead end with it.) > > That's a waste. An easily remedied waste. > > How long does something have to stay in beta? I'm finding other > versions of the seq_trace documentation with copyrights going back to > 1991. If, as you say, the requirements come out of the late 90s, > that's like FIFTEEN YEARS IN BETA. > >> - Kenneth agreeing on a mailing list that Lamport clocks are used >> ?is a service to the community, but not a commitment to stay with >> ?this design choice indefinitely (not that I think there are that many >> ?other good ways to do it). > > You mean if I write to seq_trace as a spec that uses a pretty standard > (and in fact famous) way of generating trace tokens, it might change > out from under me? May I ask, why? To what? Or would that be inquiring > of Ericsson's trade secrets? > >> - Inserting details about the implementation in the Reference Manual >> ?elevates that information to become part of the interface. > > In the case of seq_trace implementing Lamport clocks, that sounds like > an excellent idea to me, given the vintage of the idea and how, in the > test of time, that idea has withstood over three decades. > >> Another, perhaps more appropriate, place to insert the reference is >> as a comment in the source code. It isn't there, however. > > Gosh, yet another place where you save people some comprehension time > by just writing four words. I wonder how many more? > >> Granted, the interface *as described* pretty much commits to Lamport >> clocks. It does seem reasonable to mention that, although I'm still >> not convinced that many will find it that helpful. > > Of course not. After all, somebody would have to read all 2000+ papers > that cite Lamport's paper, before they could be *really* sure. > > [snip] >> Perhaps this thesis can be of interest then? >> >> http://www.student.nada.kth.se/~nstahle/staahle_niklas.pdf >> >> (Interestingly, the thesis project was run at Ericsson, on real-time >> 'transaction' or 'sequence' tracing. While it mentions Lamport and >> a few other techniques, it fails to mention that both the AXE and >> Erlang - both established Ericsson technologies at the time - >> already supported this feature. ;-) > > Yeah, when you don't identify the wheel by name, making it hard to > find in a search, people reinvent that wheel. How amazing! > > I've also seen a professor's assignment online: he starts with simple > trace framework in Erlang, and the student is then asked to implement > Lamport clocks. No mention of seq_trace. I guess it's because that > professor searched on "Lamport clocks" in the Erlang documentation and > said, "Hm, that's odd: no Lamport clocks." > > Nice job of hiding gold. Next step: cast doubt on whether gold is > actually all that valuable, when compared to something unstated that > you might invent one of these days. > >>>> The OTP team is known to accept patches to the documentation, >>>> so please feel free to contribute to a more helpful way to describe >>>> the tracing support. I'm sure it would be universally appreciated. >>> >>> The one time I tried to get a patch into anything in Erlang (something >>> that was causing a build under FreeBSD to segfault), it didn't make it >>> into the next release -- even though other people helped get it into >>> shape for submission. Pardon me for being a little leery of the >>> process. >> >> Patching can be pretty hairy, but note that patching the documentation >> is much easier than patching the emulator. ;-) > > I'm sure it *can* be hairy, but mine was a single-character change, as > I recall, and it only asked that a C auto array have a dimension of 1 > rather than zero (a write to that array unsurprising caused the stack > to be trashed on the platform I was trying to build on, which made > using source-level debugging a non-starter, which made debugging a > total pain in the ass.) > >>> And I suggest that you NOT take the opportunity to add any supposed >>> "history" about how AXE implemented Lamport clocks until you can >>> demonstrate that it did. From what I can see in the documentation you >>> pointed to, AXE did not. >> >> Actually, there is some mention of it in the seq_trace docs already: > > [snip] > > It shows traces "inspired" by AXE, but without the sequence tokens > that would suggest that AXE implemented Lamport clocks -- IF it ever > did. > >> More Ericsson history than that is hardly needed, ... > > I'd say that history is entirely dispensable. I'd even throw out the > AXE traces, if you could use the space to instead say something about > how useful the trace tokens can be. > >> ... but does >> somewhat strengthen my claim that these systems were the main >> source of inspiration. > > We're not talking about whether AXE was "the main source of > inspiration". We're talking about whether the seq_trace documentation > (especially given how confusing it is to begin with) should mention > that it implements Lamport clocks, since, after all, (a) it does, and > (b) lots of people who do concurrent programming know what Lamport > clocks are already -- including about half-a-dozen in the Erlang group > (at least if what they cite in their publications is any indication.) > > -michael turner From ulf@REDACTED Tue Jun 5 09:52:24 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 09:52:24 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: On 5 Jun 2012, at 08:01, Michael Turner wrote: > We're not talking about whether AXE was "the main source of > inspiration". We're talking about whether the seq_trace documentation > (especially given how confusing it is to begin with) should mention > that it implements Lamport clocks, since, after all, (a) it does, and > (b) lots of people who do concurrent programming know what Lamport > clocks are already -- including about half-a-dozen in the Erlang group > (at least if what they cite in their publications is any indication.) *You* are talking about that, and in the process, firing off rants insulting people who have worked much harder than you to make Erlang a solid platform, implying that they are lazy, possibly dishonest and ignorant. I claim that seq_trace implements Lamport clocks "by accident", and that it was not the original purpose, nor a complete solution to the problem. It may be "all it does", but it's not all it should do. I don't say that out of ignorance, but because I still recall many of the discussions before, during and after the implementation of seq_trace. Meanwhile, you tried to submit a patch once, and since it went badly, you can't be bothered to submit a suggestion for how to fix this documentation bug that upsets you (but apparently not that many others, judging by the lack of sympathetic uproar). Still, you are making some good points. If you would dispense with the sarcasm, chances are people would be more inclined to listen. Also, if you would be less focused on defending your position, we might actually be able to agree on something. The reason I reply to you is not because I want to get back at you for insulting me (I think even you could agree that I have a track record of favoring constructive dialogue here), but because I think this is an area that really needs improvement. There is a clear process in place: either ask the OTP team nicely to do this - understanding that they have a ton of higher-priority tasks on their plates, or figure out how to build a documentation patch yourself. The bar of acceptance is much lower for documentation patches. Having patched the docs myself, my own experience is that the biggest frustration is dealing with FOP - it runs out of heap and throws exceptions everytime I try to build the whole docs. That, and of course the problem of writing good documentation in the first place. I've made this suggestion to you before - I even suggested that your documentation patch would be universally appreciated (provided it improves on the current docs of course). You have declined. In my book, this leaves you with the "ask nicely" option. That, or you can try to pay someone money to make your itch their priority. Erlang Solutions actually offers that service (and no, I work for neither them nor Ericsson). But please consider this: seq_trace was never intended as a general implementation of Lamport clocks. It is a tracing facility, and the purpose, as I've been trying to tell you, was to allow for tracing selectively on e.g. a call setup sequence in a phone switch*. You shouldn't rely on it for any other purpose. The reference to "beta status" remains there, I would guess, because people still find the API and descriptions confusing. I think that confusion goes much deeper than just lacking a reference to Lamport clocks. Maybe the API should really change into something that no longer looks very much like Lamport clocks? That seq_trace is completely independent of the built-in tracing is also misleading. While you can run seq_trace without tracing enabled, the match specifications for the trace BIFs have support for seq_trace. The seq_trace functionality is also at least partly implemented in erl_bif_trace.c (Actually, not requiring tracing is a great feature, not least if it's to be used for other purposes - which it shouldn't in its present state - but one can turn it around and note that one of the biggest drawbacks of erlang's tracing support is that only one tracer per process at a time can be supported. This makes it hard to make broader use of tracing.) It might well be that the best way forward would be to provide a clean API for Lamport clocks - and seq_trace is pretty close to that already - and then rework the sequence tracing support (which is badly needed, and shamefully under-used) to make clear and intuitive use of it. This could achieve two good things: making Lamport clocks available in Erlang (not just by accident, but as a documented and supported feature), and making sequence tracing more intuitive to use. But now we're looking at a slightly larger and more difficult task. I venture a guess that this is very close to what you've been trying to say, but your refusal to accept that the seq_trace API was not meant to implement Lamport clocks, and might well depart from them if it makes it more fit for purpose, reduces the chances that your suggestions for improvement will actually be accepted. You need to accept the whole picture. * Just to expand a bit on what the challenge is: in a phone switch, or similar, operating under potentially high load and with fairly extreme availability demands, running a sequence trace is not primarily a question of ordering concurrent events - for practical purposes, the trace timestamps are often sufficient for that - the big problem is to selectively triggering trace output at just the right places and turning it off as soon as it is no longer needed. In this respect, Lamport clocks are one brilliant and convenient technique that can be built on to solve part of the problem. The thesis I linked to lists a few other approaches. (And no, the failure to mention Erlang's support for real-time tracing in that thesis is more likely to be due to internal rivalry, or simply lack of interest in technologies that they can't use anyway - Erlang due to past policy issues and AXE since it's a legacy system using a weird programming language). BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From ulf@REDACTED Tue Jun 5 10:19:54 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 10:19:54 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: On 5 Jun 2012, at 09:47, Michael Turner wrote: > All I know is: if that's what you're doing, that's > what you should call it. An alternative, as I expanded on in my last email, which I sent just before reading this one, is that perhaps they should be doing something else instead. Seq_trace is not well understood for the purpose it was intended for. It should perhaps be reworked entirely. If so, it does seem like a good idea to change seq_trace to 'lamport', make it clearly a generic implementation of Lamport clocks, for whatever purpose. This could be done today. As it affects the VM, it should be an EEP, I think. The initial implementation of 'lamport' could be completely based on seq_trace, but renaming functions and changing the documentation so that it clearly references relevant papers and illustrates how it could be used. It ought to be perfect for e.g. "model tracing", similar to what 'et' does (another API that is woefully under-used since the documentation turns people away). Code could be inserted as "executable comments" basically indicating "we are now in in the model". With such code in place, one could do quite sophisticated visualizations of a running system It doesn't seem like such a module ought to have a system_tracer() function. Rather, tracing on Lamport clock events should then be more intuitively integrated into the tracing BIFs (halfway there already). Actually, 'et' handles seq_trace events and processes them for use in the visualization. However, the documentation doesn't make the connection. The seq_trace events are included in the type signatures, but never mentioned elsewhere. This is interesting. It seems as if 'et' could rely entirely on seq_trace. Instead, it more or less mandates global tracing. Why? > And speaking of what to call things: I don't think you should still be > calling seq_trace "beta", if (as Ulf says) it originated ca 1997. I'd > do the interface differently, but the more important thing to me now > is stability and correctness. I agree this is a problem, like with parameterized modules. You shouldn't have beta or unsupported features lingering for years. Either make them supported or remove them and possibly provide something better. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Tue Jun 5 12:03:02 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 5 Jun 2012 19:03:02 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: Ulf, you're saying *I'm* off-topic, in how I'm responding to you? Have you already forgotten how YOU joined this conversation? Here it is again: Me: > I started using seq_trace. It has its own documentation problems, of > course. For example: since seq_trace is an implementation of Lamport > clocks, you should *say* somewhere (like, in the first paragraph, > maybe?) that it's an implementation of Lamport clocks. Don't make it > sound like your own invention. You: "Actually, I believe it is more correct to say that it's based on the "forlopp trace" thst exists in Ericsson's AXE switches (using the proprietary language PLEX). Whether forlopp trace was based on Lamport clocks, I couldn't say. I don't know when it was introduced in the AXE (Lamport published his paper in 1978, the same year that the first digital AXE was taken into service), and to what extent it was informed by Lamport's work." But now, when I say this: >> We're talking about whether the seq_trace documentation >> (especially given how confusing it is to begin with) should mention >> that it implements Lamport clocks You respond with this: "*You* are talking about that, ...." ALL of the apparent topic drift on this particular point, since your first response to me, is from you. I've tried to stay on the point: if you're using Lamport clocks, exposing them in an API, *admitting* (as Kenneth Lundin did, on this list, in 2007) that you're using them, then the documentation should say so. To save people time, if nothing else. But especially so that people who are looking for a Lamport clock implementation in Erlang will be able to find it easily in searches. > I claim that seq_trace implements Lamport clocks "by > accident", and that it was not the original purpose, nor a complete > solution to the problem. (a) You have no evidence of this, but worse -- (b) all of the evidence I've been able to find is on the other side: open-literature publications by members of the Erlang group, showing they are familiar, perhaps even intimate, with Lamport's work. As to the "original purpose" of "accidentally" implementing Lamport clocks in seq_trace, what, pray tell, WAS the original purpose of an "accidental" implementation of them, if it wasn't basically the same as Lamport's purpose? Just to have some intriguing pairs of numbers to look at, in otherwise-boring traces? > or figure out how to build a documentation patch yourself. If in fact you represent the Ericsson point of view on this issue, I'd be wasting my time: it'll be rejected for the reason that "we invented that independently." I'd like to know what the Ericsson point of view is, before I try something that might be futile. > Also, if you would be less focused on defending your position, we > might actually be able to agree on something. If you would be less focused on defending a position that I'm not sure the Erlang group actually takes, we might actually be able to agree on something. So far, only Robert Virding has spoken up on this issue. I pointed out to him that he's coauthor on a 1993 publication that cites Lamport's paper -- long before the 1997-8 timeframe you give for seq_trace requirements acquisition and implementation. He hasn't responded since. > ... dealing with FOP - it runs out of heap and throws exceptions > everytime I try to build the whole docs. Yes. If I want to make a four-word change, I'll might have to spend all day getting set up. That was my last experience with trying to get the Erlang/OTP documentation toolchain working (admittedly on a slightly unusual platform.) I'm used to wikis: learn a little markup, and you can fix any problem almost instantly. Right now, there's someone in Ericsson who could make my proposed four-word change almost instantly, since they've got it all set up already. > [purpose] was to allow for tracing selectively on e.g. > a call setup sequence in a phone switch*. You shouldn't rely on it for > any other purpose. *Boink* The beta-warning in the seq_trace documentation says: "... the programming interface still might undergo minor adjustments (possibly incompatible) based on feedback from users." What little feedback I can find in public suggests that people are only too happy to have discovered that seq_trace implements Lamport clocks. And yet you think I shouldn't "rely" on their continued presence, because dropping them (or changing them to something else incompatible) is a possible "minor adjustment" to the "interface"? My idea of a "minor adjustment" would be to define SeqTraceInfo (and maybe the timestamp) as a record rather than a bare tuple. I would suggest some such change, except it's really not that important compared to backward compatibility for other users of seq_trace out there. > your refusal to accept that the seq_trace API was not > meant to implement Lamport clocks, and might well depart from them If you mean "might well depart" in the *future*, how is that a "minor interface adjustment"? SerialInfo - the timestamp for Lamport logical clocks - is all over the API. No, that's a "major algorithm change." If you mean "might well depart" *now*, why did Kenneth Lundin say it implements them? If he's wrong, why did nobody in Ericsson correct him? Lamport clocks are Lamport clocks, regardless of "intent." > That seq_trace is completely independent of the built-in tracing is also > misleading. WTF? Where did I say it was "completely independent"? Where did anyone say it was? And it seems pretty "built-in" to me. I mean, how do you do seq_trace (including the Lamport clock counter increments) without a hack to Send? That's not very far above bare metal. > ... one of the biggest drawbacks > of erlang's tracing support is that only one tracer per process at a time > can be supported. *Boink*. seq_trace is *part* of "erlang's tracing support". In what way is it limited to "one tracer *per* process at a time"? It's limited to one *tracer process* at a time, but, if I'm reading the documentation right (*sigh*), it can selectively trace an arbitrary number of tokens through any number of processes they might propagate into. If you want individual tracer processes on (say) a per-token basis, or on a per-process basis, I guess you could just use your single tracer process as a dispatcher for the others. A few more lines of code, no biggie. > (And no, the failure to mention Erlang's support for real-time > tracing in that thesis is more likely to be due to internal rivalry, > or simply lack of interest in technologies that they can't use > anyway - Erlang due to past policy issues and AXE since it's > a legacy system using a weird programming language). *Double boink*. The thesis was submitted in late 2008. I see Kenneth Lundin affirming, in mid-2007, that seq_trace implements Lamport clocks, for an Erlang/OTP that had been open sourced for years by then. You're saying it's possible that the author of that thesis might not have been able to find out about seq_trace (or Lamport clocks), or was not able to use seq_trace (or its Lamport clocks), because of "internal rivalry" or "past policy issues"??? Things must be a lot weirder in there than I ever suspected. And yet I'm supposed to be confident that if I submit a patch to the seq_trace documentation informing users that it implements Lamport clocks, it's very likely to be taken up? Please tell me you really meant something else by that last, but just totally garbled it. -michael turner On Tue, Jun 5, 2012 at 4:52 PM, Ulf Wiger wrote: > > On 5 Jun 2012, at 08:01, Michael Turner wrote: > >> We're not talking about whether AXE was "the main source of >> inspiration". We're talking about whether the seq_trace documentation >> (especially given how confusing it is to begin with) should mention >> that it implements Lamport clocks, since, after all, (a) it does, and >> (b) lots of people who do concurrent programming know what Lamport >> clocks are already -- including about half-a-dozen in the Erlang group >> (at least if what they cite in their publications is any indication.) > > *You* are talking about that, and in the process, firing off rants > insulting people who have worked much harder than you to make > Erlang a solid platform, implying that they are lazy, possibly dishonest > and ignorant. I claim that seq_trace implements Lamport clocks "by > accident", and that it was not the original purpose, nor a complete > solution to the problem. It may be "all it does", but it's not all it should > do. I don't say that out of ignorance, but because I still recall many of > the discussions before, during and after the implementation of > seq_trace. > > Meanwhile, you tried to submit a patch once, and since it went > badly, you can't be bothered to submit a suggestion for how to fix > this documentation bug that upsets you (but apparently not that > many others, judging by the lack of sympathetic uproar). > > Still, you are making some good points. If you would dispense with > the sarcasm, chances are people would be more inclined to listen. > Also, if you would be less focused on defending your position, we > might actually be able to agree on something. > > The reason I reply to you is not because I want to get back at you > for insulting me (I think even you could agree that I have a track > record of favoring constructive dialogue here), but because I think > this is an area that really needs improvement. > > There is a clear process in place: either ask the OTP team nicely to > do this - understanding that they have a ton of higher-priority tasks > on their plates, or figure out how to build a documentation patch yourself. > The bar of acceptance is much lower for documentation patches. > Having patched the docs myself, my own experience is that the biggest > frustration is dealing with FOP - it runs out of heap and throws exceptions > everytime I try to build the whole docs. That, and of course the problem > of writing good documentation in the first place. > > I've made this suggestion to you before - I even suggested that your > documentation patch would be universally appreciated (provided it > improves on the current docs of course). You have declined. In my book, > this leaves you with the "ask nicely" option. That, or you can try to pay > someone money to make your itch their priority. Erlang Solutions actually > offers that service (and no, I work for neither them nor Ericsson). > > But please consider this: seq_trace was never intended as a general > implementation of Lamport clocks. It is a tracing facility, and the purpose, > as I've been trying to tell you, was to allow for tracing selectively on e.g. > a call setup sequence in a phone switch*. You shouldn't rely on it for > any other purpose. The reference to "beta status" remains there, I would > guess, because people still find the API and descriptions confusing. > I think that confusion goes much deeper than just lacking a reference > to Lamport clocks. Maybe the API should really change into something > that no longer looks very much like Lamport clocks? > > That seq_trace is completely independent of the built-in tracing is also > misleading. While you can run seq_trace without tracing enabled, the > match specifications for the trace BIFs have support for seq_trace. > The seq_trace functionality is also at least partly implemented in > erl_bif_trace.c > > (Actually, not requiring tracing is a great feature, not least if it's to be > used for other purposes - which it shouldn't in its present state - but > one can turn it around and note that one of the biggest drawbacks > of erlang's tracing support is that only one tracer per process at a time > can be supported. This makes it hard to make broader use of tracing.) > > It might well be that the best way forward would be to provide a clean > API for Lamport clocks - and seq_trace is pretty close to that already - > and then rework the sequence tracing support (which is badly needed, > and shamefully under-used) to make clear and intuitive use of it. > This could achieve two good things: making Lamport clocks available > in Erlang (not just by accident, but as a documented and supported > feature), and making sequence tracing more intuitive to use. > > But now we're looking at a slightly larger and more difficult task. > > I venture a guess that this is very close to what you've been trying > to say, but your refusal to accept that the seq_trace API was not > meant to implement Lamport clocks, and might well depart from them > if it makes it more fit for purpose, reduces the chances that your > suggestions for improvement will actually be accepted. You need to > accept the whole picture. > > * Just to expand a bit on what the challenge is: in a phone switch, > or similar, operating under potentially high load and with fairly > extreme availability demands, running a sequence trace is not > primarily a question of ordering concurrent events - for practical > purposes, the trace timestamps are often sufficient for that - > the big problem is to selectively triggering trace output at just > the right places and turning it off as soon as it is no longer needed. > In this respect, Lamport clocks are one brilliant and convenient > technique that can be built on to solve part of the problem. The > thesis I linked to lists a few other approaches. > > (And no, the failure to mention Erlang's support for real-time > tracing in that thesis is more likely to be due to internal rivalry, > or simply lack of interest in technologies that they can't use > anyway - Erlang due to past policy issues and AXE since it's > a legacy system using a weird programming language). > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From michael.eugene.turner@REDACTED Tue Jun 5 12:42:27 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 5 Jun 2012 19:42:27 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: Ulf, when I write "seq_trace implements Lamport clocks", please try to read it as you would "This ANSI Standard C compiler implements IEEE arithmetic." That doesn't mean "this ANSI Standard C compiler *is* IEEE arithmetic." If people want to *add* new API elements to seq_trace, fine. I'd could go for a different way of calling stuff, and of representing the relevant data structures. Not important, but nice. If people want to add new features to seq_trace, fine. Maybe it could use some. Riak uses vector clocks, which are based on scalar (Lamport) clocks. They can be problematic, since vector clocks involve n-length vectors, where n is the number of processes concerned. Still, if somebody wanted to add *that* to seq_trace, I'd be cool with it. But changing "seq_trace" to "lamport" is (a) semantically wrong, since seq_trace *implements* Lamport clocks but is not *simply* Lamport clocks, and (b) pragmatically wrong, since it breaks any existing code that depends on seq_trace, and also breaks anything out there that has implemented a module called "lamport" independently. By the way, I can't figure out why it's called "sequential tracing". If somebody told me, "it has to be called 'X tracing', solve for X," I'd say, "X = 'parallel'", not "X = 'sequential'." Does the "sequential" refer to the fact that the (single) tracer process receives a stream of events? OK, but ... that's not the important thing -- precisely because those events didn't necessarily happen in the order received, in *real* time. (Whatever that is - Lamport points out that understanding of the theory of relativity gave him some insights into this problem). That's the point of having logical clocks like Lamport's - to help sort out chronology -- and, you hope -- causality -- to the extent possible when you can't rely 100% on real-time clocks. Which brings up another point, already raised by Scott Fritchie, here: http://erlang.org/pipermail/erlang-questions/2007-May/026822.html and not adequately addressed in Kenneth Lundin's reply, here: http://erlang.org/pipermail/erlang-questions/2007-May/026827.html Scott writes of seq_traces real-time (seconds/milliseconds/microseconds) timestamp: "Inviso uses the (optional) timestamp, but that's the erlang:now() value, and even an NTP time sync may not be good enough for a busy system to avoid bogus event ordering. I have enough problems with NTP on or lab machines -- it shouldn't be hard, but apparently it is, because their NTP daemons aren't running 1 time in 8. "Don't get me started about time drift in Linux virtual machines, VMware and Xen both. {sigh}" Yes. And don't get ME started about a terrestrial node drifting out of sync with one that's orbiting at velocities where relativistic effects start to add up. Scott makes good points, but the documentation for seq_trace carries no cautionary notices about relying on the real-time timestamps it reports. This seems an odd omission to me, since seq_trace would seem to be especially useful in cases where real-time clocks are unreliable. You could even dispense with real-time timestamps in seq_trace, and what's left would still have a substantial raison d'etre. -michael turner On Tue, Jun 5, 2012 at 5:19 PM, Ulf Wiger wrote: > > On 5 Jun 2012, at 09:47, Michael Turner wrote: > >> All I know is: if that's what you're doing, that's >> what you should call it. > > An alternative, as I expanded on in my last email, which I sent just > before reading this one, is that perhaps they should be doing something > else instead. Seq_trace is not well understood for the purpose it was > intended for. It should perhaps be reworked entirely. > > If so, it does seem like a good idea to change seq_trace to 'lamport', > make it clearly a generic implementation of Lamport clocks, for > whatever purpose. > > This could be done today. As it affects the VM, it should be an EEP, I think. > The initial implementation of 'lamport' could be completely based on > seq_trace, but renaming functions and changing the documentation so > that it clearly references relevant papers and illustrates how it could be > used. It ought to be perfect for e.g. "model tracing", similar to what 'et' > does (another API that is woefully under-used since the documentation > turns people away). Code could be inserted as "executable comments" > basically indicating "we are now in in the model". With such > code in place, one could do quite sophisticated visualizations of a > running system > > It doesn't seem like such a module ought to have a system_tracer() > function. Rather, tracing on Lamport clock events should then be > more intuitively integrated into the tracing BIFs (halfway there already). > > Actually, 'et' handles seq_trace events and processes them for use in > the visualization. However, the documentation doesn't make the > connection. The seq_trace events are included in the type signatures, > but never mentioned elsewhere. > > This is interesting. It seems as if 'et' could rely entirely on seq_trace. > Instead, it more or less mandates global tracing. Why? > >> And speaking of what to call things: I don't think you should still be >> calling seq_trace "beta", if (as Ulf says) it originated ca 1997. ?I'd >> do the interface differently, but the more important thing to me now >> is stability and correctness. > > I agree this is a problem, like with parameterized modules. You shouldn't > have beta or unsupported features lingering for years. Either make them > supported or remove them and possibly provide something better. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From ulf@REDACTED Tue Jun 5 13:56:53 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 13:56:53 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> On 5 Jun 2012, at 12:03, Michael Turner wrote: > I've tried to stay on the point: if > you're using Lamport clocks, exposing them in an API, *admitting* (as > Kenneth Lundin did, on this list, in 2007) that you're using them, > then the documentation should say so. I have agreed that it wouldn't hurt to add that to the existing documentation, but have also argued that one needs to remember the *purpose* of seq_trace, and discuss whether the current API is the right one, and what changes to the documentation would best help users to make use of it. It could well be that such a process would result in an API documentation that *does not* expose Lamport clocks, or (as I suggested), creates a separate component that exposes Lamport clocks in a more obvious and generic way. > To save people time, if nothing > else. But especially so that people who are looking for a Lamport > clock implementation in Erlang will be able to find it easily in > searches. If that is the purpose, then creating a separate 'lamport' module would be a much better solution, obviously. > >> I claim that seq_trace implements Lamport clocks "by >> accident", and that it was not the original purpose, nor a complete >> solution to the problem. > > (a) You have no evidence of this, I was there, remember? With 'by accident' I mean that they could have solved it differently, and then the API would not have exposed Lamport clocks, and they would still have fulfilled the requirement. I'm not saying they didn't know they were using Lamport clocks. I'm saying it was not what the customer asked for, and the man page, such as it is, reflects what the customer had ordered. I guess a better way of putting it is that it was coincidental. > As to the "original purpose" of "accidentally" implementing Lamport > clocks in seq_trace, what, pray tell, WAS the original purpose of an > "accidental" implementation of them, if it wasn't basically the same > as Lamport's purpose? Just to have some intriguing pairs of numbers to > look at, in otherwise-boring traces? What the original purpose was is exactly what I have tried to tell you. I won't repeat it here. I was not, however, present when Leslie Lamport started thinking about Lamport clocks, so I can't speak from own experience. But as you mentioned yourself, he traces it back to a different problem: http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#time-clocks "The origin of this paper was a note titled The Maintenance of Duplicate Databases by Paul Johnson and Bob Thomas. I believe their note introduced the idea of using message timestamps in a distributed algorithm. [?] Because Thomas and Johnson didn't understand exactly what they were doing, they didn't get the algorithm quite right; their algorithm permitted anomalous behavior that essentially violated causality. I quickly wrote a short note pointing this out and correcting the algorithm. [?] "It didn't take me long to realize that an algorithm for totally ordering events could be used to implement any distributed system. A distributed system can be described as a particular sequential state machine that is implemented with a network of processors. The ability to totally order the input requests leads immediately to an algorithm to implement an arbitrary state machine by a network of processors, and hence to implement any distributed system. So, I wrote this paper, which is about how to implement an arbitrary distributed state machine. As an illustration, I used the simplest example of a distributed system I could think of--a distributed mutual exclusion algorithm." My read on that: he didn't originally set out to solve the problem of capturing sequence traces in a real-time system, but noted after a while that his proposed solution was extremely general. The OTP team could have set out to implement sequence tracing, decided to do it using Lamport clocks, then realizing that the implementation could easily be generalized, and changed the API and documentation accordingly. This is not what happened. It could still happen. >> or figure out how to build a documentation patch yourself. > > If in fact you represent the Ericsson point of view on this issue, I'd > be wasting my time: it'll be rejected for the reason that "we invented > that independently." I'd like to know what the Ericsson point of view > is, before I try something that might be futile. I don't represent Ericsson, and that is not at all what I have been saying. > So far, only Robert Virding has spoken up on this issue. I > pointed out to him that he's coauthor on a 1993 publication that cites > Lamport's paper -- long before the 1997-8 timeframe you give for > seq_trace requirements acquisition and implementation. He hasn't > responded since. What Robert wrote was that he was not part of the team that implemented seq_trace. He also doesn't represent Ericsson (anymore). >> your refusal to accept that the seq_trace API was not >> meant to implement Lamport clocks, and might well depart from them > > If you mean "might well depart" in the *future*, how is that a "minor > interface adjustment"? SerialInfo - the timestamp for Lamport logical > clocks - is all over the API. No, that's a "major algorithm change." The man page speaks of minor adjustments. I argue that one should perhaps consider a major overhaul. For B/W compatibility, it would be better to introduce a new, better API. From your other mail: > But changing "seq_trace" to "lamport" is > > (a) semantically wrong, since seq_trace *implements* Lamport clocks > but is not *simply* Lamport clocks, > > and > > (b) pragmatically wrong, since it breaks any existing code that > depends on seq_trace, and also breaks anything out there that has > implemented a module called "lamport" independently. I didn't suggest making an incompatible change to seq_trace, but basing the initial implementation of 'lamport' on the seq_trace implementation. The unprefixed namespace belongs to OTP. This matter has been debated many times. It's not a very good setup, but so it is. > If you mean "might well depart" *now*, why did Kenneth Lundin say it > implements them? If he's wrong, why did nobody in Ericsson correct > him? Lamport clocks are Lamport clocks, regardless of "intent." So they are, but that doesn't necessarily mean that their use should be commited to the API and highlighted in the documentation. Note - this is meant in general terms, as I didn't object to adding a reference to Lamport in the current seq_trace documentation. An example: the Erlang docs describe how the VM samples the length of the receiver's message queue when sending a message, and penalizes the sender with extra reductions if the queue length exceeds a certain threshold. This was a cool way to implement poor-man's flow control in a single-core system, but in a many-core system, it's a pretty bad idea. As it's been committed to the documentation, it is harder to change now that it is arguably more of a burden than a feature. This is an example of why it is so important to go back to the original purpose, and ask, as Bjarne was wont to say "What's the bloody problem?" What problem did we originally set out to solve, and what changes might be needed to ensure that we solve that problem well - and keep solving it well based on where we're heading? Some things are better not added to the reference manual. > >> That seq_trace is completely independent of the built-in tracing is also >> misleading. > > WTF? Where did I say it was "completely independent"? Where did anyone > say it was? It's in the seq_trace man page. First paragraph of Description, second sentence. I didn't claim you said it, but could have been clearer about that - apologies. >> ... one of the biggest drawbacks >> of erlang's tracing support is that only one tracer per process at a time >> can be supported. > > *Boink*. seq_trace is *part* of "erlang's tracing support". In what > way is it limited to "one tracer *per* process at a time"? Erlang's tracing support allows only one tracer (process) per process. This is a well-known and documented limitation. The seq_trace system_tracer allows only one per node. A generic lamport clock implementation has no need for such limitations. And while you could use seq_trace for other purposes, these issues become distracting. As it is, seq_trace is caught somewhere in the middle. It is a fairly nice implementation of Lamport clocks, but not really intended for, or entirely fit for, use as a generic Lamport clock implementation. As a solution for sequence/transaction/forlopp tracing, it is half-baked and slightly confusing. It kindof works, but few people understand it well enough to use it. Even OTP doesn't necessarily use it in all places where it would fit. This indicates that the seq_trace API and docs could evolve in either of two different directions - or be reworked, split, and made more inutitive, addressing both issues at the same time. >> (And no, the failure to mention Erlang's support for real-time >> tracing in that thesis is more likely to be due to internal rivalry, >> or simply lack of interest in technologies that they can't use >> anyway - Erlang due to past policy issues and AXE since it's >> a legacy system using a weird programming language). > > *Double boink*. The thesis was submitted in late 2008. I see Kenneth > Lundin affirming, in mid-2007, that seq_trace implements Lamport > clocks, for an Erlang/OTP that had been open sourced for years by > then. You're saying it's possible that the author of that thesis might > not have been able to find out about seq_trace (or Lamport clocks), or > was not able to use seq_trace (or its Lamport clocks), because of > "internal rivalry" or "past policy issues"??? Not sure what you mean by "boink". I read it as an insult, but perhaps it's just your way of expressing surprise? > Things must be a lot weirder in there than I ever suspected. Indeed. But that's definitely a side track. I just mentioned in passing that you'd think a thesis about *real-time tracing* (not Lamport clocks, although it mentions them, and other techniques) should mention the two best exponents of real-time tracing in the company. Again note the bigger picture here - *tracing*. Global ordering of events is one challenge. There are others. > And yet I'm supposed to be confident that if I submit a patch to > the seq_trace documentation informing users that it implements > Lamport clocks, it's very likely to be taken up? If it improves the documentation, yes, of course. This was seconded by Gustav Simonsson, who works in the OTP team. He even suggested where to put it. The OTP team is by no means hampered by any policy (esp. *past* policy) decisions not to mention Erlang. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From ulf@REDACTED Tue Jun 5 14:08:33 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 14:08:33 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> Message-ID: On 5 Jun 2012, at 12:42, Michael Turner wrote: > Ulf, when I write "seq_trace implements Lamport clocks", please try to > read it as you would "This ANSI Standard C compiler implements IEEE > arithmetic." That doesn't mean "this ANSI Standard C compiler *is* > IEEE arithmetic." In that particular example, adding it to the documentation is an important service to the user. Indeed, it might event be part of the requirements for the compiler. > By the way, I can't figure out why it's called "sequential tracing". > If somebody told me, "it has to be called 'X tracing', solve for X," > I'd say, "X = 'parallel'", not "X = 'sequential'." Does the > "sequential" refer to the fact that the (single) tracer process > receives a stream of events? It is actually wrong, IMHO. It should be 'sequence tracing'. The problem was how to dynamically trigger a trace of e.g. one single call setup flow in a system that handled hundreds of calls per second. What has usually been done is that you turn on tracing on just about everything, then push a single call through the system. Obviously, this doesn't work in a live system, so users (used to similar tracing support in the AXE) asked for a way to trace on a single sequence of events, as it triggered various activity in the system. The huge, overshadowing problem for people needing to trace on a live system, is that you have to be really careful that whatever trace you turn on doesn't kill the system you are trying to study. This is why any such tracing support needs to be extremely well thought out, and intuitive for the people who are expected to use it. We are not there today, and the docs need improving, but those who do need to understand what the intent of the component is in the first place. > Scott makes good points, but the documentation for seq_trace carries > no cautionary notices about relying on the real-time timestamps it > reports. This seems an odd omission to me, since seq_trace would seem > to be especially useful in cases where real-time clocks are > unreliable. You could even dispense with real-time timestamps in > seq_trace, and what's left would still have a substantial raison > d'etre. Agreed, including the "don't get me started?" part. ;-) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From francesquini@REDACTED Tue Jun 5 14:25:10 2012 From: francesquini@REDACTED (Emilio De Camargo Francesquini) Date: Tue, 5 Jun 2012 14:25:10 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> Message-ID: Hello, About a month ago I wrote something similar to be able to "see", not the links/monitors/genealogy of the Erlang processes, but rater the communication graph between them. For this particular example the internal graph of communication of CouchDB. If there is interest I can upload the code to Github... This video shows the processes communication graph for CouchDB running in a machine with 24-cores. CouchDB was exercised using YCSB and the video was generated using Ubigraph. I've posted the video here: http://www.youtube.com/watch?v=2MO6Uv7KZ7g Regards, Emilio 2012/6/3 Ivan Uemlianin : > On 3 Jun 2012, at 10:11, Ulf Wiger wrote: > >> Drawing inspiration from how people solve coordination problems, >> including how people deal with uncertainty and inconsistencies, >> can be a great help when programming Erlang. > > Very good suggestion. > > What do they know of Erlang, who only Erlang know? (with apologies to Kipling) > > Ivan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From francesco@REDACTED Tue Jun 5 14:51:02 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 05 Jun 2012 13:51:02 +0100 Subject: [erlang-questions] Erlang Factory Lite Moscow - June 22nd @ 6pm Message-ID: <4FCE00B6.9050700@erlang-solutions.com> Together with Yandex, Erlang Solutions are organising a Moscow Erlang Factory Lite. It is going to be one of the biggest Erlang Factory Lites to date, with over 70 people registered in the first 24hours! Mark the 22nd of June in your calendars. We have some great talks, but are looking for more! Do you want to give a 30-minute presentations, either in English or in Russian on Erlang-related matters? We welcome submissions from developers, system designers, testers, community leaders, inventors, CTOs, evangelists, students, researchers and entrepreneurs alike. Participants at the Erlang Factory want to hear about how Erlang/OTP is used real-world scenarios, talks about tricks, tools and applications which increase productivity or enable developers to write better code. If you are interested in speaking at the first Moscow Erlang Factory Lite, please submit your talk here: http://www.erlang-factory.com/conference/Moscow2012/submit_talk For more information on the venue and to register, use this form: http://www.erlang-factory.com/conference/Moscow2012 Cheers, Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From roe.adrian@REDACTED Tue Jun 5 15:04:34 2012 From: roe.adrian@REDACTED (Adrian Roe) Date: Tue, 5 Jun 2012 14:04:34 +0100 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> Message-ID: <7830EA23-A5DC-4B56-A8FD-206EF93D0B09@gmail.com> Awesome video. I'd definitely be interested in the code behind it. Adrian Sent from my iPhone On 5 Jun 2012, at 13:25, Emilio De Camargo Francesquini wrote: > Hello, > > About a month ago I wrote something similar to be able to "see", not > the links/monitors/genealogy of the Erlang processes, but rater the > communication graph between them. For this particular example the > internal graph of communication of CouchDB. > > If there is interest I can upload the code to Github... > > This video shows the processes communication graph for CouchDB running > in a machine with 24-cores. CouchDB was exercised using YCSB and the > video was generated using Ubigraph. I've posted the video here: > > http://www.youtube.com/watch?v=2MO6Uv7KZ7g > > Regards, > > Emilio > > 2012/6/3 Ivan Uemlianin : >> On 3 Jun 2012, at 10:11, Ulf Wiger wrote: >> >>> Drawing inspiration from how people solve coordination problems, >>> including how people deal with uncertainty and inconsistencies, >>> can be a great help when programming Erlang. >> >> Very good suggestion. >> >> What do they know of Erlang, who only Erlang know? (with apologies to Kipling) >> >> Ivan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Jun 5 15:11:54 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 5 Jun 2012 15:11:54 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> Message-ID: Hi Emilio, For sure there is an interest. On Jun 5, 2012, at 2:25 PM, Emilio De Camargo Francesquini wrote: > http://www.youtube.com/watch?v=2MO6Uv7KZ7g Well done. Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jun 5 16:09:41 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 5 Jun 2012 16:09:41 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) Message-ID: On Tue, Jun 5, 2012 at 9:52 AM, Ulf Wiger wrote: > Having patched the docs myself, my own experience is that the > biggest frustration is dealing with FOP - it runs out of heap and > throws exceptions everytime I try to build the whole docs. That, and http://erlang.org/pipermail/erlang-questions/2010-September/053298.html xsltproc (from libxslt) - which you probably have installed - is enough to create man pages and HTML docs. Also, FOP - the last time I used it - made use of Apache Batik and therefore required X11. This makes it harder to run in a headless (server) environment. FOP is only required for generating PDF files. From Sergey_Zhemzhitsky@REDACTED Tue Jun 5 16:35:54 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Tue, 5 Jun 2012 14:35:54 +0000 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CBA649@exch-mbx1.msk.trd.ru> Hi there, I?m developing an erlang port driver in which I have to parse rather complex erlang terms. I?ve found that there are different APIs that can be used, so what is the difference between erl_eterm API (www.erlang.org/doc/man/erl_eterm.html), ei API (http://www.erlang.org/doc/man/ei.html)? What is the best way to parse complex erlang terms in linked in drivers? Best Regards, Sergey _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Jun 5 17:14:56 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 00:14:56 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> Message-ID: On Tue, Jun 5, 2012 at 8:56 PM, Ulf Wiger wrote: > > On 5 Jun 2012, at 12:03, Michael Turner wrote: > >> I've tried to stay on the point: if >> you're using Lamport clocks, exposing them in an API, *admitting* (as >> Kenneth Lundin did, on this list, in 2007) that you're using them, >> then the documentation should say so. > > I have agreed that it wouldn't hurt to add that to the existing > documentation, ... Actually, you've said that it could hurt -- by committing to a decision that you think might be premature. (Somehow. After 15 years.) > but have also argued that one needs to > remember the *purpose* of seq_trace, ... I can't remember any purpose of seq_trace that's not described in the documentation for it. And that documentation says, right there: "Sequential tracing makes it possible to trace all messages resulting from one initial message." There's nothing in Lamport clocks that's in conflict with that purpose. At one point it says: "In the following sections Sequential Tracing and its most fundamental concepts are described." And in the first of those following sections, it says: "The purpose [of the trace token component, Serial, with its Previous and Current] is to uniquely identify each traced event within a trace sequence and to order the messages chronologically and in the different branches if any." Then it goes on to describe "the algorithm" for updating those counters. Not "an algorithm." The algorithm. If Lamport clocks are not part and parcel of the purpose of seq_trace according to the reference manual, then I guess you're privy to some *secret* purpose of seq_trace. > ... and discuss whether > the current API is the right one, and what changes to the > documentation would best help users to make use of it. That need has existed for a while. *Failing* to describe seq_trace as implementing Lamport clocks can only have worked against that need. It means lots of people who might otherwise have had intelligent things to say about the current API and how the documentation could be improved have not even had seq_trace come to their attention. And that's a lot of people. It's potentially everyone interested in Erlang who has also had the relevant education. Is there a textbook out there on distributed systems and MIMD parallel processing that *doesn't* bring up Lamport clocks? > It could well be that such a process would result in an API > documentation that *does not* expose Lamport clocks, or > (as I suggested), creates a separate component that exposes > Lamport clocks in a more obvious and generic way. Hiding useful functionality? Changing the API to break existing code? It may well be that the sky can be turned a nice shade of green, then yellow. You want an "obvious and generic way" to "expose" Lamport clocks? seq_trace already does it (except in being so subtle as to not name them as such in the documentation.) You don't have to use those Lamport clocks in seq_trace if you don't want to. And I can't imagine a way to use Lamport clocks without also doing tracing -- that's practically what they are for. Unless by "tracing" you (narrowly) mean "written output to puzzle over for debugging purpose." MY purpose is testing (the "trace-driven development" of this thread, if anything), and when my tests pass, after the tracer process has seen messaging behavior that conforms to my spec, I don't want to see *anything* on the output. seq_trace as it stands can do that for me. Why should I complain? >> To save people time, if nothing >> else. But especially so that people who are looking for a Lamport >> clock implementation in Erlang will be able to find it easily in >> searches. > > If that is the purpose, then creating a separate 'lamport' module > would be a much better solution, obviously. Oh, yeah, obviously. Look, Lamport clocks exist to trace the behavior of processes, and seq_trace can be used without using its Lamport clocks. Why is separation better than just maintaining backward compatibility, at this point? >> >>> I claim that seq_trace implements Lamport clocks "by >>> accident", and that it was not the original purpose, nor a complete >>> solution to the problem. >> >> (a) You have no evidence of this, > > I was there, remember? No, I don't remember. How can I? I wasn't there. A claim that you independently re-invented something by accident, after years of exposure to co-workers who clearly know what that thing is (if their book is any indication -- and you must have read that book) does not qualify as evidence. And since below you contradict yourself, saying first that Lamport clock behavior was a customer requirement, then saying the OTP had discretion over whether to implement them, I don't have much reason to trust your memory. > With 'by accident' I mean that they could have solved it differently, > and then the API would not have exposed Lamport clocks, and > they would still have fulfilled the requirement. Without exposure of the Lamport clocks in the seq_trace API, there's no reason to implement them in seq_trace in the first place. Don't you understand what they do? > I'm not saying they didn't know they were using Lamport clocks. > I'm saying it was not what the customer asked for, and the man > page, such as it is, reflects what the customer had ordered. You can't have it both ways, Ulf. The man page "reflects" Lamport clocks, so you're saying the customer was asking for Lamport clocks in their tracer (whether they called them that or not). Lamport clocks are cited in the first book on Erlang. Lamport clocks are part of the implementation of Mnesia. So you're saying that the customer ordered a certain behavior, and nobody in the Erlang group recognized that the customer was asking for Lamport clocks? Making it an "accident"? What sense does that make? > I guess a better way of putting it is that it was coincidental. Color me incredulous. >> As to the "original purpose" of "accidentally" implementing Lamport >> clocks in seq_trace, what, pray tell, WAS the original purpose of an >> "accidental" implementation of them, if it wasn't basically the same >> as Lamport's purpose? Just to have some intriguing pairs of numbers to >> look at, in otherwise-boring traces? > > What the original purpose was is exactly what I have tried to > tell you. I won't repeat it here. You have not told me what that purpose putting Lamport clocks in seq_trace was. The purpose? As far as any reasonable reader should be concerned, the document *defines* seq_trace in terms of Lamport clocks -- see my excerpts above. That makes Lamport clocks integral to its purpose. At worst, the reader should be prepared for changes to the *interface*, not the implementation.. You seem to think that seq_trace could have hidden Lamport clocks, when in fact hiding them would only have defeated their purpose in a tracing package. This makes no sense at all. [snip long quote from Lamport]: > My read on that: he didn't originally set out to solve the problem > of capturing sequence traces in a real-time system, but noted > after a while that his proposed solution was extremely general. So what? I'm not crediting Lamport with seq_trace, much less with AXE forlopp's. I'm only seeking credit for Lamport in the seq_trace documentation. Which he deserves. And which we all deserve, since it makes it easier to find his work in Erlang if you already know his work, and easier (upon reading this fact in the seq_trace documentation) to find other people's work where it uses Lamport clocks for various practical purposes, results that might be implemented in Erlang, redounding the the benefit and glory of Erlang/OTP in the process. How does anybody lose? I don't get it. > The OTP team could have set out to implement sequence > tracing, decided to do it using Lamport clocks, then realizing that > the implementation could easily be generalized, and changed > the API and documentation accordingly. First you say (above) that Lamport clocks were a customer requirement. Now you're saying the OTP team had discretion in this matter. It can't be both. > This is not what happened. It could still happen. And I could skate across hell -- when it freezes over. [snip comments about Virding's contribution to this debate.] > The man page speaks of minor adjustments. I argue that one should > perhaps consider a major overhaul. For B/W compatibility, it would > be better to introduce a new, better API. Well, you're free to fork Erlang/OTP and try to sell people on the result. As it is, not having Lamport clocks mentioned explicitly in the seq_trace documentation means that there's basically no customer base to address anyway, since hardly anybody ever found out they were in there. > I didn't suggest making an incompatible change to seq_trace, You've repeatedly suggested it might be desirable. Even in this e-mail. > but basing the initial implementation of 'lamport' on the > seq_trace implementation. But if Lamport clocks subsequently disappear from the seq_trace implementation, as you seem to think should happen, you've created backwards incompatibility. So what's the difference? >> If you mean "might well depart" *now*, why did Kenneth Lundin say it >> implements them? If he's wrong, why did nobody in Ericsson correct >> him? Lamport clocks are Lamport clocks, regardless of "intent." > > So they are, but that doesn't necessarily mean that their use should > be commited to the API and highlighted in the documentation. As far as any reasonable reader should be concerned, the document *defines* seq_trace in terms of Lamport clocks -- see my excerpts above. At worst, the reader should be prepared for changes to the *interface*, not the implementation. > Note - this is meant in general terms, as I didn't object to adding > a reference to Lamport in the current seq_trace documentation. Yes, you did. You openly feared it would overcommit Erlang/OTP to Lamport clocks in the implementation of seq_trace. > An example: the Erlang docs describe how the VM samples the > length of the receiver's message queue when sending a message, > and penalizes the sender with extra reductions if the queue length > exceeds a certain threshold..... > As it's been committed to the documentation, it is harder to change > now that it is arguably more of a burden than a feature. If Erlang/OTP has overcommitted itself on one point, that still says nothing about whether seq_trace also has. I don't see where it does, and I've been using it for months. If you don't want to use seq_trace's Lamport clocks, you don't have to. (I don't -yet.) You will pay for it only in a counter-increment and some copying of those counters on each trace call -- computational costs that are completely overwhelmed, I'm sure, by everything else required to do any tracing at all. As for using Lamport clocks independent of seq_trace *as a debugging tool*, I see no reason why people can't, nor much reason why they should be bothered by the fact that they are using a package originally intended for debug traces -- it won't pose any significant added burden on them, either computationally or in coding keystrokes, over having a separate implementation. (If you can even *have* a separate implementation of Lamport clocks that doesn't basically replicate almost everything seq_trace does.) > This is an example of why it is so important to go back to the > original purpose, and ask, as Bjarne was wont to say "What's > the bloody problem?" What problem did we originally set out to > solve, and what changes might be needed to ensure that we > solve that problem well - and keep solving it well based on where > we're heading? MY problem was that I needed to record a message traffic pattern, and a way to reasonably order those messages in order to establish whether that pattern is canonical for my purposes. seq_trace does that for me. I bet it could also do that job for Riak's vector clocks (if it doesn't already -- and if it does, that's yet another argument for "it ain't broke, so don't fix it.") > Some things are better not added to the reference manual. Give me an argument that this is such a case. A concrete argument, not a handwaving one. >> WTF? Where did I say it was "completely independent"? Where did anyone >> say it was? > > It's in the seq_trace man page. First paragraph of Description, > second sentence. I didn't claim you said it, but could have > been clearer about that - apologies. Yes, if you want to report a bug against the documentation, go ahead. But it might actually be true enough -- i.e., that you could remove other tracing APIs from Erlang and seq_trace would still work just fine. "Completely independent" might be bad writing, but not necessarily *technically* inaccurate. >>> ... one of the biggest drawbacks >>> of erlang's tracing support is that only one tracer per process at a time >>> can be supported. >> >> *Boink*. seq_trace is *part* of "erlang's tracing support". In what >> way is it limited to "one tracer *per* process at a time"? > > Erlang's tracing support allows only one tracer (process) > per process. This is a well-known and documented limitation. I wouldn't know, since the documentation scared me off. As already noted. > The seq_trace system_tracer allows only one per node. An unfortunate limitation. > A generic lamport clock implementation has no need for > such limitations. Gosh, could it be that if the documentation for seq_trace had always said it implemented Lamport clocks, this shortcoming would have come to light much sooner and been remedied long ago? > And while you could use seq_trace for > other purposes, these issues become distracting. I guess it depends on how distractable you are. I find the relative simplicity of seq_trace a source of consolation: for now, it's keeping me out of something Jason described as "a special kind of hell." And I find the existence of 2000+ publications citing Lamport's paper encouraging as well: I can probably use seq_trace to solve a wide variety of testing problems. > As it is, seq_trace is caught somewhere in the middle. It is a > fairly nice implementation of Lamport clocks, but not really > intended for, or entirely fit for, use as a generic Lamport clock > implementation. As a solution for sequence/transaction/forlopp > tracing, it is half-baked and slightly confusing. It kindof works, > but few people understand it well enough to use it. Even OTP > doesn't necessarily use it in all places where it would fit. Yep. Could the fact that it was never openly identified as implementing Lamport clocks explain, in large part, why it has remained obscure? "Oh look, behind the shed: a wheel like the one we're working on now. Huh. What was it doing behind the shed?" > This indicates that the seq_trace API and docs could evolve > in either of two different directions - or be reworked, split, > and made more inutitive, addressing both issues at the > same time. Oh, whatever. Just don't break the current API, OK? [snip] > Not sure what you mean by "boink". I read it as an insult, but > perhaps it's just your way of expressing surprise? Yes. [snip] >> And yet I'm supposed to be confident that if I submit a patch to >> the seq_trace documentation informing users that it implements >> Lamport clocks, it's very likely to be taken up? > > If it improves the documentation, yes, of course. The issues here go straight to the question of what constitutes an "improvement" in this case. > This was seconded by Gustav Simonsson, who works in the OTP > team. He even suggested where to put it. Great. But what's the hangup with just *doing* it? Do you have to first circulate memos among Ericsson's lawyers or something? > The OTP team is by no means hampered by any policy (esp. > *past* policy) decisions not to mention Erlang. Well, I sure hope people are allowed to mention Erlang, if they work on OTP. It could get awkward always having to say "that language" instead. -michael turner From michael.eugene.turner@REDACTED Tue Jun 5 17:30:38 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 00:30:38 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: Right you are, Tuncer. IIRC, I had to do some surgery on the build scripts to circumvent FOP, since I was trying to build in a server environment. I don't remember any obvious command-line option for getting around it. Maybe things are better now. At the time, it seemed a needless complication for those of us who have been (and perhaps still could be) motivated to improve the documentation. But really: just give me a wiki. Give everybody a wiki. Write scripts to update the repo from the wiki diffs, if that's so important. Generally, lower the bar for entry, for those who want to improve the documentation, and the documentation *will* improve. (Yes, there's such a thing as too low; I've policed wikis for spam, it can get tedious. But there are well-known, easily implemented policies that prevent 99.9% of that kind of thing.) -michael turner On Tue, Jun 5, 2012 at 11:09 PM, Tuncer Ayaz wrote: > On Tue, Jun 5, 2012 at 9:52 AM, Ulf Wiger wrote: > >> Having patched the docs myself, my own experience is that the >> biggest frustration is dealing with FOP - it runs out of heap and >> throws exceptions everytime I try to build the whole docs. That, and > > http://erlang.org/pipermail/erlang-questions/2010-September/053298.html > > xsltproc (from libxslt) - which you probably have installed - is > enough to create man pages and HTML docs. Also, FOP - the last time I > used it - made use of Apache Batik and therefore required X11. This > makes it harder to run in a headless (server) environment. FOP is > only required for generating PDF files. From ulf@REDACTED Tue Jun 5 17:46:47 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 17:46:47 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> Message-ID: <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> On 5 Jun 2012, at 17:14, Michael Turner wrote: > No, I don't remember. How can I? I wasn't there. Because I told you so previously in this thread. Whatever factual points you may have raised in this discussion, I've had enough of your sarcasm and insults. Feel free to do with the information as you wish, and good luck either submitting a patch on the documentation or getting someone else to do it for you. /Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Tue Jun 5 18:43:04 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 01:43:04 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> Message-ID: On Wed, Jun 6, 2012 at 12:46 AM, Ulf Wiger wrote: > Whatever factual points you may have raised in this discussion, > I've had enough of your sarcasm and insults. Nice timing, because I've had enough of the contradictions in what you write. Speaking of which, is it really true, as you say, that seq_trace doesn't work between nodes? In "Distribution", the documentation says: "Sequential tracing between nodes is performed transparently .... In order to be able to perform sequential tracing between distributed Erlang nodes, the distribution protocol has been extended (in a backward compatible way). An Erlang node which supports sequential tracing can communicate with an older (OTP R3B) node but messages passed within that node can of course not be traced." Always helps to know what you're talking about, and have actual evidence for what you claim. In this case, I have no evidence, not having tried it. -michael turner From ulf@REDACTED Tue Jun 5 18:55:30 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Jun 2012 18:55:30 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> Message-ID: On 5 Jun 2012, at 18:43, Michael Turner wrote: > Speaking of which, is it really true, as you say, that seq_trace > doesn't work between nodes? It's not true, and I didn't say so. I wrote: > However, ordering may not have been a terribly difficult problem > to manage in AXE, as it was a single-CPU system, never distributed. > When the OTP team implemented similar support, obviously they > had to make something that worked in a distributed system I wrote that the *AXE* was never distributed. As far as I know, it still isn't. And I wrote that OTP *obviously* had to make something that worked in a distributed system. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From parseerror@REDACTED Tue Jun 5 18:59:26 2012 From: parseerror@REDACTED (Ryan Flynn) Date: Tue, 5 Jun 2012 12:59:26 -0400 Subject: [erlang-questions] What is shutdown_error? Message-ID: What is shutdown_error? I can't figure out what to make of shutdown_error; all I can find is a brief mention that it exists in sasl docs. Full error: Supervisor foo_sup had child foo_server started with foo_server:start_link() at <0.379.0> exit with reason killed in context shutdown_error Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Jun 6 00:53:49 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 6 Jun 2012 10:53:49 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On 6/06/2012, at 3:30 AM, Michael Turner wrote: > But really: just give me a wiki. Give everybody a wiki. Write scripts > to update the repo from the wiki diffs, if that's so important. For the sake of suffering humanity, forfend! The next time someone offers me a wiki instead of documentation, I am going to be exceedingly angry. I have never yet found any wikified documentation that was worth printing to wipe my arse with. I do apologise for the vulgarity, but good documentation needs good technical writing and technical writing is a skill that is distinct from programming, although some people can do both well. It needs a *global* view as well as attention to fine detail. > Generally, lower the bar for entry, for those who want to improve the > documentation, and the documentation *will* improve. Someone recently wrote something about the sky turning green then yellow, didn't they? How is *lowering the bar* going to result in anything *improving*? It might get more *voluminous*, but better? What we really need is a small number of good technical writers and money to pay them. From ok@REDACTED Wed Jun 6 01:03:31 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 6 Jun 2012 11:03:31 +1200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> Message-ID: On 6/06/2012, at 12:25 AM, Emilio De Camargo Francesquini wrote: > This video shows the processes communication graph for CouchDB running > in a machine with 24-cores. CouchDB was exercised using YCSB and the > video was generated using Ubigraph. I've posted the video here: > > http://www.youtube.com/watch?v=2MO6Uv7KZ7g (a) When you say "communication graph" what actually do you mean? (b) What does it signify in the movie when nodes go shooting away from th core and disappear off-screen? From hd2010@REDACTED Wed Jun 6 01:50:24 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 01:50:24 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <4FCE9B40.4040405@eonblast.com> On 6/4/12 6:52 AM, Michael Turner wrote: > I started using seq_trace ... an implementation of Lamport clocks. ... Just say it. I would like to try a brief on what Lamport clocks are, based on [1] + [2]. I think that it's freakishly exciting because it obviously is an application of the Special Theory of Relativity, as Lamport mentions. From that, I sense a promise of finally understanding something I run up against ever since tackling Erlang, how to /think/ parallel. Special Relativity really might be the answer to that: it cancels out the physical clock that is unattainable as part of any solution at any rate. Giving it up, in essence skipping this reaching for an objective reference for events, may be the liberating move to cut the knot. In the process I also found a new favored quote: "Systems in which an event can happen before itself do not seem to be physically meaningful." Lamport clocks are simple counters, one per process. They help to order events occurring in parallel processes, with 'false positives' as price for the ease of the algorithm. False positives would be events listed as having happening one after the other while "really" (causally) having occurred "at the same time". This matters less because an order of events can never express with certainty what event /did/ affect which other event, but only which event /did not/ (the later the earlier). And this is not violated when concurrent events are ordered in sequence. What is lost is only the accurate reflection of possibility: ordered by Lamport clocks, some events that look like they possibly could have affected some other (later) events, in reality could not. Lamport clocks work like this: * Every process has a counter. * Before any 'event' it increments it. * Also before sending a message, it increments it. * The process sends its counter value with every message. * The receiving process sets its counter to the greater of itself or the received. * It increments it. * It assigns it to the event of receiving. I found Lamport's original paper [2] much more accessible than usually, and the illustrations he gives (Fig. 1 - 3) are easy to grasp and illuminating. It's really mostly about the 3 1/2 first pages. Writes Lamport: "Acknowledgment. The use of timestamps to order operations, and the concept of anomalous behavior are due to Paul Johnson and Robert Thomas." Best, Henning [1] http://en.wikipedia.org/wiki/Lamport_timestamps [2] http://research.microsoft.com/users/lamport/pubs/time-clocks.pdf -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Wed Jun 6 02:40:55 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 02:40:55 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> Message-ID: <4FCEA717.4050508@eonblast.com> I think from the pyre, this must be salvaged as the actual point I was going for: On 6/4/12 6:52 AM, Michael Turner wrote: > I'm now > doing unit testing on a module as I develop it further, based on > collecting and filtering seq_trace results Is this a common approach, who else does this and how? Using self-written stuff based on seq_trace, or et_collector? > I should make a decision: do I keep building ever more sophisticated > match filtering on top of seq_trace, undoubtedly reinventing wheel > after wheel, or do I bite the bullet and plunge into what Jayson > Vantuyl describes as "hell"? One Open Source loving word regarding the 'hell' quote: > I'm hardly the only one who should be embarrassed: > > "Erlang tracing is a seething pile of pain that involves reasonably > complex knowledge of clever ports, tracing return formats, and > specialized tracing MatchSpecs (which are really their own special > kind of hell). The tracing mechanism is very powerful indeed, but it > can be hard to grasp." > > Obviously, that kind of statement has no place in the official > documentation of a professional product. Honest statements like the one you are citing may be the actual luxury and strong value of Open Source efforts, which are not directly a commercial product. Possible only because they /are/ not a product. For one user, fresh air like this only creates trust and allows to (even very precisely) set expectations and alertness to shortfalls in both documentation and package. That is not to paint the situation rosy, it would be lovely if it was better. But I appreciate the honesty, very much. Best, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Jun 6 04:47:26 2012 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 05 Jun 2012 19:47:26 -0700 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCEA717.4050508@eonblast.com> References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> Message-ID: <4FCEC4BE.8040608@gmail.com> On 06/05/2012 05:40 PM, Henning Diedrich wrote: > > One Open Source loving word regarding the 'hell' quote: >> I'm hardly the only one who should be embarrassed: >> >> "Erlang tracing is a seething pile of pain that involves reasonably >> complex knowledge of clever ports, tracing return formats, and >> specialized tracing MatchSpecs (which are really their own special >> kind of hell). The tracing mechanism is very powerful indeed, but it >> can be hard to grasp." >> >> Obviously, that kind of statement has no place in the official >> documentation of a professional product. > > Honest statements like the one you are citing may be the actual luxury and strong value of Open Source efforts, which are not directly a commercial product. Possible only because they /are/ not a product. For one user, fresh air like this only creates trust and allows to (even very precisely) set expectations and alertness to shortfalls in both documentation and package. > > That is not to paint the situation rosy, it would be lovely if it was better. But I appreciate the honesty, very much. > I agree that honesty in the documentation is much more beneficial, due to it being Open Source, rather than the duplicity that might otherwise be present in corporate documentation ("Politics-Oriented Software Development": Documentation, http://www.kuro5hin.org/story/2005/1/28/32622/4244). The issues mentioned previously in the email thread seem to indicate that Trace-Driven Development would require more documentation and details, especially documentation that is immediately relevant to the beginner. Putting redbug into OTP seems like it would help reduce the learning curve, just since it is the popular approach to Erlang tracing and has existed for some time. The goal with such changes would be to make tracing simpler in a way that discourages the natural programmer reaction to information discovery during interactive debugging (of systems that are not yet live), which is inserting print/logging statements. The tendency to use print statements is a habit from other languages and is often the common denominator when debugging, so making tracing as simple as a print statement (documentation-wise and usability-wise) seems like a sensible goal. - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Wed Jun 6 07:05:14 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 14:05:14 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> Message-ID: Ulf, I misinterpreted this: > The seq_trace system_tracer allows only one [tracer (process)] per node. > > A generic lamport clock implementation has no need for > such limitations. I read this as claiming that seq_trace tracing is limited to a node, which is (now, after a good night's sleep) rather obviously an overstretched interpretation, at best. That said, however, the seq_trace documentation seems to say conflicting things on this point, and I assumed you were working from where it says this: "The system tracer will only receive those trace events that occur locally within the Erlang node. To get the whole picture of a sequential trace that involves processes on several Erlang nodes, the output from the system tracer on each involved node must be merged (off line)." If the documentation *also* says the following, later -- "Sequential tracing between nodes is performed transparently. [C-node part snipped] In order to be able to perform sequential tracing between distributed Erlang nodes, the distribution protocol has been extended (in a backward compatible way). An Erlang node which supports sequential tracing can communicate with an older (OTP R3B) node but messages passed within that node can of course not be traced." ... then it seems very contradictory to me. I haven't tried to multi-node work with seq_trace yet, so I don't quite get any of this. Maybe the two statements can be reconciled (with some fact that's currently unstated?) My big concern right now: I'd like to build on seq_trace, but maybe it's a foundation of shifting sands? It's happened to me before. I started building a graphics interface last year on the assumption that gs was going to be around for a while. Then I noticed I seemed to be taking a long walk on an incompleted pier. I inquired. It turned out that support for it had been abandoned. Without notice in the documentation of deprecation. (That notice has since been added.) -michael turner > I wrote that the *AXE* was never distributed. As far as I know, it > still isn't. Well, then: since the logical clocks of Lamport's scheme address the problem of *physical* clock skew between systems, if AXE was never distributed, then it never needed to implement Lamport clocks, and thus it never implemented the full functionality of seq_trace. So can we finally lay that claim of simultaneous independent invention to rest? It seems to have become some kind of assumption among Erlang insiders, but there's no evidence for it, and (apparently) no motivation at the time to have done it. -michael turner On Wed, Jun 6, 2012 at 1:55 AM, Ulf Wiger wrote: > > On 5 Jun 2012, at 18:43, Michael Turner wrote: > >> Speaking of which, is it really true, as you say, that seq_trace >> doesn't work between nodes? > > It's not true, and I didn't say so. > > I wrote: > >> However, ordering may not have been a terribly difficult problem >> to manage in AXE, as it was a single-CPU system, never distributed. >> When the OTP team implemented similar support, obviously they >> had to make something that worked in a distributed system > > I wrote that the *AXE* was never distributed. As far as I know, it > still isn't. And I wrote that OTP *obviously* had to make something > that worked in a distributed system. > > BR, > Ulf W > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From michael.eugene.turner@REDACTED Wed Jun 6 08:02:30 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 15:02:30 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCEA717.4050508@eonblast.com> References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> Message-ID: On Wed, Jun 6, 2012 at 9:40 AM, Henning Diedrich wrote: > I think from the pyre, this must be salvaged as the actual point I was going > for: > > > On 6/4/12 6:52 AM, Michael Turner wrote: > >> I'm now >> doing unit testing on a module as I develop it further, based on >> collecting and filtering seq_trace results > > > Is this a common approach, who else does this and how? I'd like to know. I'm guessing that one of the big problems I've had with understanding Ulf here is that, for him, tracing is, by definition, a way to collect data about *anomalous* behavior. To me, tracing is "selectively (and only on occasion) collecting data about behavior." Period. You can do whatever you want with that data. The behavior doesn't have to be pathological. In fact, you can use the data as some assurance of correctness - the "occasion" can be running a test suite. Which is to say, the "trace-driven development" of this thread. Is it problematic? Of course it is. I accept the possibility that some of my seq_trace-enabled tests could pass with flying colors only because the delays incurred by recording the trace tokens suppress some race condition that actually does arise in production runs. But, hey, that's testing for you: it can only prove the presence of errors, not their absence. On your point about documentation (which is basically a good one, don't get me wrong): Jason: > "Erlang tracing is a seething pile of pain that involves reasonably > complex knowledge of clever ports, tracing return formats, and > specialized tracing MatchSpecs (which are really their own special > kind of hell). The tracing mechanism is very powerful indeed, but it > can be hard to grasp." Me: > Obviously, that kind of statement has no place in the official > documentation of a professional product. (Which Ulf agrees with!) You write: > Honest statements like the one you are citing may be the actual luxury and > strong value of Open Source efforts, which are not directly a commercial > product. Possible only because they /are/ not a product. For one user, fresh > air like this only creates trust and allows to (even very precisely) set > expectations and alertness to shortfalls in both documentation and package. > > That is not to paint the situation rosy, it would be lovely if it was > better. But I appreciate the honesty, very much. There are three issues here: (1) how honest one should be in a given context, (2) in what style one should couch that honesty, (3) whether Erlang is "directly" a commercial product. Last one first: we hear a lot in this forum about how Erlang/OTP falls short in one way or another because of "priorities". These priorities are those of a company, Ericsson, that has ways of making money off of Erlang/OTP. Erlang/OTP is thus an open source effort *and* a commercial product. You can say it's not "directly" commercial, because they don't sell Erlang/OTP distros. But an economist would say, "It's just the degenerate case of the shelf price being zero." There are plenty of software products where the non-zero shelf-price is negligible compared to all the other costs to the user -- support contracts, buying add-ins, employing consultants to configure the thing, and to reconfigure it, and so on. Being a product means Erlang/OTP gets the best of both worlds in some ways, but also the worst of both worlds in other ways. This thread dissolved into acrimony between me and Ulf, and I believe largely because seq_trace has been somewhat the victim of a "worst of both worlds" scenario, of a kind I can only speculate about since the disambiguating details are behind the corporate veil. How that relates to the first two issues: I appreciated Jason's wording when it was in the context of his blog. Unfortunately, when those words are put in the documentation for a corporation's product, they convey the message "Wow, we got some heavy shit here -- but you need it, so you better buy one of our training courses run either by us or some of our alumni, to help get you through it." I.e., it comes across as neither appropriate in style nor exactly ethical in the promotion of services. (Pretty bad advertising either way.) On a blog, I can take Jason's"special kind of hell" with a grain of salt, and with real amusement. But on what amounts to a corporate website? As long as we're extolling honesty here, let's say it: isn't Ericsson paying the bills for erlang.org? Passages like these, in *that* context, leave me feeling confused and disturbed. (But they also diverted me to seq_trace, which it turns out is quite the nugget of gold. So it's not all bad.) While we're on appropriate style, let me point out the kind of thing I prefer. Steve Johnson, in his yacc paper, wrote this: "... it is better that the keywords be reserved; that is, be forbidden for use as variable names. There are powerful stylistic reasons for preferring this, anyway." and, in a version long since gone, he added a footnote: * He says, weakly. It made me sad to see that footnote disappear, as AT&T Unix distros became ever more official in the early 80s. The footnote had epitomized his quietly self-deprecating wit. Removing it left him sounding like some kind of pontificator he never was (at least as far as I could tell from his talks at U.C. Berkeley back then, which were always standing-room-only). Official documentation has to be official. But it doesn't have to be overbearing. The best stuff always reminds you, every few pages, that there's a human being back there somewhere, who only wants to help. By all means, write vividly and well. Especially on blogs. But if there's a blog entry that describes an aspect of your product as "a special kind of hell," that's a signal that you need to improve your documentation (and maybe your product) not simply echo the sentiment. Oh, yes, I know: "priorities." *Sigh*. -michael turner > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Wed Jun 6 08:29:11 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 08:29:11 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <3880F2A1-6E21-4222-B069-7031FA6451C9@feuerlabs.com> <01792E3C-9F0B-481E-A243-70C43B391646@feuerlabs.com> <33E5A8CC-97D7-4105-82FD-23E8ECD252CE@feuerlabs.com> <44B5A392-2B14-4E6D-9D0D-F075F6E73763@feuerlabs.com> <7002BE82-A7CE-4BBB-902F-922698DFC06F@feuerlabs.com> Message-ID: On 6 Jun 2012, at 07:05, Michael Turner wrote: > Ulf, I misinterpreted this: > >> The seq_trace system_tracer allows only one [tracer (process)] per node. >> >> A generic lamport clock implementation has no need for >> such limitations. > > I read this as claiming that seq_trace tracing is limited to a node, > which is (now, after a good night's sleep) rather obviously an > overstretched interpretation, at best. In Erlang/OTP parlance, the tracer is a process that receives the trace output generated when tracing is enabled. In the normal trace support, you can have 0 or 1 tracers per process. The trace messages generated can very well be the result of messages received from, or sent to, other nodes. In order to collect and merge trace output, you can use ttb. TTB interfaces with 'dbg', and both (as well as 'et') are able to process seq_trace output. TTB does have some utility functions to help with sequence tracing, but when it merges trace output, it only looks at the timestamp (even for seq_trace output). Since seq_trace tokens are passed to the receiving process in a message send, it has to work across nodes, but the *trace output*, for both sequence trace and normal trace, is only emitted to the respective local tracer(s) on each node. (The trace BIFs can also use an IP port or file descriptor as a tracer, but the principle is the same). > That said, however, the seq_trace documentation seems to say > conflicting things on this point, and I assumed you were working from > where it says this: > > "The system tracer will only receive those trace events that occur > locally within the Erlang node. To get the whole picture of a > sequential trace that involves processes on several Erlang nodes, the > output from the system tracer on each involved node must be merged > (off line)." There are two levels to seq_trace: - the exchanging of tokens to maintain the counters - the collection of possibly emitted trace data The former works across nodes. The latter assumes the presence of some multi-node collector and merge function. OTP offers ttb, which handles seq_trace output, but not completely (see above). The thing about seq_trace that is slightly different from normal trace is that it invites the active cooperation of the processes themselves (e.g. calling functions like seq_trace:print() and seq_trace:set_token()). The "infection process" works transparently, once initiated. It can also be initiated through trace patterns, making it entirely transparent to the processes being traced. Actually, I have on several occasions raised the suggestion that there should be a function to emit a trace event from within the code. Today, that's normally done by calling some empty function (which is also how 'et' does it). For some reason, I - and apparently others - have overlooked that seq_trace in fact has such a function. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Wed Jun 6 08:53:53 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 15:53:53 +0900 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCEC4BE.8040608@gmail.com> References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> <4FCEC4BE.8040608@gmail.com> Message-ID: Michael Truog: "The tendency to use print statements is a habit from other languages and is often the common denominator when debugging, so making tracing as simple as a print statement (documentation-wise and usability-wise) seems like a sensible goal." A very sensible goal. Early on in using seq_trace, I mixed the styles, but it was annoying because my io:format() calls resulted in message passing in the io module, which got traced by seq_trace when tracing was on, and my trace collections were often trashed with distracting stuff about io. Clearly, I needed some way to filter my traces, or I had to give up print-statement-style tracing. But because I'd been scared off of trace-matching, and didn't want to give up prints, I decided instead to write something like this, in my improvised layer over seq_trace called "ts": format(Fmt,Args) -> Save = seq_trace:set_token([]), % turn off tracing, if it's on io:format (Fmt, Args), seq_trace:set_token(Save). % turn tracing back on again, if it was off Wherever I had an io:format call in my code, I replaced the "io" with "ts". This is just a stopgap. My ts can't wrap everything in OTP. It shows only what a relative Erlang newbie (i.e., me) would do with a non-mainstream tracer package (seq_trace) while grappling with a mix of styles. And that I mixed styles is probably important too: I think newbies will want to cling to the old life-preserver while they grope for the ladder. Unless I'm out in left field yet again. -michael turner On Wed, Jun 6, 2012 at 11:47 AM, Michael Truog wrote: > On 06/05/2012 05:40 PM, Henning Diedrich wrote: > > > One Open Source loving word regarding the 'hell' quote: > > I'm hardly the only one who should be embarrassed: > > "Erlang tracing is a seething pile of pain that involves reasonably > complex knowledge of clever ports, tracing return formats, and > specialized tracing MatchSpecs (which are really their own special > kind of hell). The tracing mechanism is very powerful indeed, but it > can be hard to grasp." > > Obviously, that kind of statement has no place in the official > documentation of a professional product. > > > Honest statements like the one you are citing may be the actual luxury and > strong value of Open Source efforts, which are not directly a commercial > product. Possible only because they /are/ not a product. For one user, fresh > air like this only creates trust and allows to (even very precisely) set > expectations and alertness to shortfalls in both documentation and package. > > That is not to paint the situation rosy, it would be lovely if it was > better. But I appreciate the honesty, very much. > > > I agree that honesty in the documentation is much more beneficial, due to it > being Open Source, rather than the duplicity that might otherwise be present > in corporate documentation ("Politics-Oriented Software Development": > Documentation, http://www.kuro5hin.org/story/2005/1/28/32622/4244). > > The issues mentioned previously in the email thread seem to indicate that > Trace-Driven Development would require more documentation and details, > especially documentation that is immediately relevant to the beginner. > Putting redbug into OTP seems like it would help reduce the learning curve, > just since it is the popular approach to Erlang tracing and has existed for > some time.? The goal with such changes would be to make tracing simpler in a > way that discourages the natural programmer reaction to information > discovery during interactive debugging (of systems that are not yet live), > which is inserting print/logging statements.? The tendency to use print > statements is a habit from other languages and is often the common > denominator when debugging, so making tracing as simple as a print statement > (documentation-wise and usability-wise) seems like a sensible goal. > > - Michael > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Wed Jun 6 08:55:55 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 08:55:55 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> Message-ID: On 6 Jun 2012, at 08:02, Michael Turner wrote: > I'd like to know. I'm guessing that one of the big problems I've had > with understanding Ulf here is that, for him, tracing is, by > definition, a way to collect data about *anomalous* behavior. To me, > tracing is "selectively (and only on occasion) collecting data about > behavior." Period. You can do whatever you want with that data. The > behavior doesn't have to be pathological. In fact, you can use the > data as some assurance of correctness - the "occasion" can be running > a test suite. Which is to say, the "trace-driven development" of this > thread. No, I don't think that. For sure, tracing is indispensable for debugging - not least since you can trace on exceptions - but it is equally important for profiling, for example. There are several uses that one could imagine for permanent service in a live system: event triggers, memory monitoring, etc. Unfortunately, turning on such tracers would mean that the processes being thus monitored couldn't be traced for purposes of debugging. This makes the (one tracer per process) limitation more limiting that it may at first seem, and forces most people to reserve tracing for debugging and profiling purposes during testing. That Lamport clocks are useful for other things was illustrated even by Lamport in his original paper, as he used them to solve the mutual exclusion problem (in his later musings, he noted that some people thought the paper was *only* about implementing mutexes: > Many computer scientists claim to have read it. But I have > rarely encountered anyone who was aware that the paper > said anything about state machines. People seem to think > that it is about either the causality relation on events in a > distributed system, or the distributed mutual exclusion > problem. People have insisted that there is nothing about > state machines in the paper. I've even had to go back and > reread it to convince myself that I really did remember what > I had written. (http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#time-clocks) It would be absolutely brilliant to have native and *general* support for Lamport clocks in the Erlang VM. You think they already exist, in seq_trace, whereas I think Lamport clocks are only coincidentally exposed there as part of a solution to selectively observe sequences of events in a running system, subject to the usual limitations of the tracing sub- system - limitations that in practice render them near-useless for other purposes, even if OTP were to approve of such uses, which they don't. (In the email where Kenneth admitted that seq_trace implemented Lamport clocks, he also wrote that you shouldn't use them for any other purpose than that described in the seq_trace docs.) Changing this would require a strategic decision and some deep thinking from the OTP team. Granted, for *your* intended purpose, they are absolutely fine. It's entirely in line with what they were first made for. (In fact, Quviq's QuickCheck relies on Lamport's "happens before" relation to reduce the state space during random testing of concurrent Erlang code. They don't use seq_trace, though - nor, normally, the built-in tracing.) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Wed Jun 6 09:26:45 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 16:26:45 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: > What we really need is a small number of good technical writers > and money to pay them. Agreed. But guess what? Page after page of Erlang/OTP documentation can be produced as evidence that the team doesn't have the budget for it. For example, this very thread branches off from a discussion of tracing that includes a fight over what I consider an egregious omission from the seq_trace documentation. And here's one of the relevant pages about Erlang tracing, from 2007 http://web.archive.org/web/20071116203524/http://www.erlang.org//doc/apps/et/et_intro.html Grammatical error: "The following prerequisites is required for understanding ...." Here's that same page, now: http://www.erlang.org/doc/apps/et/et_intro.html Same damned error. And, just after the heading "1.1 Scope and Purpose", there's a new error: a paragraph consisting of a single apostrophe. Both versions contain a chilling circularity in the "prerequisites": "familiarity with the Erlang system and Erlang programming in general and the especially the art of Erlang tracing." "Especially the art of Erlang tracing"? But, um, wait: isn't "the art of Erlang tracing" precisely what I'm aiming to learn, in reading the "et" documentation for which this passage is the introduction? Or if not, well, where DO I pick that up? Given my choice of worlds, Richard, I'd choose what you prefer: technical writers on salary. And given a choice of technical writers, I'd choose the bodhisattvas of the field: people with masters degrees in computer science who have decided that working on clearer documentation will help lead, in its own small way, to the enlightenment of all sentient beings. Given *reality*, I propose a wiki. -michael On Wed, Jun 6, 2012 at 7:53 AM, Richard O'Keefe wrote: > > On 6/06/2012, at 3:30 AM, Michael Turner wrote: >> But really: just give me a wiki. Give everybody a wiki. Write scripts >> to update the repo from the wiki diffs, if that's so important. > > For the sake of suffering humanity, forfend! > > The next time someone offers me a wiki instead of documentation, > I am going to be exceedingly angry. ?I have never yet found any > wikified documentation that was worth printing to wipe my arse with. > > I do apologise for the vulgarity, but good documentation needs > good technical writing and technical writing is a skill that is > distinct from programming, although some people can do both well. > It needs a *global* view as well as attention to fine detail. > >> Generally, lower the bar for entry, for those who want to improve the >> documentation, and the documentation *will* improve. > > Someone recently wrote something about the sky turning green then > yellow, didn't they? ?How is *lowering the bar* going to result in > anything *improving*? ?It might get more *voluminous*, but better? > > What we really need is a small number of good technical writers > and money to pay them. > From erlangsiri@REDACTED Wed Jun 6 10:25:29 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 6 Jun 2012 10:25:29 +0200 Subject: [erlang-questions] What is shutdown_error? In-Reply-To: References: Message-ID: sorry, forgot to copy the list 2012/6/6 Siri Hansen > shutdown_error is used by supervisor (and application_master) to inform at > something went a bit wrong during shutdown or a child. When shutting down a > child, supervisor will normally do exit(ChildPid,shutdown) and then monitor > the child to check the actual exit reason. If it terminates with Reason /= > shutdown, supervisor will report a shutdown_error. > > Also, if a child does not respond to supervisor's attempt of graceful > shutdown within the shutdown time specified in the child spec, then the > supervisor will kill it brutally and report a shutdown_error. This might be > what happened in your case, since the reason reported is 'killed'. > > Regards > /siri > > 2012/6/5 Ryan Flynn > >> What is shutdown_error? I can't figure out what to make of >> shutdown_error; all I can find is a brief mention that it exists in sasl >> docs. Full error: >> >> Supervisor foo_sup had child foo_server started with >> foo_server:start_link() at <0.379.0> exit with reason killed in context >> shutdown_error >> >> Thanks in advance. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jun 6 10:31:11 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 6 Jun 2012 09:31:11 +0100 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: <4FCEA717.4050508@eonblast.com> References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> Message-ID: On 6 Jun 2012, at 01:40, Henning Diedrich wrote: > I think from the pyre, this must be salvaged as the actual point I was going for: > > On 6/4/12 6:52 AM, Michael Turner wrote: >> >> I'm now >> doing unit testing on a module as I develop it further, based on >> collecting and filtering seq_trace results > > Is this a common approach, who else does this and how? > I do the 'collect intermediate data for later verification' a fair bit, sometimes using seq_trace, but not always. > Using self-written stuff based on seq_trace, or et_collector? > I had been doing self written stuff - common_test configuration that gets 'auto applied' during init_per_suite/init_per_testcase. I have also used the dbg module (and redbug from time to time, after discovering it by accident when using eper to look at node monitoring approaches) to simply log useful trace information during test runs, which can help easily diagnose why a test is failing. I have thought many times that the (various?) tracing facilities could make for the foundations of a nice testing framework. I am now going to play with et_collector - an application which I'd not paid any attention to TBH - and see where all the different pieces end up. BTW, if anyone has built, or plans to build, stable and general purpose testing tools based on the trace facilities, I will be *VERY* interested. I've been working on some code instrumentation tooling that allows for comments in production code that get transformed into actual function calls when compiled with -Dtest or whatever. One of the several uses I have for this (apart from introducing artificial time delays and the like) is to make trace calls, so et:trace_me is a good target for these kind of things. I had planned to implement the trace calls with some additional meta-variables so you can capture process/call state easily, for example: %% NB: DO NOT REMOVE - this *comment* is used to generate code during test runs %% %% BUG: arbitrary delays in shutdown notification can lead to cluster failures - bug #99741 %% %% @trace [pid, stack] %% @delay [random, 1000, 1000000, ms] gen_server:reply(ReplyTo, shutting_down), .... I have not finished thinking about how the trace facilities can/should be used as a general purpose mechanism to verify behaviour yet. Using 'ordinary' tracing is subject to data not making it to the tracer process IIRC, so seq_trace is presumably the way to go if you want to verify things. Setting trace tokens and the like can be easily done with instrumentation, minimising the need to litter production code with 'test-only' expressions. This could be done with annotations, *special* comments or configuration driven build-time transforms. Further thoughts about *this* topic would be most interesting IMHO. >> From ulf@REDACTED Wed Jun 6 10:31:57 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 10:31:57 +0200 Subject: [erlang-questions] Trace-Driven Development In-Reply-To: References: <4FCA6332.5000109@eonblast.com> <4FCEA717.4050508@eonblast.com> Message-ID: <68194BC0-3EA6-40AA-9421-CFF2783A53A6@feuerlabs.com> On 6 Jun 2012, at 08:02, Michael Turner wrote: > There are three issues here: > (1) how honest one should be in a given context, Indeed, and perhaps also in what *context* that honesty even applies and can be considered correct. Erlang tracing may be a "seething pile of pain" from one perspective, yet such a statement can easily convey the idea that tracing in Erlang is inferior to tracing in other language environments. Technology constantly evolves, of course, and this could eventually become true (a bit like Erlang's error messages were considered very helpful until python came along and set a new standard - eventually forcing Erlang to improve too). > This thread dissolved into acrimony between me and Ulf, > and I believe largely because seq_trace has been > somewhat the victim of a "worst of both > worlds" scenario, of a kind I can only speculate about > since the disambiguating details are behind the > corporate veil. Discussions in a public forum do not need to dissolve into acrimony just because there is disagreement. I tried to offer *some* insight into the background of seq_trace, but you need to respect the fact that I am not at liberty to discuss Ericsson-internal project details, *especially* since I no longer work for Ericsson. But I can share part of the blame. Offering you a public link mentioning forlopp tracing in AXE was not intended to mean "look, here is proof of X", but mostly as a kind of alibi for myself - since there exists documentation on the net mentioning these Ericsson-proprietary aspects, I consider myself justified in mentioning them, without violating my confidentiality obligations to my former employer. I wasn't clear about this intent, and you read something else into it. Nor did I ever try to claim that Ericsson 'invented' either tracing, forlopp tracing or Lamport clocks. I have observed, though, in other contexts, that some of the early work at Ericsson that led, among other things, to Erlang, actually happened in *parallel* with much of the seminal work by Dijkstra, Lamport, Hoare et al. Bjarne D?cker, Mike Williams and others were discussing these things internally already in the '70s (I have seen some cute stencil reports, clearly typewriter-written with traditional glue used to insert pictures, '70s discussions about the importance of selective message reception, long before Erlang came about). The designs of the AXE, laid down in the early '70s, where to some (probably fairly large) extent based on the experiences of building the AKE switch in the '60s. Surely they were also tracking closely what happened at e.g. Bell Labs, since they were pushing the envelope at the same time. Ericsson was actually a bit player until the AXE came to dominate the switch market, and its designs had a *huge* influence internally. http://www.ericssonhistory.com/templates/Ericsson/Article.aspx?id=2095&ArticleID=1384&CatID=362&epslanguage=EN http://books.google.se/books?id=07NmhqkOqwsC&pg=PR14&lpg=PR14#v=onepage&q&f=false (pg 233) Of course, even though the AXE *control system* was single-CPU, it was a multi-processor system, and telephony systems have formed distributed systems, using signaling protocols, since at least the 60s - see the above book, page 451, for example. I don't know if they did tracing across processors or across switches even. I have no documents describing AXE tracing to that detail, and I have not worked on the AXE myself (although many of my colleagues at the time had). One technology that I long wondered if Ericsson had invented was the Specification and Description Language, SDL. It was so pervasive at Ericsson, and seemed to have been used from the beginning of time. On page 267 in the above book, the origins are traced to Kawashima 1971, but LM Ericsson was represented in the early standardization work at the time, and was certainly one of the early adopters, possibly also shaping parts of the standard. For this reason, I've assumed the habit of not excluding the possibility that certain things were 'known' inside large telecom companies (at least among a select few) before they were made known to the world, perhaps by some other institution. But to find out, you typically have to sit down with some old guy and talk to them - maybe they even have some old stencils archived that they can pull out. You won't find out through normal citation searches. This was all before the Web. Sorting out who informed whom this many years afterwards is not easy, and when building products in a proprietary environment, one is usually not in a habit of doing so (although one should). I would not expect industrial programmers in the 90s to be much helped by mentions of Lamport clocks, however technically accurate. Many Ericsson programmers at the time, though, *would* be helped by examples comparing to AXE and MD110. Again, this is just background. Obviously, in the age of the Web, Open Source and NoSQL, the documentation needs to evolve, become more transparent and relevant to the programmers of today. For this purpose, I think the OTP documentation process is too formal. Blogs, wikis, etc are an excellent complement, but community contributions to the OTP documentation is also a good way forward. Erlang does straddle two worlds - one that is pretty fanatical about transparency, and one that is almost exactly the opposite. Much of Erlang's documentation evolved to serve the latter. The discussion about the "special kind of hell" quote illustrates what tends to happen when perspectives collide. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From francesquini@REDACTED Wed Jun 6 11:26:33 2012 From: francesquini@REDACTED (Emilio De Camargo Francesquini) Date: Wed, 6 Jun 2012 11:26:33 +0200 Subject: [erlang-questions] A visual tour of Erlang In-Reply-To: References: <65A60910-98D6-4BFB-8FFA-7ED33C3AA2FA@feuerlabs.com> <78639939-BAF1-4B59-B406-58C566279315@llaisdy.com> Message-ID: Hello Richard, a) For each message sent inside the VM there is an edge in the graph representing it. As the number of messages is huge and the processes that send these messages are being created and destroyed all the time, the edges "decay" after a few seconds and are removed from the graph. When a vertex is no longer connected to any other vertex, that is, it has no connected edges, it is removed from the animation. So in fact the graph represents the processes and the messages exchanged between them during the execution of CouchDB. This is what I called communication graph. b) The shooting away, as you call it, is just a side-effect of the algorithm used to draw the graph. It is a force-directed layout, so when the last edge is disconnected from the vertex it gets "repelled" from the rest of the vertices, and as soon as it has no edges connected to it, it gets removed from the animation. The actual execution speed is 8X faster than it is shown in the video. For those interested in the code, I'll try to upload the code somewhere until the end of this week... Best Regards Emilio 2012/6/6 Richard O'Keefe : > > On 6/06/2012, at 12:25 AM, Emilio De Camargo Francesquini wrote: >> This video shows the processes communication graph for CouchDB running >> in a machine with 24-cores. CouchDB was exercised using YCSB and the >> video was generated using Ubigraph. I've posted the video here: >> >> http://www.youtube.com/watch?v=2MO6Uv7KZ7g > > (a) When you say "communication graph" what actually do you mean? > (b) What does it signify in the movie when nodes go shooting away > ? ?from th core and disappear off-screen? > From erlang@REDACTED Wed Jun 6 12:36:19 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Jun 2012 12:36:19 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On Wed, Jun 6, 2012 at 9:26 AM, Michael Turner wrote: >> What we really need is a small number of good technical writers >> and money to pay them. > > Agreed. But guess what? Page after page of Erlang/OTP documentation > can be produced as evidence that the team doesn't have the budget for > it. Correct. The Erlang/OTP documentation has never had a specific budget. The documentation has to be "good enough for internal use" - Internally you can "always ask the guy who wrote the code" - it is a source of never-ending amazement to me that external users can figure out how do do things *without* asking the authors. For external use the documentation must be better. I agree with Richard on this one - a wiki cannot answer "why" questions - inspecting the code might reveal *how* it works but not *why* it was written this way. I don't want to guess why code does what it does - I want the author to tell me. My person favorite model is "one author - many eyes". I'm really impressed by the real-world haskell site (see for example http://book.realworldhaskell.org/read/functional-programming.html) This is a "three authors many eyes" publication. The authors are responsable for the text - but many people can comment, and the barrier for entry for commenting is very small. The comments can be replied to by other people who read the text and the discussion takes place exactly where it should (ie "under" the paragraph). This way, long discussions about clocks and how they work, can be hidden away under the paragraphs in the text to which they refer. In later versions the authors can choose to revise the text in the light of the comments or ignore them. When my prag book was published it first came out as PDF version, when 70% was written - overnight I has comments on 500 pages of text - this was * incredibly* helpful and really improved the quality of the text. I got immediate feedback at a paragraph level - so was able to reformulate the text *before* chopping down a load of trees. This is the best of worlds - the book still has a "voice" a point of view that I can agree with or disagree with - but many eyes can review the text and find typos and point out things that they do not understand or which need further clarification. I am currently trying to make a similar system for the Erlang documentation that is inspired by the real-world haskell web-site - so hopefully this will improve matters. Given that we are not going to suddenly employ a host of professional technical writers to improve the documentation, I hope a user-driven commenting system with low barriers for entry will help improve matters. We will see, this is an experiment after all. I also wish to target e-books - so I can read all the documentation on a pad computer. > For example, this very thread branches off from a discussion of > tracing that includes a fight over what I consider an egregious > omission from the seq_trace documentation. And here's one of the > relevant pages about Erlang tracing, from 2007 > > ?http://web.archive.org/web/20071116203524/http://www.erlang.org//doc/apps/et/et_intro.html > > Grammatical error: "The following prerequisites is required for > understanding ...." > > Here's that same page, now: > > ?http://www.erlang.org/doc/apps/et/et_intro.html > > Same damned error. And, just after the heading "1.1 ?Scope and > Purpose", there's a new error: a paragraph consisting of a single > apostrophe. > > Both versions contain a chilling circularity in the "prerequisites": > "familiarity with the Erlang system and Erlang programming in general > and the especially the art of Erlang tracing." > > "Especially the art of Erlang tracing"? But, um, wait: isn't "the art > of Erlang tracing" precisely what I'm aiming to learn, in reading the > "et" documentation for which this passage is the introduction? Or if > not, well, where DO I pick that up? > > Given my choice of worlds, Richard, I'd choose what you prefer: > technical writers on salary. And given a choice of technical writers, > I'd choose the bodhisattvas of the field: people with masters degrees > in computer science who have decided that working on clearer > documentation will help lead, in its own small way, to the > enlightenment of all sentient beings. > > Given *reality*, I propose a wiki. The trouble with wiki's is a lot on the stuff on wikis is plain *wrong* and nobody bothers to correct the mistakes. We often see the "stupidity of crowds" in action. Hidden in the OTP documentation is the stuff for many books - if anybody feels like writing a book go ahead. Or write a chapter. If you write a chapter and send it to the guys who wrote the code you'll find them *very* helpfully and they will happily point out erros and misunderstanding in your text. We've got books about Erlang now we need books about OTP internals, debugging, databases, making scalable systems, ... Cheers /Joe > > -michael > > On Wed, Jun 6, 2012 at 7:53 AM, Richard O'Keefe wrote: >> >> On 6/06/2012, at 3:30 AM, Michael Turner wrote: >>> But really: just give me a wiki. Give everybody a wiki. Write scripts >>> to update the repo from the wiki diffs, if that's so important. >> >> For the sake of suffering humanity, forfend! >> >> The next time someone offers me a wiki instead of documentation, >> I am going to be exceedingly angry. ?I have never yet found any >> wikified documentation that was worth printing to wipe my arse with. >> >> I do apologise for the vulgarity, but good documentation needs >> good technical writing and technical writing is a skill that is >> distinct from programming, although some people can do both well. >> It needs a *global* view as well as attention to fine detail. >> >>> Generally, lower the bar for entry, for those who want to improve the >>> documentation, and the documentation *will* improve. >> >> Someone recently wrote something about the sky turning green then >> yellow, didn't they? ?How is *lowering the bar* going to result in >> anything *improving*? ?It might get more *voluminous*, but better? >> >> What we really need is a small number of good technical writers >> and money to pay them. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From siddharth.karandikar@REDACTED Wed Jun 6 13:47:32 2012 From: siddharth.karandikar@REDACTED (Siddharth Karandikar) Date: Wed, 6 Jun 2012 17:17:32 +0530 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: Hi all, I have put together a program for this. This program fetches email archives from erlang-questions and uses information available in email headers to identify unanswered mails. It then generates pages to browse through these mails, view and reply. This program is currently running on my server. Its currently configured to keep track of mails within last 120 days. I am fetching new content and updating the pages twice a day. Please have a look - http://178.79.140.33/index.html Comments/suggestions are welcome! On the side note, Can anyone tell me how frequently does these mail archives get updated? So that I will tune the schedule on my server accordingly. Thanks, Siddharth On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: > On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >> On 01/06/2012 08:22, Joe Armstrong wrote: >>> >>> I think it would be a good idea if those of us who might qualify as being >>> "knowledgeable persons" could take the trouble to occasionally answer >>> ?questions from apparent beginners that have not been answered. >>> >>> After a threshold of say 3 days - then at least *somebody* could attempt >>> an answer so that beginner to Erlang don't feel unloved when they >>> reach to this list. >> >> >> It would be easier to do this if there was a some kind of separate feed for >> mail that hasn't been answered in the last 3 days or whatever. Otherwise you >> loose a lot of cycles actually keeping track of who has asked what, and I >> suspect that is the time killer for a lot of people, rather than not having >> time to answer basic questions. >> >> I have *no idea* about managing mailing list software, so I'm not even sure >> if that idea is feasible. > > Pretty easy to write an erlang program to do this and run it once a > day with crontab > > The mails can be obtained programmatically from > > http://erlang.org/pipermail/erlang-questions/ > > The rest is 'left as an exercise to the reader' > > /Joe > >>> >>> Of course, some extremely knowledgable and well-known people ask >>> extremely difficult >>> questions here - and zero replies should probably >>> be taken as a sign that "nobody knows the answer" >>> >>> We could even have volunteers among those who posses the knowledge to >>> answer >>> all unanswered beginners questions. >>> >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mononcqc@REDACTED Wed Jun 6 13:52:48 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 06 Jun 2012 07:52:48 -0400 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <4FCF4490.5060001@ferd.ca> That's pretty cool. If you could give it something like a treshold period (we don't put unanswered e-mails newer than 3 days) and then stick an RSS (or Atom) feed in there, I'd probably stick it in a reader to be able to quickly get notifications on messages I might be able to help with. On Wed Jun 6 07:47:32 2012, Siddharth Karandikar wrote: > Hi all, > > I have put together a program for this. > > This program fetches email archives from erlang-questions and uses > information available in email headers to identify unanswered mails. > It then generates pages to browse through these mails, view and reply. > > This program is currently running on my server. Its currently > configured to keep track of mails within last 120 days. I am fetching > new content and updating the pages twice a day. > > Please have a look - http://178.79.140.33/index.html > Comments/suggestions are welcome! > > On the side note, Can anyone tell me how frequently does these mail > archives get updated? So that I will tune the schedule on my server > accordingly. > > Thanks, > Siddharth > > On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: >> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >>> On 01/06/2012 08:22, Joe Armstrong wrote: >>>> >>>> I think it would be a good idea if those of us who might qualify as being >>>> "knowledgeable persons" could take the trouble to occasionally answer >>>> questions from apparent beginners that have not been answered. >>>> >>>> After a threshold of say 3 days - then at least *somebody* could attempt >>>> an answer so that beginner to Erlang don't feel unloved when they >>>> reach to this list. >>> >>> >>> It would be easier to do this if there was a some kind of separate feed for >>> mail that hasn't been answered in the last 3 days or whatever. Otherwise you >>> loose a lot of cycles actually keeping track of who has asked what, and I >>> suspect that is the time killer for a lot of people, rather than not having >>> time to answer basic questions. >>> >>> I have *no idea* about managing mailing list software, so I'm not even sure >>> if that idea is feasible. >> >> Pretty easy to write an erlang program to do this and run it once a >> day with crontab >> >> The mails can be obtained programmatically from >> >> http://erlang.org/pipermail/erlang-questions/ >> >> The rest is 'left as an exercise to the reader' >> >> /Joe >> >>>> >>>> Of course, some extremely knowledgable and well-known people ask >>>> extremely difficult >>>> questions here - and zero replies should probably >>>> be taken as a sign that "nobody knows the answer" >>>> >>>> We could even have volunteers among those who posses the knowledge to >>>> answer >>>> all unanswered beginners questions. >>>> >>>> >>>> Cheers >>>> >>>> /Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hd2010@REDACTED Wed Jun 6 13:55:23 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 13:55:23 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <4FCF452B.4010300@eonblast.com> On 6/6/12 12:36 PM, Joe Armstrong wrote: > it is a source of never-ending amazement > to me that external users can figure out how do do things *without* asking the > authors. Glad to hear that, I immediately feel less stupid. On 6/6/12 12:53 AM, Richard O'Keefe wrote: > What we really need is ... money. While we may be able to all agree on that one, there is one related thing that may come cheap. As noted before, Erlang docs seem harder to Google than other docs or programming related information. In my experience, this keeps being a massive productivity loss. While I don't understand why that is, this should be sth. that without money, only with decent SEO knowledge, could be fixed very easily, for a big effect. Especially given the fact that the HTML is generated from sufficiently abstract sources. My suspicion is that it has to do with the frame layout. But it should be possible to compensate for that and better guide the search engines through it. I have Googled unsuccessfully, repeatedly, even for stuff that I /knew/ I had read in the Erlang docs. But could not find, except by finally remembering which links I followed from what to where to end up at the forgotten place. I even reverted to grepping the downloaded doc htmls, which I can't recall I ever did for other languages. Unfortunately, it seems the great effort of http://erldocs.com/ did not help this either. Does anyone with the right skill set have a comment on that? Concretely: why, if we can agree on that, is it so hard to use Google to find stuff one is looking for in the Erlang docs; and what can be done to solve that?) > worth printing to wipe my arse with. > > I do apologise There is always that key, top right-hand on the keyboard for immediate regrets. Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Jun 6 14:10:37 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 6 Jun 2012 14:10:37 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <20120526203817.GA10058@localhost> References: <20120526203817.GA10058@localhost> Message-ID: Bump On Sat, May 26, 2012 at 10:38 PM, Motiejus Jak?tys wrote: > Hi, > > For term() to pid() mapping I've always used gproc. For the same thing > my coworker always used pg2. > > I am making a comparison now. There are some features in gproc which are > not in pg2: > > * Await for registration > * Per-process properties > * Aggregate counters > * QLC > > Whereas pg2 "natively" supports mapping one key to several processes > (which can be easily achieved with QLC in gproc). > > Any more differences between them? -- Motiejus Jak?tys From samuelrivas@REDACTED Wed Jun 6 14:13:30 2012 From: samuelrivas@REDACTED (Samuel) Date: Wed, 6 Jun 2012 14:13:30 +0200 Subject: [erlang-questions] multiple behaviours In-Reply-To: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> References: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> Message-ID: > I am compiling the canopen stack (git://github.com/tonyrog/canopen.git) and > get the following warning: > > ../src/co_os_app.erl:37: Warning: conflicting behaviours - callback > index_specification/2 required by both 'co_stream_app' and 'co_app' (line > 36) > > The question is: And no question followed ;) Ideally you could suppress the warning as you really intend to share the same function for both behaviours, but as far as I know there is not such an option in the compiler, so you either suppress all warnings for that module or learn to live with this one. Or, as you also suggest, write a new behaviour combining those two. If this situation is not common in your design I would suggest this solution as prevents you from going against what I think is the intended use of behaviours. I guess the original idea was to have at most a behaviour per module, but more authoritative voices may have a word about this. If you was planning to have many of those behaviour combinations, the second option seems bad idea, as the result could be messy. Best -- Samuel From hd2010@REDACTED Wed Jun 6 14:19:41 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 14:19:41 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <4FCF4ADD.5090509@eonblast.com> On 6/6/12 12:36 PM, Joe Armstrong wrote: > I'm really impressed by the real-world haskell site (see for example > http://book.realworldhaskell.org/read/functional-programming.html) Since you already mentioned that you were thinking of UUIDs, maybe the way was to set up a global comment repository and provide JavaScript links and alternatively a browser plugin that can link to /any/ paragraph on any page to let comments appear after any paragraph anywhere. The JS could be for sites to tie in, which intent to participate. They would not have to change anything in their structure as the JS would just inspect the DOM and add the 'n comments' hooks. The browser plugin for all other. This annotated web could be complemented with the possibility to also store links between any two paragraphs explicitly, beyond the obvious option of writing URLs into the comment proper. To compensate for multiple copies or dynamic URLs, the text, stripped of tags, could be hashed to arrive at a technical reference. This would all only work in 70% of cases. Superficial changes would break the hashes. But it could be an easy way to phase such efforts in. Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Jun 6 14:23:23 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 6 Jun 2012 14:23:23 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: Great! There is a small typo and an encoding error in the mailto links though: * it should be erlang-questions@; * the spaces in the subject should be encoded as %20. Furthermore, I think you should prepend the subject with "Re: ". Thanks for your work, maybe you should publish the code on GitHub for further improvements! -- Anthony Ramine Le 6 juin 2012 ? 13:47, Siddharth Karandikar a ?crit : > Hi all, > > I have put together a program for this. > > This program fetches email archives from erlang-questions and uses > information available in email headers to identify unanswered mails. > It then generates pages to browse through these mails, view and reply. > > This program is currently running on my server. Its currently > configured to keep track of mails within last 120 days. I am fetching > new content and updating the pages twice a day. > > Please have a look - http://178.79.140.33/index.html > Comments/suggestions are welcome! > > On the side note, Can anyone tell me how frequently does these mail > archives get updated? So that I will tune the schedule on my server > accordingly. > > Thanks, > Siddharth > > On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: >> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >>> On 01/06/2012 08:22, Joe Armstrong wrote: >>>> >>>> I think it would be a good idea if those of us who might qualify as being >>>> "knowledgeable persons" could take the trouble to occasionally answer >>>> questions from apparent beginners that have not been answered. >>>> >>>> After a threshold of say 3 days - then at least *somebody* could attempt >>>> an answer so that beginner to Erlang don't feel unloved when they >>>> reach to this list. >>> >>> >>> It would be easier to do this if there was a some kind of separate feed for >>> mail that hasn't been answered in the last 3 days or whatever. Otherwise you >>> loose a lot of cycles actually keeping track of who has asked what, and I >>> suspect that is the time killer for a lot of people, rather than not having >>> time to answer basic questions. >>> >>> I have *no idea* about managing mailing list software, so I'm not even sure >>> if that idea is feasible. >> >> Pretty easy to write an erlang program to do this and run it once a >> day with crontab >> >> The mails can be obtained programmatically from >> >> http://erlang.org/pipermail/erlang-questions/ >> >> The rest is 'left as an exercise to the reader' >> >> /Joe >> >>>> >>>> Of course, some extremely knowledgable and well-known people ask >>>> extremely difficult >>>> questions here - and zero replies should probably >>>> be taken as a sign that "nobody knows the answer" >>>> >>>> We could even have volunteers among those who posses the knowledge to >>>> answer >>>> all unanswered beginners questions. >>>> >>>> >>>> Cheers >>>> >>>> /Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Jun 6 14:27:48 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Wed, 06 Jun 2012 14:27:48 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <20120526203817.GA10058@localhost> References: <20120526203817.GA10058@localhost> Message-ID: <4FCF4CC4.8040206@ninenines.eu> On 05/26/2012 10:38 PM, Motiejus Jak?tys wrote: > Hi, Hello, > For term() to pid() mapping I've always used gproc. For the same thing > my coworker always used pg2. > > I am making a comparison now. There are some features in gproc which are > not in pg2: > > * Await for registration > * Per-process properties > * Aggregate counters > * QLC > > Whereas pg2 "natively" supports mapping one key to several processes > (which can be easily achieved with QLC in gproc). Unless I misunderstand, gproc's properties are exactly this, mapping one key to several processes. -- Lo?c Hoguin Erlang Cowboy Nine Nines From n.oxyde@REDACTED Wed Jun 6 14:37:41 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 6 Jun 2012 14:37:41 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <1D9CD642-72A7-4366-9379-CB2F478F23A3@gmail.com> The ">>" link could also be improved by adding the month to the site: URL and putting the subject in intitle: surrounded with quotes: site:http://erlang.org/pipermail/erlang-questions/ [erlang-questions] -heart question site:http://erlang.org/pipermail/erlang-questions/ intitle:"[erlang-questions] -heart question" I'm pretty confident it could be an "I'm Feeling Lucky" URL with those improvements. Regards, -- Anthony Ramine Le 6 juin 2012 ? 14:23, Anthony Ramine a ?crit : > Great! > > There is a small typo and an encoding error in the mailto links though: > > * it should be erlang-questions@; > * the spaces in the subject should be encoded as %20. > > Furthermore, I think you should prepend the subject with "Re: ". > > Thanks for your work, maybe you should publish the code on GitHub for > further improvements! > > -- > Anthony Ramine > > Le 6 juin 2012 ? 13:47, Siddharth Karandikar a ?crit : > >> Hi all, >> >> I have put together a program for this. >> >> This program fetches email archives from erlang-questions and uses >> information available in email headers to identify unanswered mails. >> It then generates pages to browse through these mails, view and reply. >> >> This program is currently running on my server. Its currently >> configured to keep track of mails within last 120 days. I am fetching >> new content and updating the pages twice a day. >> >> Please have a look - http://178.79.140.33/index.html >> Comments/suggestions are welcome! >> >> On the side note, Can anyone tell me how frequently does these mail >> archives get updated? So that I will tune the schedule on my server >> accordingly. >> >> Thanks, >> Siddharth >> >> On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: >>> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson wrote: >>>> On 01/06/2012 08:22, Joe Armstrong wrote: >>>>> >>>>> I think it would be a good idea if those of us who might qualify as being >>>>> "knowledgeable persons" could take the trouble to occasionally answer >>>>> questions from apparent beginners that have not been answered. >>>>> >>>>> After a threshold of say 3 days - then at least *somebody* could attempt >>>>> an answer so that beginner to Erlang don't feel unloved when they >>>>> reach to this list. >>>> >>>> >>>> It would be easier to do this if there was a some kind of separate feed for >>>> mail that hasn't been answered in the last 3 days or whatever. Otherwise you >>>> loose a lot of cycles actually keeping track of who has asked what, and I >>>> suspect that is the time killer for a lot of people, rather than not having >>>> time to answer basic questions. >>>> >>>> I have *no idea* about managing mailing list software, so I'm not even sure >>>> if that idea is feasible. >>> >>> Pretty easy to write an erlang program to do this and run it once a >>> day with crontab >>> >>> The mails can be obtained programmatically from >>> >>> http://erlang.org/pipermail/erlang-questions/ >>> >>> The rest is 'left as an exercise to the reader' >>> >>> /Joe >>> >>>>> >>>>> Of course, some extremely knowledgable and well-known people ask >>>>> extremely difficult >>>>> questions here - and zero replies should probably >>>>> be taken as a sign that "nobody knows the answer" >>>>> >>>>> We could even have volunteers among those who posses the knowledge to >>>>> answer >>>>> all unanswered beginners questions. >>>>> >>>>> >>>>> Cheers >>>>> >>>>> /Joe >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From eric@REDACTED Wed Jun 6 14:50:53 2012 From: eric@REDACTED (Eric Moritz) Date: Wed, 6 Jun 2012 08:50:53 -0400 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <4FCF4CC4.8040206@ninenines.eu> References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> Message-ID: Yes, if you register a property key, you can retrieve all the procs that have that property set. For instance a fanout pub/sub can be accomplished doing: subscribe(Channel) -> gproc:reg({p, l, {subscribers, Channel}}). publish(Channel, Msg) -> gproc:send({p, l, {subscribers, Channel}}, Msg). If a process calls publish/2, the message will be sent to every process that called subscribe/1. I'm on my phone so it's a bit hard to look up the details, if you want to get a list of pids that called subscribe, look at the code of gproc:send/2. I think the function is called lookup_pids/1 but I may be wrong. The etorrent project uses gproc. That's a good read if you want to see how to use gproc. He uses property keys for each torrent file process as well as using await to delay initialization of gen_servers that depend on other gen_servers. Eric Moritz. On Jun 6, 2012 8:27 AM, "Lo?c Hoguin" wrote: > On 05/26/2012 10:38 PM, Motiejus Jak?tys wrote: > >> Hi, >> > > Hello, > > For term() to pid() mapping I've always used gproc. For the same thing >> my coworker always used pg2. >> >> I am making a comparison now. There are some features in gproc which are >> not in pg2: >> >> * Await for registration >> * Per-process properties >> * Aggregate counters >> * QLC >> >> Whereas pg2 "natively" supports mapping one key to several processes >> (which can be easily achieved with QLC in gproc). >> > > Unless I misunderstand, gproc's properties are exactly this, mapping one > key to several processes. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Jun 6 15:15:30 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 6 Jun 2012 15:15:30 +0200 Subject: [erlang-questions] debug output from yaws Message-ID: Hello! I'm working on an application that uses the yaws webserver. This application also contains the reshd module (see http://erlang.org/pipermail/erlang-questions/2001-May/003162.html), so I can connect to the erlang shell of the node even if yaws was started as daemon mode. My problem is: if I connect to the shell and turn on tracing (using dbg), I can't see it's output, because the stdout and stderr are redirected to /dev/null. Is there a way to get the debug output without restarting yaws? From magnus.henoch@REDACTED Wed Jun 6 15:18:27 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Wed, 06 Jun 2012 14:18:27 +0100 (BST) Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <1D9CD642-72A7-4366-9379-CB2F478F23A3@gmail.com> Message-ID: <242fd643-abff-4d30-b5d8-e228508d6c69@knuth> Another way to accomplish that: since you have the message id for the question, you can make a link to mid.gmane.org, like this: http://mid.gmane.org/CAFVMu-pzwGOQp8JKjKJh_w3L2oF5goGCPCQGopMP8nxGEosCVA%40mail.gmail.com ----- Original Message ----- > The ">>" link could also be improved by adding the month to the site: > URL > and putting the subject in intitle: surrounded with quotes: > > site:http://erlang.org/pipermail/erlang-questions/ > [erlang-questions] -heart question > site:http://erlang.org/pipermail/erlang-questions/ > intitle:"[erlang-questions] -heart question" > > I'm pretty confident it could be an "I'm Feeling Lucky" URL with > those > improvements. > > Regards, > > -- > Anthony Ramine > > > Le 6 juin 2012 ? 14:23, Anthony Ramine a ?crit : > > > Great! > > > > There is a small typo and an encoding error in the mailto links > > though: > > > > * it should be erlang-questions@; > > * the spaces in the subject should be encoded as %20. > > > > Furthermore, I think you should prepend the subject with "Re: ". > > > > Thanks for your work, maybe you should publish the code on GitHub > > for > > further improvements! > > > > -- > > Anthony Ramine > > > > Le 6 juin 2012 ? 13:47, Siddharth Karandikar a ?crit : > > > >> Hi all, > >> > >> I have put together a program for this. > >> > >> This program fetches email archives from erlang-questions and uses > >> information available in email headers to identify unanswered > >> mails. > >> It then generates pages to browse through these mails, view and > >> reply. > >> > >> This program is currently running on my server. Its currently > >> configured to keep track of mails within last 120 days. I am > >> fetching > >> new content and updating the pages twice a day. > >> > >> Please have a look - http://178.79.140.33/index.html > >> Comments/suggestions are welcome! > >> > >> On the side note, Can anyone tell me how frequently does these > >> mail > >> archives get updated? So that I will tune the schedule on my > >> server > >> accordingly. > >> > >> Thanks, > >> Siddharth > >> > >> On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong > >> wrote: > >>> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson > >>> wrote: > >>>> On 01/06/2012 08:22, Joe Armstrong wrote: > >>>>> > >>>>> I think it would be a good idea if those of us who might > >>>>> qualify as being > >>>>> "knowledgeable persons" could take the trouble to occasionally > >>>>> answer > >>>>> questions from apparent beginners that have not been answered. > >>>>> > >>>>> After a threshold of say 3 days - then at least *somebody* > >>>>> could attempt > >>>>> an answer so that beginner to Erlang don't feel unloved when > >>>>> they > >>>>> reach to this list. > >>>> > >>>> > >>>> It would be easier to do this if there was a some kind of > >>>> separate feed for > >>>> mail that hasn't been answered in the last 3 days or whatever. > >>>> Otherwise you > >>>> loose a lot of cycles actually keeping track of who has asked > >>>> what, and I > >>>> suspect that is the time killer for a lot of people, rather than > >>>> not having > >>>> time to answer basic questions. > >>>> > >>>> I have *no idea* about managing mailing list software, so I'm > >>>> not even sure > >>>> if that idea is feasible. > >>> > >>> Pretty easy to write an erlang program to do this and run it once > >>> a > >>> day with crontab > >>> > >>> The mails can be obtained programmatically from > >>> > >>> http://erlang.org/pipermail/erlang-questions/ > >>> > >>> The rest is 'left as an exercise to the reader' > >>> > >>> /Joe > >>> > >>>>> > >>>>> Of course, some extremely knowledgable and well-known people > >>>>> ask > >>>>> extremely difficult > >>>>> questions here - and zero replies should probably > >>>>> be taken as a sign that "nobody knows the answer" > >>>>> > >>>>> We could even have volunteers among those who posses the > >>>>> knowledge to > >>>>> answer > >>>>> all unanswered beginners questions. > >>>>> > >>>>> > >>>>> Cheers > >>>>> > >>>>> /Joe > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From tony@REDACTED Wed Jun 6 15:33:15 2012 From: tony@REDACTED (Tony Rogvall) Date: Wed, 6 Jun 2012 15:33:15 +0200 Subject: [erlang-questions] multiple behaviours In-Reply-To: References: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> Message-ID: On 6 jun 2012, at 14:13, Samuel wrote: >> I am compiling the canopen stack (git://github.com/tonyrog/canopen.git) and >> get the following warning: >> >> ../src/co_os_app.erl:37: Warning: conflicting behaviours - callback >> index_specification/2 required by both 'co_stream_app' and 'co_app' (line >> 36) >> >> The question is: > > And no question followed ;) > > Ideally you could suppress the warning as you really intend to share > the same function for both behaviours, but as far as I know there is > not such an option in the compiler, so you either suppress all > warnings for that module or learn to live with this one. > Yupp. > Or, as you also suggest, write a new behaviour combining those two. If > this situation is not common in your design I would suggest this > solution as prevents you from going against what I think is the > intended use of behaviours. I guess the original idea was to have at > most a behaviour per module, but more authoritative voices may have a > word about this. > Any takers on this topic (meaning people involved in the design of course) > If you was planning to have many of those behaviour combinations, the > second option seems bad idea, as the result could be messy. > Thanks. Wow, someone answered a question sent 7 days ago :-) /Tony From ulf@REDACTED Wed Jun 6 15:33:54 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 15:33:54 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> Message-ID: Yes, this is correct. Gproc supports a few different types of register entries: - names, which are unique (can be any term) - properties, which can be held by multiple processes simultaneously - counters, which are properties with update_counter() semantics - aggregate counters, which maintain the sum of all counters with the same name as the aggregate counter. Properties are very useful for pub-sub, but also for highlighting various characteristics of the process. In my initial presentation of gproc, I had integrated it into OTP and modified the behaviors to register a {p,l,{behaviour,B}} property. This way, you could easily query the system for a certain type of behavior, and drill further to inspect other things. (This had the unfortunate side-effect that many thought you had to have a patched OTP to use gproc. Next time, I will try to be smarter). Thus, if you inspect a process, either with gproc:i(), or gproc:info(Pid [, gproc]), you can easily tell from its registered names and properties quite a bit about the process. BTW, I've been thinking about adding a function, gproc:bcast(Key, Msg), which would be similar to: rpc:abcast(gproc, send, [Key, Msg]) but with more predictable sequencing behavior, most likely making use of a gproc_bcast server to make sure all messages go the same way. Wild cheers and enthusiasm may intice me to do it sooner rather than later?. ;-) BR, Ulf W On 6 Jun 2012, at 14:50, Eric Moritz wrote: > Yes, if you register a property key, you can retrieve all the procs that have that property set. For instance a fanout pub/sub can be accomplished doing: > > subscribe(Channel) -> > gproc:reg({p, l, {subscribers, > Channel}}). > > publish(Channel, Msg) -> > gproc:send({p, l, {subscribers, > Channel}}, > Msg). > > If a process calls publish/2, the message will be sent to every process that called subscribe/1. > > I'm on my phone so it's a bit hard to look up the details, if you want to get a list of pids that called subscribe, look at the code of gproc:send/2. I think the function is called lookup_pids/1 but I may be wrong. > > The etorrent project uses gproc. That's a good read if you want to see how to use gproc. He uses property keys for each torrent file process as well as using await to delay initialization of gen_servers that depend on other gen_servers. > Eric Moritz. > On Jun 6, 2012 8:27 AM, "Lo?c Hoguin" wrote: > On 05/26/2012 10:38 PM, Motiejus Jak?tys wrote: > Hi, > > Hello, > > For term() to pid() mapping I've always used gproc. For the same thing > my coworker always used pg2. > > I am making a comparison now. There are some features in gproc which are > not in pg2: > > * Await for registration > * Per-process properties > * Aggregate counters > * QLC > > Whereas pg2 "natively" supports mapping one key to several processes > (which can be easily achieved with QLC in gproc). > > Unless I misunderstand, gproc's properties are exactly this, mapping one key to several processes. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 6 15:37:57 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Wed, 06 Jun 2012 15:37:57 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> Message-ID: <4FCF5D35.304@ninenines.eu> On 06/06/2012 03:33 PM, Ulf Wiger wrote: > BTW, I've been thinking about adding a function, gproc:bcast(Key, Msg), > which would be similar to: > > rpc:abcast(gproc, send, [Key, Msg]) > > but with more predictable sequencing behavior, most likely making use of > a gproc_bcast server to make sure all messages go the same way. > > Wild cheers and enthusiasm may intice me to do it sooner rather than > later?. ;-) Wild cheers and enthusiasm! -- Lo?c Hoguin Erlang Cowboy Nine Nines From michael.eugene.turner@REDACTED Wed Jun 6 15:50:22 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 6 Jun 2012 22:50:22 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On Wed, Jun 6, 2012 at 7:36 PM, Joe Armstrong wrote: > I agree with Richard on this one - a wiki cannot answer "why" questions - > inspecting the code might reveal *how* it works but not *why* it was > written this way. I don't see where Richard specifically raised that point, but it's incorrect in any case. There's no reason a wiki can't answer "why" questions. Indeed, the very first wiki in existence is "meta" on this point, asking why wikis work at all (much to the bemusement of Ward Cunningham, the surprised inventor) http://c2.com/cgi/wiki?WhyWikiWorks And many wikis do explain the reasons for things. For example, Wikipedia has elaborate, polished internal articles explaining the "why" of its numerous policies and guidelines. A wiki is a text repository. It doesn't limit the themes of its prose. (Except insofar as obscenity-style vandalism is a "theme" -- there are automatic reverters for such edits.) It would take AI for wiki software to enforce any such rule anyway, right? > I don't want to guess why code does what it does - I want the author to tell me. Well, now you know my frustration upon first encountering Lamport clocks in seq_trace as snippets of pseudocode, but not called Lamport clocks. "Why that way?" wasn't satisfactorily answered by the ext, and I was only lucky that the vague memory of Lamport clocks nagged at me until I got to the bottom of the question. If the seq_trace documentation had *merely* said, "seq_trace implements Lamport clocks," I would have been much further down the road to its reasons even if I *hadn't* already been familiar with them from decades before. Why? Because I could search on "Lamport clocks" and find lots of *other* people (including Lamport himself) explaining the "why." Now, my frustration is: I don't think anybody at Ericsson is going to fix this soon, and I'd like to fix it, but I have to build the frickin' docs from scratch just to add four words. Then submit it as a patch. > My person favorite model is "one author - many eyes". I would say that "many eyes" is what actually distinguishes "author" from "writer" or "journalist." The author has *authority* - to be a select community's final decider about content almost as much as the constructor of it. Almost all of the better books I've read have featured long lists of names in the Acknowledgments, together with an apology to all those whose names and contributions had become too hazy for specifics. But only the author is The Author. Problem is, the author model is expensive. Cheap, soon, good -- pick any two. An author can build a book faster than any crowd, and it can be as good as anything out there, not to mention better than a crowd's collective effort. But that's not cheap. That's your bottleneck. > I'm really impressed by the real-world haskell site (see for example > http://book.realworldhaskell.org/read/functional-programming.html) > > This is a "three authors many eyes" publication. It's impressive, but it doesn't solve the problem: as you yourself admit, Erlang/OTP has no budget for one author, much less three of them. Besides, for the existing Erlang/OTP docs, you don't *need* authorship. You need editing. For nothing. Well-managed wikis can excel at that. > The authors are responsable > for the text - but many people can comment, and the barrier for entry > for commenting is very small. Joe, if you've never done it before, go to a locked-down page http://en.wikipedia.org/wiki/Wikipedia:Protection_policy#semi on Wikipedia, like this one http://en.wikipedia.org/wiki/Barack_Obama and click on the "Talk" link. http://en.wikipedia.org/wiki/Talk:Barack_Obama Then, if you like, add your own comment -- you won't need to ask anybody's permission. Richard asks, "What's to be gained by lowering the bar?" Clearly he envisions some hoi polloi stinking up the joint. So let me be careful about what I mean by "bar": it's the difficulty of getting started as a contributor. You can still control who's a contributor, while making the getting-started part much easier in every other way. If you sloshed all of the Erlang/OTP docs onto a wiki, then limited the editor base to 10 people all of whom you'd interviewed by phone for 10 minutes to get a handle on their Erlang/OTP street-smarts, and then later verified that they had advanced degrees in computer science, you WILL have lowered the bar, as far as I'm concerned. The bar is defined, right now, as the skills and orientation and *personal time* required to start submitting patches to Erlang/OTP documentation -- a long, multi-step, hardly-snag-free process. Well, if all someone wants to do is change "is" to "are" for plural agreement, clearing *that* bar is a hell of lot to ask, isn't it? No wonder you have typos that sit there for years on end. > The comments can be replied to by other > people who read the text > and the discussion takes place exactly where it should (ie "under" the > paragraph). > This way, long discussions about clocks and how they work, can be hidden away > under the paragraphs in the text to which they refer. Nice, but with (for example) MediaWiki's footnote extension, this can be done by enclosing a link to a section of the article's Talk page with ... tags. Not as smooth, but still very doable. > This is the best of worlds - the book still has a "voice" a point of > view that I can > agree with or disagree with - but many eyes can review the text and find typos > and point out things that they do not understand or which need further > clarification. The best of worlds is always gonna cost ya. And that's the bottleneck in improving existing online Erlang/OTP docs. A voice is perhaps essential for an authored work. But reference manuals don't really need much "voice" -- they are encyclopedic. Nor do tutorials -- voice can even distract in that case. They just need correct information, couched in correct prose of a relatively uniform and generic style, within a reasonably well-organized structure. Keeping the bar for contributing even the most minor of corrections to Erlang/OTP docs as high as the bar for making major changes serves no purpose that I can understand. An ongoing source of pain for me: stuff that should really be hyperlinked in the Erlang/OTP docs. There's not nearly enough of that kind of thing. A paucity of internal links wouldn't be so bad if there was a search box -- but there isn't! And that wouldn't be so bad if I could go back out to Google and get the right page, as the first link -- but I don't, I often get the old, out-of-date pages served ranking higher. A wiki would solve all those problems. > I am currently trying to make a similar system for the Erlang documentation > that is inspired by the real-world haskell web-site - so hopefully > this will improve matters. It would, I'm sure, but in the meantime, those errors and unlinked phrases just sit there on erlang.org, crying out for improvement. You could install a wiki in perhaps a few hours. You could probably write scripts that translate existing content to wiki markup in a few days. (I've done that kind of thing myself.) If proper reflection of contributed changes back into github is important, a bit more effort would get you something that could reap the edit-deltas out of the wiki (which stores them as deltas -- nothing is ever lost) and submits them. I wouldn't be surprised if code for that already exists. > Given that we are not going to suddenly employ a host of professional > technical writers to improve the documentation, I hope a user-driven > commenting system with > low barriers for entry will help improve matters. You'll still need someone who has the time to go through that existing stuff and figure out what to act on. I.e., those *authors* Erlang/OTP presently can't afford. The Wiki Way is: give readers the right to be an *editor* too. > The trouble with wiki's is a lot on the stuff on wikis is plain > *wrong* Solving that problem is largely a question of (1) access control, which is a reasonably evolved (if still somewhat underutilized) aspect of the more mature wiki packages -- and -- (2) making sure that people who are formally responsible for document correctness get notifications (and links to the diffs -- also automated) whenever there are changes in the pages they are responsible for. These people needn't be on the Ericsson payroll (though undoubtedly employees should be getting notifications); they can also be trusted volunteers. > and nobody > bothers to correct the mistakes. Not "nobody". For example, I'm one of those weird people who will spend 30 seconds to sign up for almost any wiki I see a typo in. If the Erlang docs were on a wiki, I probably would have done hundreds of corrections of minor stuff (typos, grammatical errors, needed links) by now. > We often see the "stupidity of > crowds" in action. When you can decide who's in the crowd, you can control its IQ. So it only depends on how intelligent *you* are, in making the choices. With packages like MediaWiki, you can decide. You can even automate a reasonably intelligent decision, to some extent. Here's fairly simple piece of code: require an e-mail address in the signup for wiki editing rights, search the archives of the Erlang questions mailing list for that e-mail address, and if the time between the aspiring editor's first contribution (if any) and their most recent one is less than three months, send a refusal. A few worthy cases might fall through the cracks, but just mention that you're down there under those cracks, ready to help, and handle them case-by-case. [snip] > We've got books about Erlang now we need books about OTP internals, debugging, > databases, making scalable systems, ... I'd like that, but ... what if the bootstrap to a more-Erlang-authors world is better quality online documentation attracting more people to the system in the first place? And what if the bootstrap to better online documentation is greater interactivity in the Erlang community in refining what online documentation already exists? And what if you're closer to the book goals if you do something that could facility that greater community interactivity sooner, perhaps starting even next week? That's not out of the realm of possibilities, with wikis. -michael turner >> On Wed, Jun 6, 2012 at 7:53 AM, Richard O'Keefe wrote: >>> >>> On 6/06/2012, at 3:30 AM, Michael Turner wrote: >>>> But really: just give me a wiki. Give everybody a wiki. Write scripts >>>> to update the repo from the wiki diffs, if that's so important. >>> >>> For the sake of suffering humanity, forfend! >>> >>> The next time someone offers me a wiki instead of documentation, >>> I am going to be exceedingly angry. ?I have never yet found any >>> wikified documentation that was worth printing to wipe my arse with. >>> >>> I do apologise for the vulgarity, but good documentation needs >>> good technical writing and technical writing is a skill that is >>> distinct from programming, although some people can do both well. >>> It needs a *global* view as well as attention to fine detail. >>> >>>> Generally, lower the bar for entry, for those who want to improve the >>>> documentation, and the documentation *will* improve. >>> >>> Someone recently wrote something about the sky turning green then >>> yellow, didn't they? ?How is *lowering the bar* going to result in >>> anything *improving*? ?It might get more *voluminous*, but better? >>> >>> What we really need is a small number of good technical writers >>> and money to pay them. >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From siddharth.karandikar@REDACTED Wed Jun 6 15:56:52 2012 From: siddharth.karandikar@REDACTED (Siddharth Karandikar) Date: Wed, 6 Jun 2012 19:26:52 +0530 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <242fd643-abff-4d30-b5d8-e228508d6c69@knuth> References: <1D9CD642-72A7-4366-9379-CB2F478F23A3@gmail.com> <242fd643-abff-4d30-b5d8-e228508d6c69@knuth> Message-ID: Thanks Anthony for quickly pointing out typos and enhancements. * it should be erlang-questions@; >> Fixed * the spaces in the subject should be encoded as %20. >> Done * prepend the subject with "Re: ". >> Done * Using quoted subject in google query >> Done Thanks Magnus for very interesting gmane info. It has been incorporated. :) * Giving http://mid.gmane.org/ URL >> Done On Wed, Jun 6, 2012 at 6:48 PM, Magnus Henoch wrote: > Another way to accomplish that: since you have the message id for the question, > you can make a link to mid.gmane.org, like this: > > http://mid.gmane.org/CAFVMu-pzwGOQp8JKjKJh_w3L2oF5goGCPCQGopMP8nxGEosCVA%40mail.gmail.com > > ----- Original Message ----- >> The ">>" link could also be improved by adding the month to the site: >> URL >> and putting the subject in intitle: surrounded with quotes: >> >> ? ? ? site:http://erlang.org/pipermail/erlang-questions/ >> ? ? ? [erlang-questions] -heart question >> ? ? ? site:http://erlang.org/pipermail/erlang-questions/ >> ? ? ? intitle:"[erlang-questions] -heart question" >> >> I'm pretty confident it could be an "I'm Feeling Lucky" URL with >> those >> improvements. >> >> Regards, >> >> -- >> Anthony Ramine >> >> >> Le 6 juin 2012 ? 14:23, Anthony Ramine a ?crit : >> >> > Great! >> > >> > There is a small typo and an encoding error in the mailto links >> > though: >> > >> > * it should be erlang-questions@; >> > * the spaces in the subject should be encoded as %20. >> > >> > Furthermore, I think you should prepend the subject with "Re: ". >> > >> > Thanks for your work, maybe you should publish the code on GitHub >> > for >> > further improvements! >> > >> > -- >> > Anthony Ramine >> > >> > Le 6 juin 2012 ? 13:47, Siddharth Karandikar a ?crit : >> > >> >> Hi all, >> >> >> >> I have put together a program for this. >> >> >> >> This program fetches email archives from erlang-questions and uses >> >> information available in email headers to identify unanswered >> >> mails. >> >> It then generates pages to browse through these mails, view and >> >> reply. >> >> >> >> This program is currently running on my server. Its currently >> >> configured to keep track of mails within last 120 days. I am >> >> fetching >> >> new content and updating the pages twice a day. >> >> >> >> Please have a look - http://178.79.140.33/index.html >> >> Comments/suggestions are welcome! >> >> >> >> On the side note, Can anyone tell me how frequently does these >> >> mail >> >> archives get updated? So that I will tune the schedule on my >> >> server >> >> accordingly. >> >> >> >> Thanks, >> >> Siddharth >> >> >> >> On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong >> >> wrote: >> >>> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson >> >>> wrote: >> >>>> On 01/06/2012 08:22, Joe Armstrong wrote: >> >>>>> >> >>>>> I think it would be a good idea if those of us who might >> >>>>> qualify as being >> >>>>> "knowledgeable persons" could take the trouble to occasionally >> >>>>> answer >> >>>>> questions from apparent beginners that have not been answered. >> >>>>> >> >>>>> After a threshold of say 3 days - then at least *somebody* >> >>>>> could attempt >> >>>>> an answer so that beginner to Erlang don't feel unloved when >> >>>>> they >> >>>>> reach to this list. >> >>>> >> >>>> >> >>>> It would be easier to do this if there was a some kind of >> >>>> separate feed for >> >>>> mail that hasn't been answered in the last 3 days or whatever. >> >>>> Otherwise you >> >>>> loose a lot of cycles actually keeping track of who has asked >> >>>> what, and I >> >>>> suspect that is the time killer for a lot of people, rather than >> >>>> not having >> >>>> time to answer basic questions. >> >>>> >> >>>> I have *no idea* about managing mailing list software, so I'm >> >>>> not even sure >> >>>> if that idea is feasible. >> >>> >> >>> Pretty easy to write an erlang program to do this and run it once >> >>> a >> >>> day with crontab >> >>> >> >>> The mails can be obtained programmatically from >> >>> >> >>> http://erlang.org/pipermail/erlang-questions/ >> >>> >> >>> The rest is 'left as an exercise to the reader' >> >>> >> >>> /Joe >> >>> >> >>>>> >> >>>>> Of course, some extremely knowledgable and well-known people >> >>>>> ask >> >>>>> extremely difficult >> >>>>> questions here - and zero replies should probably >> >>>>> be taken as a sign that "nobody knows the answer" >> >>>>> >> >>>>> We could even have volunteers among those who posses the >> >>>>> knowledge to >> >>>>> answer >> >>>>> all unanswered beginners questions. >> >>>>> >> >>>>> >> >>>>> Cheers >> >>>>> >> >>>>> /Joe >> >>>>> _______________________________________________ >> >>>>> erlang-questions mailing list >> >>>>> erlang-questions@REDACTED >> >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> >> >>>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- > Magnus Henoch > Erlang Solutions Ltd > http://www.erlang-solutions.com/ > From mahesh@REDACTED Wed Jun 6 16:38:59 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Wed, 6 Jun 2012 10:38:59 -0400 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> Message-ID: <4BB28964-22BA-4A22-87E4-6C83E9877CA1@dieswaytoofast.com> http://www.youtube.com/watch?v=R8yLKEj1WhQ&feature=related Seriously tho'. If, and when you get around to it, some of us would be ridiculously grateful, and would happily by you as much beer as you could possibly drink. Or Negronis. Whichever :-) Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Blog | Twitter | Google+ On Jun 6, 2012, at 9:33 AM, Ulf Wiger wrote: > > Yes, this is correct. > > Gproc supports a few different types of register entries: > > - names, which are unique (can be any term) > - properties, which can be held by multiple processes simultaneously > - counters, which are properties with update_counter() semantics > - aggregate counters, which maintain the sum of all counters with > the same name as the aggregate counter. > > Properties are very useful for pub-sub, but also for highlighting various characteristics of the process. In my initial presentation of gproc, I had integrated it into OTP and modified the behaviors to register a {p,l,{behaviour,B}} property. This way, you could easily query the system for a certain type of behavior, and drill further to inspect other things. > > (This had the unfortunate side-effect that many thought you had to have a patched OTP to use gproc. Next time, I will try to be smarter). > > Thus, if you inspect a process, either with gproc:i(), or gproc:info(Pid [, gproc]), you can easily tell from its registered names and properties quite a bit about the process. > > BTW, I've been thinking about adding a function, gproc:bcast(Key, Msg), which would be similar to: > > rpc:abcast(gproc, send, [Key, Msg]) > > but with more predictable sequencing behavior, most likely making use of a gproc_bcast server to make sure all messages go the same way. > > Wild cheers and enthusiasm may intice me to do it sooner rather than later?. ;-) > > BR, > Ulf W > > On 6 Jun 2012, at 14:50, Eric Moritz wrote: > >> Yes, if you register a property key, you can retrieve all the procs that have that property set. For instance a fanout pub/sub can be accomplished doing: >> >> subscribe(Channel) -> >> gproc:reg({p, l, {subscribers, >> Channel}}). >> >> publish(Channel, Msg) -> >> gproc:send({p, l, {subscribers, >> Channel}}, >> Msg). >> >> If a process calls publish/2, the message will be sent to every process that called subscribe/1. >> >> I'm on my phone so it's a bit hard to look up the details, if you want to get a list of pids that called subscribe, look at the code of gproc:send/2. I think the function is called lookup_pids/1 but I may be wrong. >> >> The etorrent project uses gproc. That's a good read if you want to see how to use gproc. He uses property keys for each torrent file process as well as using await to delay initialization of gen_servers that depend on other gen_servers. >> Eric Moritz. >> On Jun 6, 2012 8:27 AM, "Lo?c Hoguin" wrote: >> On 05/26/2012 10:38 PM, Motiejus Jak?tys wrote: >> Hi, >> >> Hello, >> >> For term() to pid() mapping I've always used gproc. For the same thing >> my coworker always used pg2. >> >> I am making a comparison now. There are some features in gproc which are >> not in pg2: >> >> * Await for registration >> * Per-process properties >> * Aggregate counters >> * QLC >> >> Whereas pg2 "natively" supports mapping one key to several processes >> (which can be easily achieved with QLC in gproc). >> >> Unless I misunderstand, gproc's properties are exactly this, mapping one key to several processes. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Jun 6 17:04:35 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 6 Jun 2012 11:04:35 -0400 Subject: [erlang-questions] debug output from yaws In-Reply-To: References: Message-ID: On Wed, Jun 6, 2012 at 9:15 AM, Attila Rajmund Nohl wrote: > Hello! > > I'm working on an application that uses the yaws webserver. This > application also contains the reshd module (see > http://erlang.org/pipermail/erlang-questions/2001-May/003162.html), so > I can connect to the erlang shell of the node even if yaws was started > as daemon mode. My problem is: if I connect to the shell and turn on > tracing (using dbg), I can't see it's output, because the stdout and > stderr are redirected to /dev/null. > > Is there a way to get the debug output without restarting yaws? Why not use dbg:trace_port/2 with dbg:tracer/2 to redirect the trace output to a file or to another system? --steve From n.oxyde@REDACTED Wed Jun 6 17:09:14 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 6 Jun 2012 17:09:14 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <1D9CD642-72A7-4366-9379-CB2F478F23A3@gmail.com> <242fd643-abff-4d30-b5d8-e228508d6c69@knuth> Message-ID: You're welcome! I made a mistake in my last email, I suggested adding the month in the google search but didn't do it right in the example: You already have the months of the mails so you could do this: site:http://erlang.org/pipermail/erlang-questions/ "[erlang-questions] multiple behaviours" site:http://erlang.org/pipermail/erlang-questions/2012-May/ "[erlang-questions] multiple behaviours" -- Anthony Ramine Le 6 juin 2012 ? 15:56, Siddharth Karandikar a ?crit : > Thanks Anthony for quickly pointing out typos and enhancements. > * it should be erlang-questions@; >>> Fixed > * the spaces in the subject should be encoded as %20. >>> Done > * prepend the subject with "Re: ". >>> Done > * Using quoted subject in google query >>> Done > > Thanks Magnus for very interesting gmane info. > It has been incorporated. :) > * Giving http://mid.gmane.org/ URL >>> Done > > > On Wed, Jun 6, 2012 at 6:48 PM, Magnus Henoch > wrote: >> Another way to accomplish that: since you have the message id for the question, >> you can make a link to mid.gmane.org, like this: >> >> http://mid.gmane.org/CAFVMu-pzwGOQp8JKjKJh_w3L2oF5goGCPCQGopMP8nxGEosCVA%40mail.gmail.com >> >> ----- Original Message ----- >>> The ">>" link could also be improved by adding the month to the site: >>> URL >>> and putting the subject in intitle: surrounded with quotes: >>> >>> site:http://erlang.org/pipermail/erlang-questions/ >>> [erlang-questions] -heart question >>> site:http://erlang.org/pipermail/erlang-questions/ >>> intitle:"[erlang-questions] -heart question" >>> >>> I'm pretty confident it could be an "I'm Feeling Lucky" URL with >>> those >>> improvements. >>> >>> Regards, >>> >>> -- >>> Anthony Ramine >>> >>> >>> Le 6 juin 2012 ? 14:23, Anthony Ramine a ?crit : >>> >>>> Great! >>>> >>>> There is a small typo and an encoding error in the mailto links >>>> though: >>>> >>>> * it should be erlang-questions@; >>>> * the spaces in the subject should be encoded as %20. >>>> >>>> Furthermore, I think you should prepend the subject with "Re: ". >>>> >>>> Thanks for your work, maybe you should publish the code on GitHub >>>> for >>>> further improvements! >>>> >>>> -- >>>> Anthony Ramine >>>> >>>> Le 6 juin 2012 ? 13:47, Siddharth Karandikar a ?crit : >>>> >>>>> Hi all, >>>>> >>>>> I have put together a program for this. >>>>> >>>>> This program fetches email archives from erlang-questions and uses >>>>> information available in email headers to identify unanswered >>>>> mails. >>>>> It then generates pages to browse through these mails, view and >>>>> reply. >>>>> >>>>> This program is currently running on my server. Its currently >>>>> configured to keep track of mails within last 120 days. I am >>>>> fetching >>>>> new content and updating the pages twice a day. >>>>> >>>>> Please have a look - http://178.79.140.33/index.html >>>>> Comments/suggestions are welcome! >>>>> >>>>> On the side note, Can anyone tell me how frequently does these >>>>> mail >>>>> archives get updated? So that I will tune the schedule on my >>>>> server >>>>> accordingly. >>>>> >>>>> Thanks, >>>>> Siddharth >>>>> >>>>> On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong >>>>> wrote: >>>>>> On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson >>>>>> wrote: >>>>>>> On 01/06/2012 08:22, Joe Armstrong wrote: >>>>>>>> >>>>>>>> I think it would be a good idea if those of us who might >>>>>>>> qualify as being >>>>>>>> "knowledgeable persons" could take the trouble to occasionally >>>>>>>> answer >>>>>>>> questions from apparent beginners that have not been answered. >>>>>>>> >>>>>>>> After a threshold of say 3 days - then at least *somebody* >>>>>>>> could attempt >>>>>>>> an answer so that beginner to Erlang don't feel unloved when >>>>>>>> they >>>>>>>> reach to this list. >>>>>>> >>>>>>> >>>>>>> It would be easier to do this if there was a some kind of >>>>>>> separate feed for >>>>>>> mail that hasn't been answered in the last 3 days or whatever. >>>>>>> Otherwise you >>>>>>> loose a lot of cycles actually keeping track of who has asked >>>>>>> what, and I >>>>>>> suspect that is the time killer for a lot of people, rather than >>>>>>> not having >>>>>>> time to answer basic questions. >>>>>>> >>>>>>> I have *no idea* about managing mailing list software, so I'm >>>>>>> not even sure >>>>>>> if that idea is feasible. >>>>>> >>>>>> Pretty easy to write an erlang program to do this and run it once >>>>>> a >>>>>> day with crontab >>>>>> >>>>>> The mails can be obtained programmatically from >>>>>> >>>>>> http://erlang.org/pipermail/erlang-questions/ >>>>>> >>>>>> The rest is 'left as an exercise to the reader' >>>>>> >>>>>> /Joe >>>>>> >>>>>>>> >>>>>>>> Of course, some extremely knowledgable and well-known people >>>>>>>> ask >>>>>>>> extremely difficult >>>>>>>> questions here - and zero replies should probably >>>>>>>> be taken as a sign that "nobody knows the answer" >>>>>>>> >>>>>>>> We could even have volunteers among those who posses the >>>>>>>> knowledge to >>>>>>>> answer >>>>>>>> all unanswered beginners questions. >>>>>>>> >>>>>>>> >>>>>>>> Cheers >>>>>>>> >>>>>>>> /Joe >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -- >> Magnus Henoch >> Erlang Solutions Ltd >> http://www.erlang-solutions.com/ >> From daniel.goertzen@REDACTED Wed Jun 6 17:35:16 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 6 Jun 2012 10:35:16 -0500 Subject: [erlang-questions] cowboy Accept header, rest module Message-ID: I have 2 cowboy questions: 1. I am trying to parse the Accept header using... cowboy_http_req:parse_header('Accept', Req) The raw header is... *{'Accept',<<"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8">>} * * * *and the output is ...* * [{{<<"text">>,<<"html">>,[]},1000,[]}, {{<<"application">>,<<"xhtml+xml">>,[]},1000,[]}, {{<<"application">>,<<"xml">>,[]},900,[]}, {{<<"*">>,<<"*">>,[]},800,[]}]* What do all the parts of the tuple mean? (it is mainly the number that is piquing my curiosity) 2. I am looking to make a restful web interface using cowboy. The basic cowboy_http_protocol module with its dispatch table seems very nice for this, and along with its request parsing I have difficulty imagining what more I could need. What does the cowboy_http_rest module buy you? Note that this is my first web interface in any language and I had trouble penetrating the webmachine documentation (I suspect knowing nothing about mochiweb is causing problems for me). Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Jun 6 17:37:54 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 6 Jun 2012 17:37:54 +0200 Subject: [erlang-questions] debug output from yaws In-Reply-To: References: Message-ID: 2012/6/6 Steve Vinoski : > On Wed, Jun 6, 2012 at 9:15 AM, Attila Rajmund Nohl > wrote: >> Hello! >> >> I'm working on an application that uses the yaws webserver. This >> application also contains the reshd module (see >> http://erlang.org/pipermail/erlang-questions/2001-May/003162.html), so >> I can connect to the erlang shell of the node even if yaws was started >> as daemon mode. My problem is: if I connect to the shell and turn on >> tracing (using dbg), I can't see it's output, because the stdout and >> stderr are redirected to /dev/null. >> >> Is there a way to get the debug output without restarting yaws? > > Why not use dbg:trace_port/2 with dbg:tracer/2 to redirect the trace > output to a file or to another system? I was hoping for something simpler, so I could get not only the output from dbg, but from io:format/2 calls too. If there's no other solution, I have to use something like that you propose, and also io:format/3. From cgsmcmlxxv@REDACTED Wed Jun 6 17:40:58 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 6 Jun 2012 17:40:58 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: Siddharth, are you sure the list is correct? Related to the unanswered messages, there are some mails containing advertisement for different Erlang-related events and I don't suppose those need answers via erlang-question mailing list. Usually, I follow erlang-question (along with some others) daily and the newbies questions are answered quite fast, especially if they are noobs questions. As you can notice, I am not answering too often and that supports what I just said. I am not a knowledgeable user of Erlang, so, I may not be able to answer to all the questions, but I think the Erlang community is active and knowledgeable enough and it doesn't need more "energy drinks". And, yes, I agree that if someone has his/her message unanswered, he/she should kindly bump his/her message. CGS On Wed, Jun 6, 2012 at 1:47 PM, Siddharth Karandikar < siddharth.karandikar@REDACTED> wrote: > Hi all, > > I have put together a program for this. > > This program fetches email archives from erlang-questions and uses > information available in email headers to identify unanswered mails. > It then generates pages to browse through these mails, view and reply. > > This program is currently running on my server. Its currently > configured to keep track of mails within last 120 days. I am fetching > new content and updating the pages twice a day. > > Please have a look - http://178.79.140.33/index.html > Comments/suggestions are welcome! > > On the side note, Can anyone tell me how frequently does these mail > archives get updated? So that I will tune the schedule on my server > accordingly. > > Thanks, > Siddharth > > On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: > > On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson > wrote: > >> On 01/06/2012 08:22, Joe Armstrong wrote: > >>> > >>> I think it would be a good idea if those of us who might qualify as > being > >>> "knowledgeable persons" could take the trouble to occasionally answer > >>> questions from apparent beginners that have not been answered. > >>> > >>> After a threshold of say 3 days - then at least *somebody* could > attempt > >>> an answer so that beginner to Erlang don't feel unloved when they > >>> reach to this list. > >> > >> > >> It would be easier to do this if there was a some kind of separate feed > for > >> mail that hasn't been answered in the last 3 days or whatever. > Otherwise you > >> loose a lot of cycles actually keeping track of who has asked what, and > I > >> suspect that is the time killer for a lot of people, rather than not > having > >> time to answer basic questions. > >> > >> I have *no idea* about managing mailing list software, so I'm not even > sure > >> if that idea is feasible. > > > > Pretty easy to write an erlang program to do this and run it once a > > day with crontab > > > > The mails can be obtained programmatically from > > > > http://erlang.org/pipermail/erlang-questions/ > > > > The rest is 'left as an exercise to the reader' > > > > /Joe > > > >>> > >>> Of course, some extremely knowledgable and well-known people ask > >>> extremely difficult > >>> questions here - and zero replies should probably > >>> be taken as a sign that "nobody knows the answer" > >>> > >>> We could even have volunteers among those who posses the knowledge to > >>> answer > >>> all unanswered beginners questions. > >>> > >>> > >>> Cheers > >>> > >>> /Joe > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 6 17:41:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 06 Jun 2012 17:41:23 +0200 Subject: [erlang-questions] cowboy Accept header, rest module In-Reply-To: References: Message-ID: <4FCF7A23.4030903@ninenines.eu> On 06/06/2012 05:35 PM, Daniel Goertzen wrote: > I have 2 cowboy questions: > > 1. I am trying to parse the Accept header using... > > cowboy_http_req:parse_header('Accept', Req) > > The raw header is... > > *{'Accept',<<"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8">>}* > * > * > *and the output is ...* > > *[{{<<"text">>,<<"html">>,[]},1000,[]}, > {{<<"application">>,<<"xhtml+xml">>,[]},1000,[]}, > {{<<"application">>,<<"xml">>,[]},900,[]}, > {{<<"*">>,<<"*">>,[]},800,[]}]* > > > What do all the parts of the tuple mean? (it is mainly the number that > is piquing my curiosity) {{type, subtype, params}, qvalue, extensions} qvalue is pretty much the 0.9 * 1000, so a value between 0 and 1000 indicating the weight of the media type. > 2. I am looking to make a restful web interface using cowboy. The basic > cowboy_http_protocol module with its dispatch table seems very nice for > this, and along with its request parsing I have difficulty imagining > what more I could need. What does the cowboy_http_rest module buy you? > Note that this is my first web interface in any language and I had > trouble penetrating the webmachine documentation (I suspect knowing > nothing about mochiweb is causing problems for me). It does everything for you automatically. You just need to write a few callbacks describing the resource, and all the header/status code jungle is done automatically for you following REST conventions. You can use normal HTTP handlers, but if you're going to write an API I'd recommend using REST. Just check the diagram and resource infos from the Webmachine documentation, the rest of it (dispatch, replying and retrieving infos about the request) you should look at the Cowboy documentation. I'm always available for questions, feel free to mail me or open tickets if something happens in an unexpected way. -- Lo?c Hoguin Erlang Cowboy Nine Nines From hd2010@REDACTED Wed Jun 6 18:20:42 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 18:20:42 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <4FCF835A.4080004@eonblast.com> On 6/6/12 5:40 PM, CGS wrote: > more "energy drinks". I like it. I'll donate www.notanswered.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From siddharth.karandikar@REDACTED Wed Jun 6 18:21:09 2012 From: siddharth.karandikar@REDACTED (Siddharth Karandikar) Date: Wed, 6 Jun 2012 21:51:09 +0530 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: Current logic to identify unanswered mails is purely based on mail-ids found in mail headers. So it is very much possible that many of announcement kind of mails will go unanswered (as no answered is expected) and will show up here in the list. Thanks, Siddharth On Wed, Jun 6, 2012 at 9:10 PM, CGS wrote: > Siddharth, are you sure the list is correct? > > Related to the unanswered messages, there are some mails containing > advertisement for different Erlang-related events and I don't suppose those > need answers via erlang-question mailing list. > > Usually, I follow erlang-question (along with some others) daily and the > newbies questions are answered quite fast, especially if they are noobs > questions. As you can notice, I am not answering too often and that supports > what I just said. I am not a knowledgeable user of Erlang, so, I may not be > able to answer to all the questions, but I think the Erlang community is > active and knowledgeable enough and it doesn't need more "energy drinks". > > And, yes, I agree that if someone has his/her message unanswered, he/she > should kindly bump his/her message. > > CGS > > > > > On Wed, Jun 6, 2012 at 1:47 PM, Siddharth Karandikar > wrote: >> >> Hi all, >> >> I have put together a program for this. >> >> This program fetches email archives from erlang-questions and uses >> information available in email headers to identify unanswered mails. >> It then generates pages to browse through these mails, view and reply. >> >> This program is currently running on my server. Its currently >> configured to keep track of mails within last 120 days. I am fetching >> new content and updating the pages twice a day. >> >> Please have a look - http://178.79.140.33/index.html >> Comments/suggestions are welcome! >> >> On the side note, Can anyone tell me how frequently does these mail >> archives get updated? So that I will tune the schedule on my server >> accordingly. >> >> Thanks, >> Siddharth >> >> On Fri, Jun 1, 2012 at 1:06 PM, Joe Armstrong wrote: >> > On Fri, Jun 1, 2012 at 9:27 AM, Tim Watson >> > wrote: >> >> On 01/06/2012 08:22, Joe Armstrong wrote: >> >>> >> >>> I think it would be a good idea if those of us who might qualify as >> >>> being >> >>> "knowledgeable persons" could take the trouble to occasionally answer >> >>> ?questions from apparent beginners that have not been answered. >> >>> >> >>> After a threshold of say 3 days - then at least *somebody* could >> >>> attempt >> >>> an answer so that beginner to Erlang don't feel unloved when they >> >>> reach to this list. >> >> >> >> >> >> It would be easier to do this if there was a some kind of separate feed >> >> for >> >> mail that hasn't been answered in the last 3 days or whatever. >> >> Otherwise you >> >> loose a lot of cycles actually keeping track of who has asked what, and >> >> I >> >> suspect that is the time killer for a lot of people, rather than not >> >> having >> >> time to answer basic questions. >> >> >> >> I have *no idea* about managing mailing list software, so I'm not even >> >> sure >> >> if that idea is feasible. >> > >> > Pretty easy to write an erlang program to do this and run it once a >> > day with crontab >> > >> > The mails can be obtained programmatically from >> > >> > http://erlang.org/pipermail/erlang-questions/ >> > >> > The rest is 'left as an exercise to the reader' >> > >> > /Joe >> > >> >>> >> >>> Of course, some extremely knowledgable and well-known people ask >> >>> extremely difficult >> >>> questions here - and zero replies should probably >> >>> be taken as a sign that "nobody knows the answer" >> >>> >> >>> We could even have volunteers among those who posses the knowledge to >> >>> answer >> >>> all unanswered beginners questions. >> >>> >> >>> >> >>> Cheers >> >>> >> >>> /Joe >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From mjtruog@REDACTED Wed Jun 6 18:44:59 2012 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 06 Jun 2012 09:44:59 -0700 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: References: <20120526203817.GA10058@localhost> Message-ID: <4FCF890B.4080302@gmail.com> The other differences are mentioned in previous email threads. pg2 replicates all name lookup information in a way that doesn't require consistency (it does use ets though, if you don't want to use ets, there is an example here https://raw.github.com/okeuday/CloudI/master/src/lib/cloudi/src/list_pg.erl). gproc uses gen_leader so it maintains a central name lookup authority. So, pg2 is AP from the CAP theorem, while gproc is CA. That means gproc has issues with partition tolerance, similar to mnesia. To me, gproc is a method by which you are able to introduce global variables into your Erlang program. This helps beginners that are not working on large codebases, but it is generally only for Erlang that is limited to a single node. An implementation of gen_leader behavior that is suppose to have better partition tolerance is here https://github.com/ngmoco/gl_async_bully , but I don't know the details. I don't see leader election as necessary in Erlang, though it is a fun intellectual exercise. On 06/06/2012 05:10 AM, Motiejus Jak?tys wrote: > Bump > > On Sat, May 26, 2012 at 10:38 PM, Motiejus Jak?tys > wrote: >> Hi, >> >> For term() to pid() mapping I've always used gproc. For the same thing >> my coworker always used pg2. >> >> I am making a comparison now. There are some features in gproc which are >> not in pg2: >> >> * Await for registration >> * Per-process properties >> * Aggregate counters >> * QLC >> >> Whereas pg2 "natively" supports mapping one key to several processes >> (which can be easily achieved with QLC in gproc). >> >> Any more differences between them? From ulf@REDACTED Wed Jun 6 19:03:01 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 19:03:01 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <4FCF890B.4080302@gmail.com> References: <20120526203817.GA10058@localhost> <4FCF890B.4080302@gmail.com> Message-ID: <69062D57-C01C-4C98-808D-DBF1D7B1857C@feuerlabs.com> This is true. I prefer to recommend people to use gproc as a mainly local registry. Of course, just as with the built-in registry, using it doesn't limit your system to running on one node. It's just that the registry is local to each node. ;-) The newest versions of gen_leader are supposedly more tolerant to netsplits, but no facility to merge the user-level data exists (esp. not in gproc). In this regard, mnesia is much better off, since there are ways to deal with netsplits and recover to a consistent state (even though those ways may not appeal to everyone). Of course, as long as this issue remains, one should at least think twice before relying on global registration in gproc. It can still work, as long as there are system-level recovery measures after netsplits that also fix gproc (e.g. the system detects the problem, identifies a consistent island, and restarts all other nodes). BR, Ulf W On 6 Jun 2012, at 18:44, Michael Truog wrote: > The other differences are mentioned in previous email threads. pg2 replicates all name lookup information in a way that doesn't require consistency (it does use ets though, if you don't want to use ets, there is an example herehttps://raw.github.com/okeuday/CloudI/master/src/lib/cloudi/src/list_pg.erl). gproc uses gen_leader so it maintains a central name lookup authority. So, pg2 is AP from the CAP theorem, while gproc is CA. That means gproc has issues with partition tolerance, similar to mnesia. To me, gproc is a method by which you are able to introduce global variables into your Erlang program. This helps beginners that are not working on large codebases, but it is generally only for Erlang that is limited to a single node. An implementation of gen_leader behavior that is suppose to have better partition tolerance is here https://github.com/ngmoco/gl_async_bully , but I don't know the details. I don't see leader election as necessary in Erlang, though it is a > fun intellectual exercise. Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From dmercer@REDACTED Wed Jun 6 19:07:06 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 6 Jun 2012 12:07:06 -0500 Subject: [erlang-questions] Erlang and the TDS protocol In-Reply-To: References: Message-ID: <015601cd4406$cdf59a10$69e0ce30$@com> Just checking to see if anyone replied to you on this and did not reply-all the list. I would be interested in hearing the answer, too, if one is available. Cheers, DBM From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Marcel Meyer Sent: Sunday, June 03, 2012 4:39 PM To: erlang-questions Subject: [erlang-questions] Erlang and the TDS protocol Dear list, I am curious why there isn't a native Erlang TDS protocol implementation. Even /otp/lib wraps a C ODBC implementation using a port. Would this not be simpler and more concise using bit syntax, since the TCP datagram unpacks in 1 line of code (more for readability, of course)? It just seems like extra work: C wraps the msg, port wraps C, ODBC app wraps port etc. There is a lot of 'wrapping' going on. Any input would be much appreciated. Kind regards, Marcel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Wed Jun 6 19:09:43 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 06 Jun 2012 19:09:43 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <4FCF8ED7.6000201@eonblast.com> On 6/6/12 5:40 PM, CGS wrote: > Siddharth, are you sure the list is correct? > > Related to the unanswered messages, there are some mails containing > advertisement for different Erlang-related events and I don't suppose > those need answers via erlang-question mailing list. > You could add an 'is not a question'-button ... then it's become stateful but the data is not that much. Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Jun 6 19:55:20 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2012 19:55:20 +0200 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <4BB28964-22BA-4A22-87E4-6C83E9877CA1@dieswaytoofast.com> References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> <4BB28964-22BA-4A22-87E4-6C83E9877CA1@dieswaytoofast.com> Message-ID: <79EB78FC-6BB3-4017-A037-A0554E9E966B@feuerlabs.com> How to refuse such an offer? :) Well, a first version can be tested now. https://github.com/esl/gproc/blob/master/doc/gproc.md#bcast-2 BR, Ulf W On 6 Jun 2012, at 16:38, Mahesh Paolini-Subramanya wrote: > http://www.youtube.com/watch?v=R8yLKEj1WhQ&feature=related > > Seriously tho'. If, and when you get around to it, some of us would be ridiculously grateful, and would happily by you as much beer as you could possibly drink. Or Negronis. Whichever :-) > > Mahesh Paolini-Subramanya > That Tall Bald Indian Guy... > Blog | Twitter | Google+ > > On Jun 6, 2012, at 9:33 AM, Ulf Wiger wrote: > >> >> Yes, this is correct. >> >> Gproc supports a few different types of register entries: >> >> - names, which are unique (can be any term) >> - properties, which can be held by multiple processes simultaneously >> - counters, which are properties with update_counter() semantics >> - aggregate counters, which maintain the sum of all counters with >> the same name as the aggregate counter. >> >> Properties are very useful for pub-sub, but also for highlighting various characteristics of the process. In my initial presentation of gproc, I had integrated it into OTP and modified the behaviors to register a {p,l,{behaviour,B}} property. This way, you could easily query the system for a certain type of behavior, and drill further to inspect other things. >> >> (This had the unfortunate side-effect that many thought you had to have a patched OTP to use gproc. Next time, I will try to be smarter). >> >> Thus, if you inspect a process, either with gproc:i(), or gproc:info(Pid [, gproc]), you can easily tell from its registered names and properties quite a bit about the process. >> >> BTW, I've been thinking about adding a function, gproc:bcast(Key, Msg), which would be similar to: >> >> rpc:abcast(gproc, send, [Key, Msg]) >> >> but with more predictable sequencing behavior, most likely making use of a gproc_bcast server to make sure all messages go the same way. >> >> Wild cheers and enthusiasm may intice me to do it sooner rather than later?. ;-) >> >> BR, >> Ulf W >> >> On 6 Jun 2012, at 14:50, Eric Moritz wrote: >> >>> Yes, if you register a property key, you can retrieve all the procs that have that property set. For instance a fanout pub/sub can be accomplished doing: >>> >>> subscribe(Channel) -> >>> gproc:reg({p, l, {subscribers, >>> Channel}}). >>> >>> publish(Channel, Msg) -> >>> gproc:send({p, l, {subscribers, >>> Channel}}, >>> Msg). >>> >>> If a process calls publish/2, the message will be sent to every process that called subscribe/1. >>> >>> I'm on my phone so it's a bit hard to look up the details, if you want to get a list of pids that called subscribe, look at the code of gproc:send/2. I think the function is called lookup_pids/1 but I may be wrong. >>> >>> The etorrent project uses gproc. That's a good read if you want to see how to use gproc. He uses property keys for each torrent file process as well as using await to delay initialization of gen_servers that depend on other gen_servers. >>> Eric Moritz. >>> On Jun 6, 2012 8:27 AM, "Lo?c Hoguin" wrote: >>> On 05/26/2012 10:38 PM, Motiejus Jak?tys wrote: >>> Hi, >>> >>> Hello, >>> >>> For term() to pid() mapping I've always used gproc. For the same thing >>> my coworker always used pg2. >>> >>> I am making a comparison now. There are some features in gproc which are >>> not in pg2: >>> >>> * Await for registration >>> * Per-process properties >>> * Aggregate counters >>> * QLC >>> >>> Whereas pg2 "natively" supports mapping one key to several processes >>> (which can be easily achieved with QLC in gproc). >>> >>> Unless I misunderstand, gproc's properties are exactly this, mapping one key to several processes. >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcel.meyer@REDACTED Wed Jun 6 20:22:59 2012 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Wed, 6 Jun 2012 14:22:59 -0400 Subject: [erlang-questions] Erlang and the TDS protocol In-Reply-To: <015601cd4406$cdf59a10$69e0ce30$@com> References: <015601cd4406$cdf59a10$69e0ce30$@com> Message-ID: Hi David, No replies yet. I was wanted to see how other db wire protocols were implemented, and saw that 'dizzyd' did the mysql driver natively. I spoke to a colleague of mine and we wondered if there might be performance issues with bit syntax and that the ODBC app wanted to go straight to the metal. The reason I was asking this is that we're talking about how you would build a proxy to a cluster of Sql Servers (not a Sql Server Cluster). Jan Kneschke of lighttpd fame is working on a MySql proxy, with failover/load balancing properties. Could be interesting for Sql Server... Regards, M On Wed, Jun 6, 2012 at 1:07 PM, David Mercer wrote: > Just checking to see if anyone replied to you on this and did not > reply-all the list. I would be interested in hearing the answer, too, if > one is available.**** > > ** ** > > Cheers,**** > > ** ** > > DBM**** > > ** ** > > *From:* erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] *On Behalf Of *Marcel Meyer > *Sent:* Sunday, June 03, 2012 4:39 PM > *To:* erlang-questions > *Subject:* [erlang-questions] Erlang and the TDS protocol**** > > ** ** > > Dear list,**** > > ** ** > > I am curious why there isn't a native Erlang TDS protocol implementation. > Even /otp/lib wraps a C ODBC implementation using a port.**** > > Would this not be simpler and more concise using bit syntax, since the TCP > datagram unpacks in 1 line of code (more for readability, of course)?**** > > It just seems like extra work: C wraps the msg, port wraps C, ODBC app > wraps port etc. There is a lot of 'wrapping' going on.**** > > ** ** > > Any input would be much appreciated.**** > > ** ** > > Kind regards,**** > > Marcel**** > > ** ** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From larshesel@REDACTED Wed Jun 6 22:49:20 2012 From: larshesel@REDACTED (Lars Hesel Christensen) Date: Wed, 6 Jun 2012 22:49:20 +0200 Subject: [erlang-questions] [ANN] Zurich Erlang User Group Message-ID: Hi all I just wanted to let you all know (and google and other search machines) that there now is an Erlang user group in Zurich / Switzerland. So if you live in, or are visiting Switzerland, you might want to check it out. For now we live on meetup.com: http://www.meetup.com/Zurich-Erlang-User-Group/, google groups: https://groups.google.com/forum/?fromgroups#!forum/zurich-erlang-user-group and twitter: https://twitter.com/#!/ZurichErlangUG Thank you, Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From spawn.think@REDACTED Wed Jun 6 23:25:49 2012 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 6 Jun 2012 23:25:49 +0200 Subject: [erlang-questions] [ANN] Zurich Erlang User Group In-Reply-To: References: Message-ID: Hi Lars, If you need also presence on LinkedIn, I created a subgroup under Erlang/OTP Professional Groups for members in Switzerland http://www.linkedin.com/groups?gid=4478020 There are already subgroups for many EUGs (Sweden, UK, Spain, Netherlands, Germany, USA, ...etc) Best of luck! Ahmed Omar On Wed, Jun 6, 2012 at 10:49 PM, Lars Hesel Christensen wrote: > Hi all > > I just wanted to let you all know (and google and other search machines) > that there now is an Erlang user group in Zurich / Switzerland. > > So if you live in, or are visiting Switzerland, you might want to check it > out. For now we live on meetup.com: > http://www.meetup.com/Zurich-Erlang-User-Group/, google groups: > https://groups.google.com/forum/?fromgroups#!forum/zurich-erlang-user-group and > twitter: https://twitter.com/#!/ZurichErlangUG > > Thank you, > Lars > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Thu Jun 7 00:40:12 2012 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 6 Jun 2012 23:40:12 +0100 Subject: [erlang-questions] up-to-date c node not working Message-ID: Hi Everyone So the instructions for writing a c-node here http://www.erlang.org/doc/tutorial/cnode.html , although still working are way out of date ( they don't use the newer ei_connect http://www.erlang.org/doc/man/ei_connect.html ) I can get the older example code working well but can't however get the newer ones going .. doing some google-ing it seems people as far back as 2005 have the same issue but nowhere can i find a fix. As far as i can tell i can successfully connect to a running Erlang node ( "e1@REDACTED" ) like so : ei_cnode ec; if (ei_connect_init(&ec, "e1", "secretcookie", n++) < 0) { printf("error!"); } else { printf("connected!"); } int fd = ei_connect(&ec, "e1@REDACTED"); if(fd<1) { printf("ei_connect error : %d",fd); return; } seems to work fine, and defiantly fails if i get the cookie wrong etc.. what does not work tho is trying to receive a message : ei_x_buff x; ei_x_new(&x); if (ei_xreceive_msg(fd, &msg, &x) > 0) { // do something } else{printf("failed");} what ever i try ei_xreceive_msg returns -1 .. if it's in a loop (as similar code found on the net) it will loop non stop and consume 95 % cpu time is there something simple i am doing wrong ? James -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Thu Jun 7 01:55:10 2012 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 7 Jun 2012 00:55:10 +0100 Subject: [erlang-questions] up-to-date c node not working In-Reply-To: References: Message-ID: Ok worked it out... the docs at http://www.erlang.org/doc/man/ei_connect.html use the same name "madonna"?for both the initialisation and the connection .. but the initialisation is for the c-node name and the connection needs the name of the erlang node to connect to... tad confusing that the docs use the same name.. but now it does work .. James On 6 June 2012 23:40, James Churchman wrote: > Hi Everyone > > So the instructions for writing a c-node here > http://www.erlang.org/doc/tutorial/cnode.html , although still working are > way out of date ( they don't use the > newer?ei_connect?http://www.erlang.org/doc/man/ei_connect.html ) > > I can get the older example code working well but can't however get the > newer ones going .. doing some google-ing it seems people as far back as > 2005 have the same issue but nowhere can i find a fix. > > As far as i can tell i can successfully connect to a running Erlang node ( > "e1@REDACTED" ) like so : > > ei_cnode ec; > > if (ei_connect_init(&ec, "e1", "secretcookie", n++) < 0) > { > ? ? printf("error!"); > } > else > { > ? ? printf("connected!"); > } > > int fd = ei_connect(&ec, "e1@REDACTED"); > if(fd<1) > { > ? ? printf("ei_connect error : %d",fd); > ? ? return; > } > > seems to work fine, and defiantly fails if i get the cookie wrong etc.. > > what does not work tho is trying to receive a message : > > ei_x_buff x; > ei_x_new(&x); > > > if (ei_xreceive_msg(fd, &msg, &x) > 0) > { > ? ? // do something > } > else{printf("failed");} > > > > what ever i try?ei_xreceive_msg returns -1 .. if it's in a loop (as similar > code found on the net) it will loop non stop and consume 95 % cpu time > > is there something simple i am doing wrong ? > > James From overminddl1@REDACTED Thu Jun 7 02:23:31 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 6 Jun 2012 18:23:31 -0600 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner wrote: >> and nobody >> bothers to correct the mistakes. > > Not "nobody". For example, I'm one of those weird people who will > spend 30 seconds to sign up for almost any wiki I see a typo in. For what it is worth, I am one as well. Some sort of user-editable documentation (even something that I could comment on like the Haskell one) would be nice. From overminddl1@REDACTED Thu Jun 7 02:49:55 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 6 Jun 2012 18:49:55 -0600 Subject: [erlang-questions] pg2 vs gproc? In-Reply-To: <79EB78FC-6BB3-4017-A037-A0554E9E966B@feuerlabs.com> References: <20120526203817.GA10058@localhost> <4FCF4CC4.8040206@ninenines.eu> <4BB28964-22BA-4A22-87E4-6C83E9877CA1@dieswaytoofast.com> <79EB78FC-6BB3-4017-A037-A0554E9E966B@feuerlabs.com> Message-ID: On Wed, Jun 6, 2012 at 11:55 AM, Ulf Wiger wrote: > > How to refuse such an offer? ?:) > > Well, a first version can be tested now. > > https://github.com/esl/gproc/blob/master/doc/gproc.md#bcast-2 Awesome, cannot wait to try it out. :) From ok@REDACTED Thu Jun 7 03:53:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 7 Jun 2012 13:53:20 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On 6/06/2012, at 7:26 PM, Michael Turner wrote: >> What we really need is a small number of good technical writers >> and money to pay them. > > Agreed. But guess what? Page after page of Erlang/OTP documentation > can be produced as evidence that the team doesn't have the budget for > it. Non sequitur. It demonstrates that the money _has_ not been spent on good technical writers, not that the money _could_ not have been so spent. > > Given *reality*, I propose a wiki. But that's way too close to the process that gave us the documentation we have. Every time I read any other outfit's Prolog documentation, I recall the Quintus Prolog documentation with gladness. Quintus had one policy which led to good documentation, and it is precisely that policy you are proposing to throw overboard wearing concrete shoes: Never let an engineer write the documentation. Our first technical writer, Kennita Watson, was technically competent. (From Stanford, so how could she not be?) But she wasn't on the development team. On more than one occasion her "how am I expected to document _that_?" resulted in changes to designs we "engineers" were perfectly happy with but that customers would not have been. Only *good* technical writers are needed, not *great* ones. And one would be a good start. From ok@REDACTED Thu Jun 7 04:12:25 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 7 Jun 2012 14:12:25 +1200 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? Message-ID: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> Zhemzhitsky Sergey troika.ru> asked about the difference between the eterm interface and the ei interface. My understanding of this is that the eterm interface lets you manipulate Erlang terms, while the ei interface lets you manipulate the (serialised) binary representation of terms. eterm: walk around/generate unpacked trees without ever going through binaries ei: sequentially generate/parse packed binaries without going through trees Does this answer the question? Why was it hard to see in the documentation? I thought the first couple of paragraphs of www.erlang.org/doc/man/{erl_eterm,ei}.html made it pretty clear. From ok@REDACTED Thu Jun 7 05:56:03 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 7 Jun 2012 15:56:03 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <049344B4-4434-459E-ABDF-4C3D73C87194@cs.otago.ac.nz> On 7/06/2012, at 1:50 AM, Michael Turner wrote: > On Wed, Jun 6, 2012 at 7:36 PM, Joe Armstrong wrote: > >> I agree with Richard on this one - a wiki cannot answer "why" questions - >> inspecting the code might reveal *how* it works but not *why* it was >> written this way. > > I don't see where Richard specifically raised that point, I read this as "I agree with Richard on this one. One reason for this is that a wiki cannot answer ..." > but it's > incorrect in any case. I think you are misreading Joe here. What I think he means is that only someone who was around and involved when a particular piece of software was constructed can answer questions like "why is this here? why was it done this way? why is that not here? why wasn't such and such done this apparently better way?" For example, looking at some particular piece of Erlang code, an outsider might entertain three hypotheses: (a) the author was ignorant and didn't know about funs or list comprehensions (b) the author knew about them but didn't like them (c) the author knew and liked them but thought they were too slow. It's the author who knows that the true reason is (d) the code was written before those features were added to the language. It's not that a >>Wiki<< cannot contain the right answer once it is known, it's that Wiki >>contributors<< other than the author or author's close associates cannot in general know what the right answer *is*. And if the original author or associate is reachable enough to put the answer in a Wiki, they are reachable enough to put the answer in a book. Last night my departmental laptop developed some sort of software fault. It boots happily into Windows but not the operating system that came with it. Only that system has the password for the broadband modem. Any Erlang documentation I read tonight will have to be good old paper. > There's no reason a wiki can't answer "why" > questions. Indeed, the very first wiki in existence is "meta" on this > point, asking why wikis work at all (much to the bemusement of Ward > Cunningham, the surprised inventor) > This is pretty much the opposite case. What we have on that page is a bunch of post hoc rationalisations about how a thing happens not to be abused, not the original inventor's theory about why it was *expected* to work. (If he had such a theory he should not have been surprised when it did!) > http://c2.com/cgi/wiki?WhyWikiWorks > > And many wikis do explain the reasons for things. For example, > Wikipedia has elaborate, polished internal articles explaining the > "why" of its numerous policies and guidelines. Which are not contributed by outsiders. > > > It's impressive, but it doesn't solve the problem: as you yourself > admit, Erlang/OTP has no budget for one author, much less three of > them. The sad thing is that the budget for a technical writer isn't _that_ high. > Richard asks, "What's to be gained by lowering the bar?" Clearly he > envisions some hoi polloi stinking up the joint. There is point in lowering the bar IF (a) there are people with superior knowledge who are currently prohibited from entering, OR (b) there are people with superior editing skills who are currently prohibited from contributing corrections. Who _are_ these people? The main barrier I see is that I cannot find anywhere a document that says "How to contribute improvements to the Erlang documentation." There _is_ a github page about "Submitting patches", which is really quite astonishingly open compared with the way things used to be, but very strongly suggests that it is about code, not documenttion. I'm pretty sure I recall seeing messages on this list saying that improvements to the documentation were welcome. >> > > Nice, but with (for example) MediaWiki's footnote extension, Please, *not* MediaWiki! A Masters student here studied MediaWiki as his topic. It's grotesquely complex and (surprise surprise, NOT) badly documented and the compatibility of the various implementations is, um, unimpressive. (Rather like Markdown, as a matter of fact. There's original Markdown, there's the variant GitHub uses, and there's the variant Erlang uses. For a documentation tool, the documentation of Markdown doesn't even reach the "barely competent" level.) > A voice is perhaps essential for an authored work. But reference > manuals don't really need much "voice" -- they are encyclopedic. Nor > do tutorials -- voice can even distract in that case. They just need > correct information, couched in correct prose of a relatively uniform > and generic style, within a reasonably well-organized structure. All of which sounds utterly unlike any Wiki I've ever seen, with the possible exception of Wikipedia. The prospect of having to view heavily fragmented documentation full of StupidWikiWords is nightmarish. From ok@REDACTED Thu Jun 7 06:24:07 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 7 Jun 2012 16:24:07 +1200 Subject: [erlang-questions] Java error making docs in R15B01 Message-ID: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> I downloaded R15B01. ./configure --prefix=$HOME/local reported that fop was missing, so I downloaded Apache FOP (fop-1.0-bin.tar), set that up (but apparently not quite right), went back to otp_src_R15B01, and did ./configure --prefix=$HOME/local make I was a little surprised that the make step apparently did not build the documentation. So I then did make docs All went well up to this step: date=`date +"%B %e %Y"`; \ xsltproc \ --stringparam docgen "/home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen" \ --stringparam gendate "$date" \ --stringparam appname "stdlib" \ --stringparam appver "1.18.1" \ --xinclude --stringparam specs_file "`pwd`/specs.xml" \ -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd \ -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd_html_entities \ /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/xsl/db_pdf.xsl book.xml > ../pdf/stdlib-1.18.1.fo fop -fo ../pdf/stdlib-1.18.1.fo -pdf ../pdf/stdlib-1.18.1.pdf Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/fop/cli/Main Caused by: java.lang.ClassNotFoundException: org.apache.fop.cli.Main at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) make[3]: *** [../pdf/stdlib-1.18.1.pdf] Error 1 make[2]: *** [docs] Error 2 make[1]: *** [docs] Error 2 make: *** [docs] Error 2 By the way, are the DTDs used in building Erlang/OTP documentation *documented* anywhere? Y'know, generating LaTeX looks better and better... From michael.eugene.turner@REDACTED Thu Jun 7 07:27:06 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 14:27:06 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On Thu, Jun 7, 2012 at 10:53 AM, Richard O'Keefe wrote: > > On 6/06/2012, at 7:26 PM, Michael Turner wrote: > >>> What we really need is a small number of good technical writers >>> and money to pay them. >> >> Agreed. But guess what? Page after page of Erlang/OTP documentation >> can be produced as evidence that the team doesn't have the budget for >> it. > > Non sequitur. ?It demonstrates that the money _has_ not been spent > on good technical writers, not that the money _could_ not have been > so spent. I'm imagining a reasonable assumption: that glaring typos and grammatical errors, at the very least, really do seem like the kind of thing they'd fix if they could get the money for it. >> >> Given *reality*, I propose a wiki. > > But that's way too close to the process that gave us the documentation > we have. ?Every time I read any other outfit's Prolog documentation, I > recall the Quintus Prolog documentation with gladness. ?Quintus had > one policy which led to good documentation, and it is precisely that > policy you are proposing to throw overboard wearing concrete shoes: Richard, I'm not aiming for anything near perfection, but rather "not embarrassing." It's embarrassing for errors to sit around for years while new ones accumulate. [To be clear: it's not me writing this next line:] > ? ? ? ?Never let an engineer write the documentation. *Never*? Not even one who's proven his/her worth for years as a technical writer? Such people do exist, you know. > Our first technical writer, Kennita Watson, was technically competent. > (From Stanford, so how could she not be?) ?But she wasn't on the > development team. ?On more than one occasion her "how am I expected > to document _that_?" resulted in changes to designs we "engineers" > were perfectly happy with but that customers would not have been. There's no excuse for not knowing your intended audience. But for a wiki, the editing is largely done *by* the intended audience, so there's at least less chance of disconnect, even if there's more room for (technical) error. > Only *good* technical writers are needed, not *great* ones. > And one would be a good start. What the existing docs need right now is editing. There are vast swathes of text calling out for minor fixes. Why let those problems sit around for yet more years? Why set the bar for improving them so high? If you're talking about how to generate *new* documentation, I agree. In principle. But it's not going to happen. My beef is primarily with what's already out there, and secondarily with how it's packaged online - searching is still a pain, the docs are still seriously underlinked. -michael turner From matti.oinas@REDACTED Thu Jun 7 08:07:04 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Thu, 7 Jun 2012 09:07:04 +0300 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FCF452B.4010300@eonblast.com> References: <4FCF452B.4010300@eonblast.com> Message-ID: On Wed, Jun 6, 2012 at 2:55 PM, Henning Diedrich wrote: > > As noted before, Erlang docs seem harder to Google than other docs or > programming related information. > > ... > > My suspicion is that it has to do with the frame layout. But it should be possible to compensate for that and better guide the search engines through it. > Many times the bad search results problem comes from the fact that search engine fails to navigate through the site correctly. I don't know if erlang.org site uses sitemaps but if not then adding sitemap would be probably the easiest way to help google and other search engines to understand the content and that way improve search results. Documentation of sitemaps can be found from http://www.sitemaps.org/ From bengt.kleberg@REDACTED Thu Jun 7 08:07:45 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 7 Jun 2012 08:07:45 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <1339049265.5968.8.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> My first job allowed me to work with 2 technical writers. They had spent 3 years learning how to do technical writing. They where much better at this than the people that had spent 4 years learning engineering. If this was solely because of the education, or a result of wanting to do the job so much that they took the time to learn it, I do not know. bengt On Wed, 2012-06-06 at 00:53 +0200, Richard O'Keefe wrote: > On 6/06/2012, at 3:30 AM, Michael Turner wrote: > > But really: just give me a wiki. Give everybody a wiki. Write scripts > > to update the repo from the wiki diffs, if that's so important. > > For the sake of suffering humanity, forfend! > > The next time someone offers me a wiki instead of documentation, > I am going to be exceedingly angry. I have never yet found any > wikified documentation that was worth printing to wipe my arse with. > > I do apologise for the vulgarity, but good documentation needs > good technical writing and technical writing is a skill that is > distinct from programming, although some people can do both well. > It needs a *global* view as well as attention to fine detail. > > > Generally, lower the bar for entry, for those who want to improve the > > documentation, and the documentation *will* improve. > > Someone recently wrote something about the sky turning green then > yellow, didn't they? How is *lowering the bar* going to result in > anything *improving*? It might get more *voluminous*, but better? > > What we really need is a small number of good technical writers > and money to pay them. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Thu Jun 7 08:46:29 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 15:46:29 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <049344B4-4434-459E-ABDF-4C3D73C87194@cs.otago.ac.nz> References: <049344B4-4434-459E-ABDF-4C3D73C87194@cs.otago.ac.nz> Message-ID: On Thu, Jun 7, 2012 at 12:56 PM, Richard O'Keefe wrote: > I think you are misreading Joe here. > What I think he means is that only someone who was around > and involved when a particular piece of software was > constructed can answer questions like "why is this here? > why was it done this way? ?why is that not here? ?why > wasn't such and such done this apparently better way?" OK, so (to bring up the example that's freshest in my mind) we have depend on, say, Ulf Wiger for our explanation of why seq_trace implements Lamport clocks, because he "was there" (as he insists) Really? Even when it seems that he -- (a) was unaware that it implements Lamport clocks, (b) was apparently ignorant of Lamport clocks before I posted about the problem that seq_trace doesn't mention (by name) that it's implementing them (c) is doubtful that they are useful in seq_trace anyway, or thinks maybe something better could be done, he's not really sure, and (d) says his NDA means he can't really fully explain what seq_trace was originally supposed to do. I read Lamport's paper when it first came out, decades ago, and I have worked with seq_trace for a couple of months now, on and off. It seems that I'm not only in a better position to tell people that seq_trace implements Lamport clocks, but also WHY it does, even though I had nothing whatsoever to do with writing seq_trace. Perhaps Kenneth Lundin would beg to differ, but he's nowhere to be seen in the current discussion. If it apparently *should* be up to me (for lack of a better candidate stepping forward), then why not make it *easier* for the change to be left to me? If that's a sad state of affairs, it's nevertheless a reality (in this case, at least.) Your ironclad rule -- "only those present at the creation" - would only leave readers more ignorant (in this case, at least.) [snip] > It's the author who knows that the true reason is [snip] Wow. That defies what I've learned from experience, from when I started to write code at age 15, to today: still writing code at age 56. If you show an old piece of code to its author and ask "why?", about half the time he just can't remember. "Did I write that part of the system?" is a pretty common answer. Told, "yes, I've verified that you're the author," the usual answer is, "well, we were writing all that stuff in a big hurry. Sorry, but it's been a very long time ...." Official documentation should go to certain depths and its coverage should have a certain breadth, but an explanation like "The API is like this because it we were writing all this stuff really fast," is something I can't remember seeing. If I ever wrote some such phrase in some official documentation, it was probably because, well, I was in a hurry! Wikipedia has a rule: "There are no emergencies on Wikipedia." (Occasionally there are, but they usually involve edits that could get the Foundation into legal trouble.) Which is to say, given that all human social action is constrained by "cheap, good, soon - pick any two", Wikipedia gets some pretty good work done for very cheap, by not having any deadlines at all. For articles that nobody is interested in, that can mean zero progress. But it also means that the good stuff in a wiki is often the product of careful effort made possible by leisurely contemplation and discussion of a choice of renderings. The Talk pages on Wikipedia also provide a repository for what would otherwise be left to fallible memory. > It's not that a >>Wiki<< cannot contain the right answer > once it is known, it's that Wiki >>contributors<< other > than the author or author's close associates cannot in > general know what the right answer *is*. "Cannot in general" might be about as useful here as "You cannot, in general, divide two numbers", because, after all, 0/0 is meaningless. I'm trying to be pragmatic: a strategy to improve what's already there, with a technical-writer/editor budget of zero dollars. > And if the original author or associate is reachable > enough to put the answer in a Wiki, they are reachable > enough to put the answer in a book. Yeah? Ulf was "reachable" on seq_trace, but he only turned out to be a colossal waste of time. Kenneth Lundin was reachable, but apparently couldn't be bothered to comment. I've put questions to the list about seq_trace more than once. Neither was answered. I could be fixing typos in articles right now. Adding links right now. If I don't know the reason for something, I can always try asking. If I can't get a response, I can always use the tried-and-true Usenet style of getting an answer: write something that might be wrong, and tell people I did it. And if those who were "there at the creation" aren't available, that doesn't mean that there aren't others who, from more recent experience, might actually know better than the creators. >> There's no reason a wiki can't answer "why" >> questions. Indeed, the very first wiki in existence is "meta" on this >> point, asking why wikis work at all (much to the bemusement of Ward >> Cunningham, the surprised inventor) >> > This is pretty much the opposite case. ?What we have on that > page is a bunch of post hoc rationalisations about how a thing happens > not to be abused, not the original inventor's theory about why it > was *expected* to work. ?(If he had such a theory he should not have > been surprised when it did!) You mistake my purpose in pointing it out. Yes, it's amateur sociologizing, but it was only presented as an existence proof that people have been answering "why" questions on wikis since their inception. >> And many wikis do explain the reasons for things. For example, >> Wikipedia has elaborate, polished internal articles explaining the >> "why" of its numerous policies and guidelines. > > Which are not contributed by outsiders. Incorrect. (Also irrelevant: they nevertheless provide reasons.) I'm just an editor at Wikipedia (which anybody can be, even anonymously), not an admin or a bureaucrat. I've made a number of contributions to Wikipedia policies, including direct edits made without the usual open discussion because they were pretty obvious improvements in expressing the policy and its purpose, and thus uncontested by people who watch those pages like hawks. It's an open process. You might try learning something about it. > The sad thing is that the budget for a technical writer isn't > _that_ high. If they can't afford it, they can't afford it. It's sad that they can't budget for it. But that's the reality. I'm just trying to suggest what might produce progress on existing docs even within that constraint. Unfortunately, it seems that you and Joe are prone to sweeping generalizations about wikis, like "nobody ever fixes things" (you've got two volunteers already, on this very thread), and "outsiders don't make policy." (I don't even know what you mean by "outsider" and I should, if there was any such thing, because I've been editing Wikipedia articles for about 6 years now.) >> Richard asks, "What's to be gained by lowering the bar?" Clearly he >> envisions some hoi polloi stinking up the joint. > > There is point in lowering the bar IF > (a) there are people with superior knowledge who are currently > ? ?prohibited from entering, OR > (b) there are people with superior editing skills who are currently > ? ?prohibited from contributing corrections. > > Who _are_ these people? If you count the current bar as "prohibitive", then, in the case of seq_trace, ME, for one. I now know a lot about how to use it, and I know more about the theoretical underpinnings, it seems, than someone (Ulf) who was there at the creation. Neurons are not a reliable storage device. "I was there" is really not the gold standard here, after more than a decade has passed. It's the solid, verifiable stuff -- code, documentation, open literature publications -- that should determine content when memory becomes unreliable. And it usually does. > The main barrier I see is that I cannot find anywhere a document > that says "How to contribute improvements to the Erlang documentation." You say "barrier", I say "bar". Is there really any difference? Yes, a serious omission. Which I would say further raises the bar. > There _is_ a github page about "Submitting patches", which is really > quite astonishingly open compared with the way things used to be, > but very strongly suggests that it is about code, not documenttion. They keep saying that they welcome documentation *patches,* so I assumed it was the same process as for code. > I'm pretty sure I recall seeing messages on this list saying that > improvements to the documentation were welcome. Well, now you know how you're supposed to do it. And I think that's a pretty high bar, for documentation. >> Nice, but with (for example) MediaWiki's footnote extension, > > Please, *not* MediaWiki! ?A Masters student here studied MediaWiki > as his topic. ?It's grotesquely complex and (surprise surprise, > NOT) badly documented and the compatibility of the various > implementations is, um, unimpressive. "Various implementations" of MediaWiki? There's only one implementation of MediaWiki, Richard. (There are tons of wiki *packages*, with varying markup styles, including some with markup very similar to MediaWiki's.) Which makes me wonder if you're really talking about the same thing I am. Assuming we are: MediaWiki is quite complex for the reason that Erlang/OTP is complex: it needs to solve a lot of problems that have cropped up over a fairly long period of time. MediaWiki, like many systems that "just grew" (e.g., OTP), is "grotesque" in places. I don't know why anyone would expect otherwise. > ... (Rather like Markdown, as > a matter of fact. ?There's original Markdown, there's the variant > GitHub uses, and there's the variant Erlang uses. ?For a > documentation tool, the documentation of Markdown doesn't even > reach the "barely competent" level.) I see. So you'd say that the documentation for all the markup that went into this MediaWiki-based page http://en.wikipedia.org/wiki/Angular_momentum will be nonexistent, or patchy where it does exist? To me, it looks like people who understood the physics behind the topic were somehow also able to learn the markup for mathematics, graphics, even animations. Or were able get easily help from those who do know those things, without much trouble. Erlang/OTP? "We always welcome documentation patches." Period. That's their "getting started". >> A voice is perhaps essential for an authored work. But reference >> manuals don't really need much "voice" -- they are encyclopedic. Nor >> do tutorials -- voice can even distract in that case. They just need >> correct information, couched in correct prose of a relatively uniform >> and generic style, within a reasonably well-organized structure. > > All of which sounds utterly unlike any Wiki I've ever seen, with the > possible exception of Wikipedia. So? If that's what you want, then in the policies and guidelines for an Erlang/OTP documentation wiki, just default to the style guide that Wikipedia uses, with exceptions only as noted. > The prospect of having to view heavily fragmented documentation I don't know what you mean by "heavily fragmented." MediaWiki articles can be as long and monolithic as people desire. The obvious way to translate the existing docs to a wiki is just a one-for-one correspondence of articles. > full of StupidWikiWords is nightmarish. That's not MediaWiki, Richard. If I want to wikilink "Richard O'Keefe" in an article about Prolog people, I just bracket it like this: "[[Richard O'Keefe]]". For example, the markup for the article about the computer scientist Richard O'Keefe looks like this: --- Dr '''Richard A. O'Keefe''' is a [[computer science|computer scientist]], currently working in Department of Computer Science at the [[University of Otago]] in [[Dunedin]], [[New Zealand]]. He concentrates on languages for [[logic programming]] and [[functional programming]] (including [[Prolog programming language|Prolog]], [[Haskell (programming language)|Haskell]], and [[Erlang programming language|Erlang]]). O'Keefe published well known and influential book on [[Prolog programming language|Prolog]] programming: ''The Craft of Prolog'' (ISBN 0-262-15039-5). --- The main impediments to reading here are in how wikilinking is made to articles whose titles don't have the exact phrasing found in the article linking to them. Those are relatively dense in this article because Wikipedia is a comprehensive encyclopedia, the potential namespace is all of human knowledge. In an Erlang-specific wiki, it's pretty unlikely that you'd need to qualify references to the language as distinct from the mathematician. If you still find that style "nightmarish", well, (1) life is full of tradeoffs, I've found this one quite liveable, and (2) WYSIWYG editors for MediaWiki wikis are making great strides these days, so help is on the way. -michael turner From Sergey_Zhemzhitsky@REDACTED Thu Jun 7 09:19:38 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Thu, 7 Jun 2012 07:19:38 +0000 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> Hi Richard, Thanks a lot for answer. It is rather helpful. >> Why was it hard to see in the documentation? As I'm kinda a newbie at erlang I understand your indignation. I supposed there should be some best practices and of cause I don't know them for now. The question was asked first of all because there are a lot of samples which use different approaches to parse erlang terms and I'd like to know the best one from the point of view of source code maintainability and simplicity. I'm also interested in not only mentioned approaches but in the alternative ones. Best Regards, Sergey -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Richard O'Keefe Sent: Thursday, June 07, 2012 6:12 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? Zhemzhitsky Sergey troika.ru> asked about the difference between the eterm interface and the ei interface. My understanding of this is that the eterm interface lets you manipulate Erlang terms, while the ei interface lets you manipulate the (serialised) binary representation of terms. eterm: walk around/generate unpacked trees without ever going through binaries ei: sequentially generate/parse packed binaries without going through trees Does this answer the question? Why was it hard to see in the documentation? I thought the first couple of paragraphs of www.erlang.org/doc/man/{erl_eterm,ei}.html made it pretty clear. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From mjtruog@REDACTED Thu Jun 7 09:30:19 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 07 Jun 2012 00:30:19 -0700 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> Message-ID: <4FD0588B.5070000@gmail.com> On 06/07/2012 12:19 AM, Zhemzhitsky Sergey wrote: >>> Why was it hard to see in the documentation? > As I'm kinda a newbie at erlang I understand your indignation. I supposed there should be some best practices and of cause I don't know them for now. > > The question was asked first of all because there are a lot of samples which use different approaches to parse erlang terms and I'd like to know the best one from the point of view of source code maintainability and simplicity. > I'm also interested in not only mentioned approaches but in the alternative ones. It seems like the eterm interface is the older/slower interface, and the ei interface is the newer/faster interface (where you have better control over memory usage). That is just based on experience, and is unlikely to be found in the documentation. - Michael From erlang@REDACTED Thu Jun 7 09:35:52 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Jun 2012 09:35:52 +0200 Subject: [erlang-questions] Java error making docs in R15B01 In-Reply-To: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> References: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> Message-ID: On Thu, Jun 7, 2012 at 6:24 AM, Richard O'Keefe wrote: > I downloaded R15B01. > ./configure --prefix=$HOME/local > reported that fop was missing, > so I downloaded Apache FOP (fop-1.0-bin.tar), > set that up (but apparently not quite right), > went back to otp_src_R15B01, and did > > ./configure --prefix=$HOME/local > make > > I was a little surprised that the make step > apparently did not build the documentation. > So I then did > make docs > > All went well up to this step: > > date=`date +"%B %e %Y"`; \ > ? ? ? ?xsltproc \ > ? ? ? ? --stringparam docgen "/home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen" \ > ? ? ? ? --stringparam gendate "$date" \ > ? ? ? ? --stringparam appname "stdlib" \ > ? ? ? ? --stringparam appver "1.18.1" \ > ? ? ? ? --xinclude --stringparam specs_file "`pwd`/specs.xml" \ > ? ? ? ? -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd \ > ? ? ? ? -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd_html_entities \ > ? ? ? ? ? /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/xsl/db_pdf.xsl book.xml > ../pdf/stdlib-1.18.1.fo > fop -fo ../pdf/stdlib-1.18.1.fo -pdf ../pdf/stdlib-1.18.1.pdf > Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/fop/cli/Main > Caused by: java.lang.ClassNotFoundException: org.apache.fop.cli.Main > ? ? ? ?at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > ? ? ? ?at java.security.AccessController.doPrivileged(Native Method) > ? ? ? ?at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > ? ? ? ?at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > ? ? ? ?at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > ? ? ? ?at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > make[3]: *** [../pdf/stdlib-1.18.1.pdf] Error 1 > make[2]: *** [docs] Error 2 > make[1]: *** [docs] Error 2 > make: *** [docs] Error 2 > > > By the way, are the DTDs used in building Erlang/OTP documentation > *documented* anywhere? No /Joe > > > Y'know, generating LaTeX looks better and better... > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paperless@REDACTED Thu Jun 7 09:46:54 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 7 Jun 2012 19:46:54 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: On 7 June 2012 12:23, OvermindDL1 wrote: > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner > wrote: >>> and nobody >>> bothers to correct the mistakes. >> >> Not "nobody". For example, I'm one of those weird people who will >> spend 30 seconds to sign up for almost any wiki I see a typo in. > > For what it is worth, I am one as well. ?Some sort of user-editable > documentation (even something that I could comment on like the Haskell > one) would be nice. One very easy to implement solution would be to put the sources of the document into GitHub and use pull requests. This is especially handy these days, given that GitHub supports in-browser editing. Are the sources for the Erlang documentation written in plain text? From Sergey_Zhemzhitsky@REDACTED Thu Jun 7 10:04:39 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Thu, 7 Jun 2012 08:04:39 +0000 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <4FD0588B.5070000@gmail.com> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> <4FD0588B.5070000@gmail.com> Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CC216B@exch-mbx2.msk.trd.ru> Hello Michael, Thanks a lot. I have understood that in case of eterms it will be necessary to traverse the parsed structure somehow to transform it into the target c-structure, so ei interface seems more natural to use as I will be creating the c-structure during parsing. Regards, Sergey -----Original Message----- From: Michael Truog [mailto:mjtruog@REDACTED] Sent: Thursday, June 07, 2012 11:30 AM To: Zhemzhitsky Sergey Cc: 'Richard O'Keefe'; erlang-questions@REDACTED Subject: Re: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? On 06/07/2012 12:19 AM, Zhemzhitsky Sergey wrote: >>> Why was it hard to see in the documentation? > As I'm kinda a newbie at erlang I understand your indignation. I supposed there should be some best practices and of cause I don't know them for now. > > The question was asked first of all because there are a lot of samples which use different approaches to parse erlang terms and I'd like to know the best one from the point of view of source code maintainability and simplicity. > I'm also interested in not only mentioned approaches but in the alternative ones. It seems like the eterm interface is the older/slower interface, and the ei interface is the newer/faster interface (where you have better control over memory usage). That is just based on experience, and is unlikely to be found in the documentation. - Michael _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From gustav@REDACTED Thu Jun 7 10:06:06 2012 From: gustav@REDACTED (Gustav Simonsson) Date: Thu, 7 Jun 2012 10:06:06 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <4FD060EE.4020902@erlang.org> On 2012-06-06 12:36, Joe Armstrong wrote: > On Wed, Jun 6, 2012 at 9:26 AM, Michael Turner > wrote: >>> What we really need is a small number of good technical writers >>> and money to pay them. >> Agreed. But guess what? Page after page of Erlang/OTP documentation >> can be produced as evidence that the team doesn't have the budget for >> it. > Correct. The Erlang/OTP documentation has never had a specific budget. > The documentation has to be "good enough for internal use" - > Internally you can "always ask the guy who wrote the code" - it is a > source of never-ending amazement > to me that external users can figure out how do do things *without* asking the > authors. For external use the documentation must be better. Today the OTP team have dedicated resources for reviewing patches. This means that if you write a documentation patch it's almost guaranteed to be included following a review, provided it improves the documentation. The time to review a documentation change is also typically much shorter compared to reviews of code changes, and can often be done by more team members. The number of documentation patches from the open-source community has been growing, with a nice amount included in R15B and R15B01. Hopefully we'll see even more doc patches as the Erlang community grows. Regards, Gustav Simonsson Erlang/OTP team > > I agree with Richard on this one - a wiki cannot answer "why" questions - > inspecting the code might reveal *how* it works but not *why* it was > written this way. > I don't want to guess why code does what it does - I want the author to tell me. > > My person favorite model is "one author - many eyes". > > I'm really impressed by the real-world haskell site (see for example > http://book.realworldhaskell.org/read/functional-programming.html) > > This is a "three authors many eyes" publication. The authors are responsable > for the text - but many people can comment, and the barrier for entry > for commenting is very small. The comments can be replied to by other > people who read the text > and the discussion takes place exactly where it should (ie "under" the > paragraph). > This way, long discussions about clocks and how they work, can be hidden away > under the paragraphs in the text to which they refer. > > In later versions the authors can choose to revise the text in the > light of the comments or ignore them. > > When my prag book was published it first came out as PDF version, when > 70% was written - overnight I has comments on 500 pages of text - this > was * incredibly* helpful and really improved the quality of the text. > I got immediate feedback at a paragraph level - so was able to > reformulate the text *before* chopping down a load of trees. > > This is the best of worlds - the book still has a "voice" a point of > view that I can > agree with or disagree with - but many eyes can review the text and find typos > and point out things that they do not understand or which need further > clarification. > > I am currently trying to make a similar system for the Erlang documentation > that is inspired by the real-world haskell web-site - so hopefully > this will improve matters. > > Given that we are not going to suddenly employ a host of professional > technical writers to improve the documentation, I hope a user-driven > commenting system with > low barriers for entry will help improve matters. We will see, this is > an experiment after all. > > I also wish to target e-books - so I can read all the documentation on > a pad computer. > > >> For example, this very thread branches off from a discussion of >> tracing that includes a fight over what I consider an egregious >> omission from the seq_trace documentation. And here's one of the >> relevant pages about Erlang tracing, from 2007 >> >> http://web.archive.org/web/20071116203524/http://www.erlang.org//doc/apps/et/et_intro.html >> >> Grammatical error: "The following prerequisites is required for >> understanding ...." >> >> Here's that same page, now: >> >> http://www.erlang.org/doc/apps/et/et_intro.html >> >> Same damned error. And, just after the heading "1.1 Scope and >> Purpose", there's a new error: a paragraph consisting of a single >> apostrophe. >> >> Both versions contain a chilling circularity in the "prerequisites": >> "familiarity with the Erlang system and Erlang programming in general >> and the especially the art of Erlang tracing." >> >> "Especially the art of Erlang tracing"? But, um, wait: isn't "the art >> of Erlang tracing" precisely what I'm aiming to learn, in reading the >> "et" documentation for which this passage is the introduction? Or if >> not, well, where DO I pick that up? >> >> Given my choice of worlds, Richard, I'd choose what you prefer: >> technical writers on salary. And given a choice of technical writers, >> I'd choose the bodhisattvas of the field: people with masters degrees >> in computer science who have decided that working on clearer >> documentation will help lead, in its own small way, to the >> enlightenment of all sentient beings. >> >> Given *reality*, I propose a wiki. > The trouble with wiki's is a lot on the stuff on wikis is plain > *wrong* and nobody > bothers to correct the mistakes. We often see the "stupidity of > crowds" in action. > > Hidden in the OTP documentation is the stuff for many books - if > anybody feels like writing a book go ahead. Or write a chapter. > > If you write a chapter and send it to the guys who wrote the code > you'll find them > *very* helpfully and they will happily point out erros and > misunderstanding in your > text. > > We've got books about Erlang now we need books about OTP internals, debugging, > databases, making scalable systems, ... > > Cheers > > /Joe > >> -michael >> >> On Wed, Jun 6, 2012 at 7:53 AM, Richard O'Keefe wrote: >>> On 6/06/2012, at 3:30 AM, Michael Turner wrote: >>>> But really: just give me a wiki. Give everybody a wiki. Write scripts >>>> to update the repo from the wiki diffs, if that's so important. >>> For the sake of suffering humanity, forfend! >>> >>> The next time someone offers me a wiki instead of documentation, >>> I am going to be exceedingly angry. I have never yet found any >>> wikified documentation that was worth printing to wipe my arse with. >>> >>> I do apologise for the vulgarity, but good documentation needs >>> good technical writing and technical writing is a skill that is >>> distinct from programming, although some people can do both well. >>> It needs a *global* view as well as attention to fine detail. >>> >>>> Generally, lower the bar for entry, for those who want to improve the >>>> documentation, and the documentation *will* improve. >>> Someone recently wrote something about the sky turning green then >>> yellow, didn't they? How is *lowering the bar* going to result in >>> anything *improving*? It might get more *voluminous*, but better? >>> >>> What we really need is a small number of good technical writers >>> and money to pay them. >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Thu Jun 7 10:40:51 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 17:40:51 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD060EE.4020902@erlang.org> References: <4FD060EE.4020902@erlang.org> Message-ID: OK, how do I get started then? And how likely is it that, someday soon, every page on the website will have a link at the top saying, "See an error? Here's how to help." With those instructions on how to get started? Occasionally accepting documentation patches whenever offered doesn't seem to be resulting in cleanup of errors that have been hanging around in the documentation for years. -michael turner On Thu, Jun 7, 2012 at 5:06 PM, Gustav Simonsson wrote: > On 2012-06-06 12:36, Joe Armstrong wrote: >> >> On Wed, Jun 6, 2012 at 9:26 AM, Michael Turner >> ?wrote: >>>> >>>> What we really need is a small number of good technical writers >>>> and money to pay them. >>> >>> Agreed. But guess what? Page after page of Erlang/OTP documentation >>> can be produced as evidence that the team doesn't have the budget for >>> it. >> >> Correct. The Erlang/OTP documentation ?has never had a specific budget. >> The documentation has to be "good enough for internal use" - >> Internally you can "always ask the guy who wrote the code" - it is a >> source of never-ending amazement >> to me that external users can figure out how do do things *without* asking >> the >> authors. For external use the documentation must be better. > > > Today the OTP team have dedicated resources for reviewing patches. This > means that > if you write a documentation patch it's almost guaranteed to be included > following a review, > provided it improves the documentation. The time to review a documentation > change is also > typically much shorter compared to reviews of code changes, and can often be > done by > more team members. > > The number of documentation patches from the open-source community has been > growing, > with a nice amount included in R15B and R15B01. Hopefully we'll see even > more doc patches > as the Erlang community grows. > > Regards, > Gustav Simonsson > Erlang/OTP team > > >> >> I agree with Richard on this one - a wiki cannot answer "why" questions - >> inspecting the code might reveal *how* it works but not *why* it was >> written this way. >> I don't want to guess why code does what it does - I want the author to >> tell me. >> >> My person favorite model is "one author - many eyes". >> >> I'm really impressed by the real-world haskell site (see for example >> http://book.realworldhaskell.org/read/functional-programming.html) >> >> This is a "three authors many eyes" publication. The authors are >> responsable >> for the text - but many people can comment, and the barrier for entry >> for commenting is very small. The comments can be replied to by other >> people who read the text >> and the discussion takes place exactly where it should (ie "under" the >> paragraph). >> This way, long discussions about clocks and how they work, can be hidden >> away >> under the paragraphs in the text to which they refer. >> >> In later versions the authors can choose to revise the text in the >> light of the comments or ignore them. >> >> When my prag book was published it first came out as PDF version, when >> 70% was written - overnight I has comments on 500 pages of text - this >> was * incredibly* helpful and really improved the quality of the text. >> I got immediate feedback at a paragraph level - so was able to >> reformulate the text *before* chopping down a load of trees. >> >> This is the best of worlds - the book still has a "voice" a point of >> view that I can >> agree with or disagree with - but many eyes can review the text and find >> typos >> and point out things that they do not understand or which need further >> clarification. >> >> I am currently trying to make a similar system for the Erlang >> documentation >> that is inspired by the real-world haskell web-site - so hopefully >> this will improve matters. >> >> Given that we are not going to suddenly employ a host of professional >> technical writers to improve the documentation, I hope a user-driven >> commenting system with >> low barriers for entry will help improve matters. We will see, this is >> an experiment after all. >> >> I also wish to target e-books - so I can read all the documentation on >> a pad computer. >> >> >>> For example, this very thread branches off from a discussion of >>> tracing that includes a fight over what I consider an egregious >>> omission from the seq_trace documentation. And here's one of the >>> relevant pages about Erlang tracing, from 2007 >>> >>> >>> ?http://web.archive.org/web/20071116203524/http://www.erlang.org//doc/apps/et/et_intro.html >>> >>> Grammatical error: "The following prerequisites is required for >>> understanding ...." >>> >>> Here's that same page, now: >>> >>> ?http://www.erlang.org/doc/apps/et/et_intro.html >>> >>> Same damned error. And, just after the heading "1.1 ?Scope and >>> Purpose", there's a new error: a paragraph consisting of a single >>> apostrophe. >>> >>> Both versions contain a chilling circularity in the "prerequisites": >>> "familiarity with the Erlang system and Erlang programming in general >>> and the especially the art of Erlang tracing." >>> >>> "Especially the art of Erlang tracing"? But, um, wait: isn't "the art >>> of Erlang tracing" precisely what I'm aiming to learn, in reading the >>> "et" documentation for which this passage is the introduction? Or if >>> not, well, where DO I pick that up? >>> >>> Given my choice of worlds, Richard, I'd choose what you prefer: >>> technical writers on salary. And given a choice of technical writers, >>> I'd choose the bodhisattvas of the field: people with masters degrees >>> in computer science who have decided that working on clearer >>> documentation will help lead, in its own small way, to the >>> enlightenment of all sentient beings. >>> >>> Given *reality*, I propose a wiki. >> >> The trouble with wiki's is a lot on the stuff on wikis is plain >> *wrong* and nobody >> bothers to correct the mistakes. We often see the "stupidity of >> crowds" in action. >> >> Hidden in the OTP documentation is the stuff for many books - if >> anybody feels like writing a book go ahead. Or write a chapter. >> >> If you write a chapter and send it to the guys who wrote the code >> you'll find them >> *very* helpfully and they will happily point out erros and >> misunderstanding in your >> text. >> >> We've got books about Erlang now we need books about OTP internals, >> debugging, >> databases, making scalable systems, ... >> >> Cheers >> >> /Joe >> >>> -michael >>> >>> On Wed, Jun 6, 2012 at 7:53 AM, Richard O'Keefe >>> ?wrote: >>>> >>>> On 6/06/2012, at 3:30 AM, Michael Turner wrote: >>>>> >>>>> But really: just give me a wiki. Give everybody a wiki. Write scripts >>>>> to update the repo from the wiki diffs, if that's so important. >>>> >>>> For the sake of suffering humanity, forfend! >>>> >>>> The next time someone offers me a wiki instead of documentation, >>>> I am going to be exceedingly angry. ?I have never yet found any >>>> wikified documentation that was worth printing to wipe my arse with. >>>> >>>> I do apologise for the vulgarity, but good documentation needs >>>> good technical writing and technical writing is a skill that is >>>> distinct from programming, although some people can do both well. >>>> It needs a *global* view as well as attention to fine detail. >>>> >>>>> Generally, lower the bar for entry, for those who want to improve the >>>>> documentation, and the documentation *will* improve. >>>> >>>> Someone recently wrote something about the sky turning green then >>>> yellow, didn't they? ?How is *lowering the bar* going to result in >>>> anything *improving*? ?It might get more *voluminous*, but better? >>>> >>>> What we really need is a small number of good technical writers >>>> and money to pay them. >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Thu Jun 7 10:42:48 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 07 Jun 2012 10:42:48 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: <4FD06988.6090701@ninenines.eu> On 06/07/2012 10:40 AM, Michael Turner wrote: > And how likely is it that, someday soon, every page on the website > will have a link at the top saying, "See an error? Here's how to > help." With those instructions on how to get started? Why don't you send a patch that does exactly that? -- Lo?c Hoguin Erlang Cowboy Nine Nines From michael.eugene.turner@REDACTED Thu Jun 7 10:51:14 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 17:51:14 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD06988.6090701@ninenines.eu> References: <4FD060EE.4020902@erlang.org> <4FD06988.6090701@ninenines.eu> Message-ID: On Thu, Jun 7, 2012 at 5:42 PM, Lo?c Hoguin wrote: > On 06/07/2012 10:40 AM, Michael Turner wrote: >> >> And how likely is it that, someday soon, every page on the website >> will have a link at the top saying, "See an error? Here's how to >> help." With those instructions on how to get started? > > > Why don't you send a patch that does exactly that? BECAUSE THAT WOULD BE A DOCUMENTATION PATCH, AND I DON'T KNOW WHERE TO START IN SUBMITTING DOCUMENTATION PATCHES. If Erlang/OTP *truly* welcomes this kind of thing, why don't they quickly write it up and post it? What you suggest would need to be something that generates the header (and footers?) of every page, so it's actually programming of a kind, and for a documentation generator I don't know anything about. It wouldn't surprise me if figuring it out would require two days of concerted effort on my part, but only 20 minutes of concerted effort on the part of someone at Ericsson. Ericsson can't afford tech writers for Erlang/OTP? OK, fine, I can understand that. But ... Ericsson can't take 20 minutes to facilitate user input on the documentation, in order to (partially) *substitute* for a lack of tech writers? With the resulting solicitation going live the same day, not in some future Erlang/OTP release? That, I cannot understand. -michael turner From gustav@REDACTED Thu Jun 7 10:53:23 2012 From: gustav@REDACTED (Gustav Simonsson) Date: Thu, 7 Jun 2012 10:53:23 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> <4FD06988.6090701@ninenines.eu> Message-ID: <4FD06C03.8070903@erlang.org> On 2012-06-07 10:51, Michael Turner wrote: > On Thu, Jun 7, 2012 at 5:42 PM, Lo?c Hoguin wrote: >> On 06/07/2012 10:40 AM, Michael Turner wrote: >>> And how likely is it that, someday soon, every page on the website >>> will have a link at the top saying, "See an error? Here's how to >>> help." With those instructions on how to get started? >> >> Why don't you send a patch that does exactly that? > BECAUSE THAT WOULD BE A DOCUMENTATION PATCH, AND I DON'T KNOW WHERE TO > START IN SUBMITTING DOCUMENTATION PATCHES. > > If Erlang/OTP *truly* welcomes this kind of thing, why don't they > quickly write it up and post it? https://github.com/erlang/otp/wiki/Submitting-patches Regards, Gustav Simonsson Erlang/OTP team > > What you suggest would need to be something that generates the header > (and footers?) of every page, so it's actually programming of a kind, > and for a documentation generator I don't know anything about. It > wouldn't surprise me if figuring it out would require two days of > concerted effort on my part, but only 20 minutes of concerted effort > on the part of someone at Ericsson. > > Ericsson can't afford tech writers for Erlang/OTP? OK, fine, I can > understand that. > > But ... Ericsson can't take 20 minutes to facilitate user input on the > documentation, in order to (partially) *substitute* for a lack of tech > writers? With the resulting solicitation going live the same day, not > in some future Erlang/OTP release? > > That, I cannot understand. > > -michael turner From essen@REDACTED Thu Jun 7 11:01:27 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 07 Jun 2012 11:01:27 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> <4FD06988.6090701@ninenines.eu> Message-ID: <4FD06DE7.2040703@ninenines.eu> On 06/07/2012 10:51 AM, Michael Turner wrote: > What you suggest would need to be something that generates the header > (and footers?) of every page, so it's actually programming of a kind, > and for a documentation generator I don't know anything about. It > wouldn't surprise me if figuring it out would require two days of > concerted effort on my part, but only 20 minutes of concerted effort > on the part of someone at Ericsson. That's a lot of assumptions, you don't seem to even have looked yet. I'll add an assumption to that: It probably would have taken less time than to complain about it. We can do so many great things by assuming instead of actually doing anything. -- Lo?c Hoguin Erlang Cowboy Nine Nines From n.oxyde@REDACTED Thu Jun 7 11:05:07 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 7 Jun 2012 11:05:07 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: Le 7 juin 2012 ? 09:46, Tim McNamara a ?crit : > On 7 June 2012 12:23, OvermindDL1 wrote: >> On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner >> wrote: >>>> and nobody >>>> bothers to correct the mistakes. >>> >>> Not "nobody". For example, I'm one of those weird people who will >>> spend 30 seconds to sign up for almost any wiki I see a typo in. >> >> For what it is worth, I am one as well. Some sort of user-editable >> documentation (even something that I could comment on like the Haskell >> one) would be nice. > > One very easy to implement solution would be to put the sources of the > document into GitHub and use pull requests. This is especially handy > these days, given that GitHub supports in-browser editing. GitHub in browser editing is annoying and does not let you create new branches. > Are the sources for the Erlang documentation written in plain text? XML (with undocumented DTD) + edoc From xiaopong.tran@REDACTED Thu Jun 7 11:06:28 2012 From: xiaopong.tran@REDACTED (Xiaopong Tran) Date: Thu, 07 Jun 2012 17:06:28 +0800 Subject: [erlang-questions] Regarding the life cycle of ERL_NIF_TERM Message-ID: <4FD06F14.3000406@gmail.com> Hi, I have a question regarding the life cycle of ERL_NIF_TERM. According to the Erlang document http://www.erlang.org/doc/man/erl_nif.html: "All ERL_NIF_TERM's belong to an environment (ErlNifEnv). A term can not be destructed individually, it is valid until its environment is destructed." And the description regarding the ErlNifEnv: "A process bound environment is passed as the first argument to all NIFs. All function arguments passed to a NIF will belong to that environment." Could someone clarify what that means exactly, in terms of the ERL_NIF_TERM life cycle? So, in this case, the environment is process bound. Does this mean that, all ERL_NIF_TERM passed in a parameters to NIF function, and the ERL_NIF_TERM return value, will never get GC, as long as the process is running? Cheers, Xiaopong From prof3ta@REDACTED Thu Jun 7 11:13:37 2012 From: prof3ta@REDACTED (Roberto Aloi) Date: Thu, 7 Jun 2012 11:13:37 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <4FCF8ED7.6000201@eonblast.com> References: <4FC86EE6.3050509@gmail.com> <4FCF8ED7.6000201@eonblast.com> Message-ID: Interesting thread, indeed, but I'm wondering if we're not re-inventing the wheel here. Aren't all of the above features provided by Q&A websites such as Stack Overflow? You can register to the "erlang' tag via RSS feed. There's a RSS feed for "unanswered" questions, etc. Just a thought. R From ulf@REDACTED Thu Jun 7 11:15:35 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 7 Jun 2012 11:15:35 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> On 7 Jun 2012, at 11:05, Anthony Ramine wrote: >> Are the sources for the Erlang documentation written in plain text? > > XML (with undocumented DTD) + edoc The DTDs *are* documented: http://www.erlang.org/doc/apps/erl_docgen/users_guide.html BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Thu Jun 7 11:18:22 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 18:18:22 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD06DE7.2040703@ninenines.eu> References: <4FD060EE.4020902@erlang.org> <4FD06988.6090701@ninenines.eu> <4FD06DE7.2040703@ninenines.eu> Message-ID: On Thu, Jun 7, 2012 at 6:01 PM, Lo?c Hoguin wrote: > On 06/07/2012 10:51 AM, Michael Turner wrote: >> >> What you suggest would need to be something that generates the header >> (and footers?) of every page, so it's actually programming of a kind, >> and for a documentation generator I don't know anything about. It >> wouldn't surprise me if figuring it out would require two days of >> concerted effort on my part, but only 20 minutes of concerted effort >> on the part of someone at Ericsson. > > > That's a lot of assumptions, you don't seem to even have looked yet. I'll > add an assumption to that: > > It probably would have taken less time than to complain about it. Well then, YOU go do it in three minutes or however long YOU think it would take. Decades of experience in software tells me that if something seems like it should take 20 minutes but requires first learning how a system works, it'll almost always take days -- if you can do it at all. Is it such a strange assumption on my part that there's somebody in Erlang/OTP who *already* knows how their own documentation system works? In any case, they can do this a lot more easily than most people, and if they are sincere in their expressed desire for documentation patches, it would get more help from more people on the documentation. It's not the MediaWiki "edit" tab, but it would be something, at least, besides the bland reaction to complaints about the documentation that "we are happy to receive documentation patches." -michael turner From magnus.henoch@REDACTED Thu Jun 7 11:25:35 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Thu, 07 Jun 2012 10:25:35 +0100 (BST) Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: Message-ID: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> ----- Original Message ----- > On 7 June 2012 12:23, OvermindDL1 wrote: > > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner > > wrote: > >>> and nobody > >>> bothers to correct the mistakes. > >> > >> Not "nobody". For example, I'm one of those weird people who will > >> spend 30 seconds to sign up for almost any wiki I see a typo in. > > > > For what it is worth, I am one as well. ?Some sort of user-editable > > documentation (even something that I could comment on like the > > Haskell > > one) would be nice. > > One very easy to implement solution would be to put the sources of > the > document into GitHub and use pull requests. This is especially handy > these days, given that GitHub supports in-browser editing. So I decided to try this out. I went to https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , clicked "Edit this file", removed the stray apostrophe on line 43, typed in a commit message, and clicked "Propose file change". I got a very fancy 500 error page that changes perspective when I move the mouse; no sign of anything being committed to my otp fork on Github. Is there a workaround for that? Regards, -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From michael.eugene.turner@REDACTED Thu Jun 7 11:30:10 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 18:30:10 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> References: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> Message-ID: It's great that DTD docs are available, but I think what's needed is something more like this: Imagine someone who'd just like to get rid of the mysterious single-apostrophe paragraph after the second subject heading in the introduction to et. That's all. It's embarrassing. Or maybe the even-more-embarrassing plural-agreement error not long after? What precise steps would this person take to (0) get the necessary batch of files (1) locate the right file (2) edit it (3) verify that the page is still generated correctly (4) submit a patch ? This kind of thing (typos, grammatical errors, text that really should be linked but is not) is all over the documentation. I'm only too happy to fix these, if I can get submitting patches down to a habit. I'm only too happy to write it up, so that others can participate. -michael turner On Thu, Jun 7, 2012 at 6:15 PM, Ulf Wiger wrote: > > On 7 Jun 2012, at 11:05, Anthony Ramine wrote: > >>> Are the sources for the Erlang documentation written in plain text? >> >> XML (with undocumented DTD) + edoc > > The DTDs *are* documented: > > http://www.erlang.org/doc/apps/erl_docgen/users_guide.html > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hd2010@REDACTED Thu Jun 7 11:31:38 2012 From: hd2010@REDACTED (H. Diedrich) Date: Thu, 07 Jun 2012 11:31:38 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> Message-ID: <4FD074FA.7010403@eonblast.com> An HTML attachment was scrubbed... URL: From hd2010@REDACTED Thu Jun 7 11:33:22 2012 From: hd2010@REDACTED (H. Diedrich) Date: Thu, 07 Jun 2012 11:33:22 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> Message-ID: <4FD07562.9000007@eonblast.com> An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jun 7 11:38:20 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 07 Jun 2012 11:38:20 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> Message-ID: <4FD0768C.2030009@ninenines.eu> On 06/07/2012 11:30 AM, Michael Turner wrote: > (0) get the necessary batch of files https://github.com/erlang/otp/wiki/Submitting-patches > (1) locate the right file lib//doc/src for user guides, lib//src for reference edocs Or good old: git grep "string you're looking for" Or any of the thousand other ways of finding what you're looking for in source code. > (2) edit it Edit it. > (3) verify that the page is still generated correctly https://github.com/erlang/otp/wiki/Documentation > (4) submit a patch https://github.com/erlang/otp/wiki/Submitting-patches -- Lo?c Hoguin Erlang Cowboy Nine Nines From michael.eugene.turner@REDACTED Thu Jun 7 11:38:57 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 18:38:57 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> Message-ID: > typed in a commit message, and clicked "Propose file change". ?I got > a very fancy 500 error page that changes perspective when I move the > mouse; no sign of anything being committed to my otp fork on Github. I tried the same, except for the plural-agreement issue on the same page, under "Prerequisites." "Repository temporarily unavailable". No estimated-time-until-update. Well, OK. -michael turner On Thu, Jun 7, 2012 at 6:25 PM, Magnus Henoch wrote: > ----- Original Message ----- >> On 7 June 2012 12:23, OvermindDL1 wrote: >> > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner >> > wrote: >> >>> and nobody >> >>> bothers to correct the mistakes. >> >> >> >> Not "nobody". For example, I'm one of those weird people who will >> >> spend 30 seconds to sign up for almost any wiki I see a typo in. >> > >> > For what it is worth, I am one as well. ?Some sort of user-editable >> > documentation (even something that I could comment on like the >> > Haskell >> > one) would be nice. >> >> One very easy to implement solution would be to put the sources of >> the >> document into GitHub and use pull requests. This is especially handy >> these days, given that GitHub supports in-browser editing. > > So I decided to try this out. ?I went to > https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , > clicked "Edit this file", removed the stray apostrophe on line 43, > typed in a commit message, and clicked "Propose file change". ?I got > a very fancy 500 error page that changes perspective when I move the > mouse; no sign of anything being committed to my otp fork on Github. > > Is there a workaround for that? > > Regards, > -- > Magnus Henoch > Erlang Solutions Ltd > http://www.erlang-solutions.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anishparameswaran@REDACTED Thu Jun 7 11:43:50 2012 From: anishparameswaran@REDACTED (Anish Parameswaran) Date: Thu, 7 Jun 2012 15:13:50 +0530 Subject: [erlang-questions] corba character code set not supported with TAO ORB Message-ID: Hi, I am ( Erlang newbie ) using erlang orber for corba access to a server running TAO ORB . But i am getting the char codeset exception as shown below , =ERROR REPORT==== 7-Jun-2012::14:42:01 === =================== Orber ================= [565] iop_ior:check_char_codeset([83951617]); Orber cannot communicate with this ORB. It doesn't support a Char CodeSet known to Orber. =========================================== ** exception throw: {'EXCEPTION',{'INV_OBJREF',[],1163001856,'COMPLETED_NO'}} in function corba:raise/1 in call from iop_ior:check_components/3 in call from iop_ior:get_key_1/5 in call from corba:call_helper/6 in call from nbiclient:name_resolv/0 The erlang version is 'Erlang R13B03 (erts-5.7.4)' Is there any provision to set the charcode set to be used / or is this error something else. Any help is highly appreciated .. Thanks Anish -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.henoch@REDACTED Thu Jun 7 11:47:05 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Thu, 07 Jun 2012 10:47:05 +0100 (BST) Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD074FA.7010403@eonblast.com> Message-ID: Yes, I was. The error page doesn't reflect that, and shows a "Login" link, but clicking that just takes me back to the dashboard. Magnus ----- Original Message ----- > > Where you logged in to your github account while doing it? > > Henning > > Magnus Henoch schrieb: > > ----- Original Message ----- > > On 7 June 2012 12:23, OvermindDL1 wrote: > > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner > wrote: > > > > and nobody > bothers to correct the mistakes. Not "nobody". For example, I'm one > of those weird people who will > spend 30 seconds to sign up for almost any wiki I see a typo in. For > what it is worth, I am one as well. ?Some sort of user-editable > documentation (even something that I could comment on like the > Haskell > one) would be nice. One very easy to implement solution would be to > put the sources of > the > document into GitHub and use pull requests. This is especially handy > these days, given that GitHub supports in-browser editing. So I > decided to try this out. I went to > https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml > , > clicked "Edit this file", removed the stray apostrophe on line 43, > typed in a commit message, and clicked "Propose file change". I got > a very fancy 500 error page that changes perspective when I move the > mouse; no sign of anything being committed to my otp fork on Github. > > Is there a workaround for that? > > Regards, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From michael.eugene.turner@REDACTED Thu Jun 7 12:00:39 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 19:00:39 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD0768C.2030009@ninenines.eu> References: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> <4FD0768C.2030009@ninenines.eu> Message-ID: Thanks, Lo?c. Now I'll start to explicate, showing that there an awful lot of assumptions about the target audience in your brief summary, while also getting the thing partly written in the process. On Thu, Jun 7, 2012 at 6:38 PM, Lo?c Hoguin wrote: > On 06/07/2012 11:30 AM, Michael Turner wrote: >> >> (0) get the necessary batch of files > > > https://github.com/erlang/otp/wiki/Submitting-patches What this needs to be clear in order to almost anyone hitting that "How you can help" link: The documentation for Erlang/OTP is stored on github, a source code repository, with the Erlang/OTP source. Indeed, some of the documentation is basically *in* the source code -- special tools are used to generate much of the reference manual pages from the code. We'll show you how to find the relevant file, below. If you don't have a github account already, you'll need one. You'll also need to learn some basic git commands. These are covered in this tutorial. >> (1) locate the right file > > > lib//doc/src for user guides, > lib//src for reference edocs How do you get from the URL of the documentation you want to fix to a file name, so that you can edit that file? In general, if it's what Erlang/OTP calls an "application" (not to be confused with the more common meaning), you can find it lib//doc/src if it's a user guide, and lib//src if it's in a reference manual. > Or good old: git grep "string you're looking for" [Note: it's not "good" or "old" for those who don't know git. I'd expand this a bit more.] > Or any of the thousand other ways of finding what you're looking for in > source code. [Yes, but if you can just tell people the top three easiest ways, you get about 99% coverage for your audience.] >> (2) edit it > > > Edit it. [Yes, but where? It looks like there's a way to do this right at github, but it also looks like it's either buggy or potentially confusing. You need to tell them how to get what they want to edit, for editing, on their own machine, if that's the preferred option. Yes, believe it: lots of people have never used github.] >> (3) verify that the page is still generated correctly > > > https://github.com/erlang/otp/wiki/Documentation [Very likely, this step precedes the others. You also want to help people out if they aren't interested in figuring out how to get FOP or other dependencies for PDF generation.] >> (4) submit a patch > > > https://github.com/erlang/otp/wiki/Submitting-patches I'd refer to this, but since the tutorial would be presumably be working through a concrete example of a document fix, I'd substitute the concrete filenames and operations wherever that document (and other supporting github documentation) keeps things abstract. As Joe says in his book, you've got to type the stuff in or you're not really learning it. So give them what they need to type in, for a concrete example. -michael turner From michael.eugene.turner@REDACTED Thu Jun 7 12:04:22 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 19:04:22 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD074FA.7010403@eonblast.com> Message-ID: On Thu, Jun 7, 2012 at 6:47 PM, Magnus Henoch wrote: > Yes, I was. ?The error page doesn't reflect that, and shows a "Login" > link, but clicking that just takes me back to the dashboard. I was also logged in. More fun: the changes I made in the editor seem to have been lost, when I return to the editing page. It would be nice if it actually worked, snag-free, or at least warned you about when it wouldn't work. -michael turner > ----- Original Message ----- >> >> Where you logged in to your github account while doing it? >> >> Henning >> >> Magnus Henoch schrieb: >> >> ----- Original Message ----- >> >> On 7 June 2012 12:23, OvermindDL1 wrote: >> >> On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner >> wrote: >> >> >> >> and nobody >> bothers to correct the mistakes. Not "nobody". For example, I'm one >> of those weird people who will >> spend 30 seconds to sign up for almost any wiki I see a typo in. For >> what it is worth, I am one as well. ?Some sort of user-editable >> documentation (even something that I could comment on like the >> Haskell >> one) would be nice. One very easy to implement solution would be to >> put the sources of >> the >> document into GitHub and use pull requests. This is especially handy >> these days, given that GitHub supports in-browser editing. So I >> decided to try this out. ?I went to >> https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml >> , >> clicked "Edit this file", removed the stray apostrophe on line 43, >> typed in a commit message, and clicked "Propose file change". ?I got >> a very fancy 500 error page that changes perspective when I move the >> mouse; no sign of anything being committed to my otp fork on Github. >> >> Is there a workaround for that? >> >> Regards, >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- > Magnus Henoch > Erlang Solutions Ltd > http://www.erlang-solutions.com/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hd2010@REDACTED Thu Jun 7 12:06:22 2012 From: hd2010@REDACTED (H. Diedrich) Date: Thu, 07 Jun 2012 12:06:22 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> Message-ID: <4FD07D1E.3060309@eonblast.com> An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Jun 7 12:13:37 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 7 Jun 2012 12:13:37 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD060EE.4020902@erlang.org> References: <4FD060EE.4020902@erlang.org> Message-ID: On 7 Jun 2012, at 10:06, Gustav Simonsson wrote: > Today the OTP team have dedicated resources for reviewing patches. This means that > if you write a documentation patch it's almost guaranteed to be included following a review, > provided it improves the documentation. The time to review a documentation change is also > typically much shorter compared to reviews of code changes, and can often be done by > more team members. As Richard pointed out, the wiki page on submitting patches could be extended with some helpful tips on how to make documentation patches. At the very least, the github wiki should have a link to the OTP installation guide, http://www.erlang.org/doc/installation_guide/INSTALL.html which among many other things covers what to do when you run into problems with FOP. For those just wanting to patch the docs of a single app, it would of course be better to learn how to avoid FOP entirely, and perhaps even a sneaky way not to have to build *all* the docs. Here's a practical result: Now, the installation manual above says that `make` takes about 5 minutes on a fast machine. I'm not sure what kind of machine that is, but as an alternative data point, it takes 21 minutes on my fairly new MacBook Air with a 1.7 GHz Core i5, 4GB RAM and an SSD. It saddens me to find out how 'unfast' it is. Running `make doc` with fakefop on the same machine takes 7 minutes. (I had FOP installed before, but uninstalled it since it drove me nuts). Once all this is done, stepping down into e.g. sasl and running `make docs` there takes less than a second. Unfortunately, this is not enough to verify the docs entirely, since I don't get the frame layout. (The wiki says to use `make release_docs`, which is good advice. The INSTALL guide starts out with `make doc`, and then enters into a discussion about targets, which doesn't necessarily help me understand which one will make the html docs render right in my browser). However, running `make release_docs` afterwards (45 secs, since docs are already made), produces the full HTML docs under $ERL_TOP/release/doc/index.html Once at this point, making a change, running `make release_docs` at the app level, and refreshing the web page, is very quick. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From n.oxyde@REDACTED Thu Jun 7 12:18:05 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 7 Jun 2012 12:18:05 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD07D1E.3060309@eonblast.com> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> <4FD07D1E.3060309@eonblast.com> Message-ID: <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> Editing directly from GitHub makes you unable to check that you wrote lines too long (under 80 columns). Also it produces nondescript branch names like "patch-1". Furthermore, Erlang/OTP explicitely forbid pull requests for contribution. For all these points, I would say it's not neat at all. -- Anthony Ramine Le 7 juin 2012 ? 12:06, H. Diedrich a ?crit : > I think github-online editing worked, quite nicely even. > > The process seems tedious but what you get for free is quite nice. > > https://github.com/erlang/otp/pull/24 > > You click "edit this file", it creates a github fork for you. > When you change and click "save" the changes are saved to your fork. > Then you are automatically put on the screen where you send a pull request. > As for example the link above. > > Pretty neat in my view. Absolutely usable so far. A fully documented and repository-based, click-and-point patching process. > > Henning > > Magnus Henoch schrieb: >> >> ----- Original Message ----- >> >>> On 7 June 2012 12:23, OvermindDL1 wrote: >>> >>>> On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner >>>> wrote: >>>> >>>>>> and nobody >>>>>> bothers to correct the mistakes. >>>>>> >>>>> Not "nobody". For example, I'm one of those weird people who will >>>>> spend 30 seconds to sign up for almost any wiki I see a typo in. >>>>> >>>> For what it is worth, I am one as well. Some sort of user-editable >>>> documentation (even something that I could comment on like the >>>> Haskell >>>> one) would be nice. >>>> >>> One very easy to implement solution would be to put the sources of >>> the >>> document into GitHub and use pull requests. This is especially handy >>> these days, given that GitHub supports in-browser editing. >>> >> >> So I decided to try this out. I went to >> https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , >> clicked "Edit this file", removed the stray apostrophe on line 43, >> typed in a commit message, and clicked "Propose file change". I got >> a very fancy 500 error page that changes perspective when I move the >> mouse; no sign of anything being committed to my otp fork on Github. >> >> Is there a workaround for that? >> >> Regards, >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick@REDACTED Thu Jun 7 12:19:39 2012 From: nick@REDACTED (Niclas Eklund) Date: Thu, 07 Jun 2012 12:19:39 +0200 Subject: [erlang-questions] corba character code set not supported with TAO ORB In-Reply-To: References: Message-ID: <4FD0803B.1090004@tail-f.com> Hi! This exception is thrown when the char/string used by the server isn't known/supported by Orber (the same might happen for wchar/wstring) or the server cannot convert those supported by Orber. Orber support ISO8859-1 (Id == 16#00010001) and ISO646-IRV (Id == 16#00010020) and TAO sure must support at least ISO8859-1, otherwise it would not be able to communicate with some of the frequently used ORBs (Orbix, VisiBroker etc). If my memory serves me right Orber and TAO has been used together before, so you should check the TAO configuration. You can also use this function corba:print_object/2 to get more "human readable" information out of the TAO IOR. Best regards, Nick On 06/07/2012 11:43 AM, Anish Parameswaran wrote: > > Hi, > > I am ( Erlang newbie ) using erlang orber for corba access > to a server running TAO ORB . > But i am getting the char codeset exception as shown below , > > > > =ERROR REPORT==== 7-Jun-2012::14:42:01 === > =================== Orber ================= > [565] iop_ior:check_char_codeset([83951617]); > Orber cannot communicate with this ORB. > It doesn't support a Char CodeSet known to Orber. > =========================================== > ** exception throw: > {'EXCEPTION',{'INV_OBJREF',[],1163001856,'COMPLETED_NO'}} > in function corba:raise/1 > in call from iop_ior:check_components/3 > in call from iop_ior:get_key_1/5 > in call from corba:call_helper/6 > in call from nbiclient:name_resolv/0 > > > > The erlang version is 'Erlang R13B03 (erts-5.7.4)' > > Is there any provision to set the charcode set to be used / or is > this error something else. > Any help is highly appreciated .. > > Thanks > Anish > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu Jun 7 12:35:19 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 7 Jun 2012 12:35:19 +0200 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL Message-ID: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> Hi everyone, As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. Using SSL with this kinda feels like using a bazooka for smashing a fly. The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ Thanks for your help. From hd2010@REDACTED Thu Jun 7 12:40:42 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 07 Jun 2012 12:40:42 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> <4FD07D1E.3060309@eonblast.com> <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> Message-ID: <4FD0852A.8050105@eonblast.com> Sorry for hitting a merry tone. I realize this was inappropriate ;-) If it's wrong for OTP I think for other purposes it's still pretty neat. From the flames in this thread, maybe OTP would care to reconsider. It's easy to look at the actual patch, the diff, at github and heal the history later. Henning On 6/7/12 12:18 PM, Anthony Ramine wrote: > Editing directly from GitHub makes you unable to check that you wrote > lines too long (under 80 columns). > Also it produces nondescript branch names like "patch-1". > > Furthermore, Erlang/OTP explicitely forbid pull requests for contribution. > > For all these points, I would say it's not neat at all. > > -- > Anthony Ramine > > Le 7 juin 2012 ? 12:06, H. Diedrich a ?crit : > >> I think github-online editing worked, quite nicely even. >> >> The process seems tedious but what you get for free is quite nice. >> >> https://github.com/erlang/otp/pull/24 >> >> You click "edit this file", it creates a github fork for you. >> When you change and click "save" the changes are saved to your fork. >> Then you are automatically put on the screen where you send a pull >> request. >> As for example the link above. >> >> Pretty neat in my view. Absolutely usable so far. A fully documented >> and repository-based, click-and-point patching process. >> >> Henning >> >> Magnus Henoch schrieb: >>> ----- Original Message ----- >>> >>>> On 7 June 2012 12:23, OvermindDL1 wrote: >>>> >>>>> On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner >>>>> wrote: >>>>> >>>>>>> and nobody >>>>>>> bothers to correct the mistakes. >>>>>>> >>>>>> Not "nobody". For example, I'm one of those weird people who will >>>>>> spend 30 seconds to sign up for almost any wiki I see a typo in. >>>>>> >>>>> For what it is worth, I am one as well. Some sort of user-editable >>>>> documentation (even something that I could comment on like the >>>>> Haskell >>>>> one) would be nice. >>>>> >>>> One very easy to implement solution would be to put the sources of >>>> the >>>> document into GitHub and use pull requests. This is especially handy >>>> these days, given that GitHub supports in-browser editing. >>>> >>> >>> So I decided to try this out. I went to >>> https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , >>> clicked "Edit this file", removed the stray apostrophe on line 43, >>> typed in a commit message, and clicked "Propose file change". I got >>> a very fancy 500 error page that changes perspective when I move the >>> mouse; no sign of anything being committed to my otp fork on Github. >>> >>> Is there a workaround for that? >>> >>> Regards, >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From larshesel@REDACTED Thu Jun 7 13:01:31 2012 From: larshesel@REDACTED (Lars Hesel Christensen) Date: Thu, 7 Jun 2012 13:01:31 +0200 Subject: [erlang-questions] [ANN] Zurich Erlang User Group In-Reply-To: References: Message-ID: Thank you for the heads up! Cheers, Lars 2012/6/6 Ahmed Omar > Hi Lars, > If you need also presence on LinkedIn, I created a subgroup under > Erlang/OTP Professional Groups for members in Switzerland > http://www.linkedin.com/groups?gid=4478020 > > There are already subgroups for many EUGs (Sweden, UK, Spain, Netherlands, > Germany, USA, ...etc) > > Best of luck! > > Ahmed Omar > > On Wed, Jun 6, 2012 at 10:49 PM, Lars Hesel Christensen < > larshesel@REDACTED> wrote: > >> Hi all >> >> I just wanted to let you all know (and google and other search machines) >> that there now is an Erlang user group in Zurich / Switzerland. >> >> So if you live in, or are visiting Switzerland, you might want to check >> it out. For now we live on meetup.com: >> http://www.meetup.com/Zurich-Erlang-User-Group/, google groups: >> https://groups.google.com/forum/?fromgroups#!forum/zurich-erlang-user-group and >> twitter: https://twitter.com/#!/ZurichErlangUG >> >> Thank you, >> Lars >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Jun 7 13:05:00 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 7 Jun 2012 13:05:00 +0200 Subject: [erlang-questions] multiple behaviours In-Reply-To: References: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> Message-ID: <54169AC2-7AEC-47CE-9EC0-F79D541B7C22@feuerlabs.com> On 6 Jun 2012, at 15:33, Tony Rogvall wrote: >> Or, as you also suggest, write a new behaviour combining those two. If >> this situation is not common in your design I would suggest this >> solution as prevents you from going against what I think is the >> intended use of behaviours. I guess the original idea was to have at >> most a behaviour per module, but more authoritative voices may have a >> word about this. >> > Any takers on this topic (meaning people involved in the design of course) This has been debated. Since my part-time job is to fill the list with old gossip about the time before discussions moved to the web, this is the story of behaviors (at least as I recall it). For my current opinion, skip to the last paragraph. The first behavior-style components were actually built in BOS, a middleware layer designed for, and partly by, the first real commercial erlang-based product: Mobility Server. BOS was the predecessor of OTP. When designing OTP, the concept of behaviors was evangelized by Joe Armstrong as a more generalized version of the support libraries in BOS. Back in those days, it wasn't easy at all to define your own behavior; the linter had hard-coded knowledge of the ones in OTP, and would not help with any 'fake' behaviors. After having read a nice thesis (can't recall where to find it) about behaviors, which also described how to hack your own behaviors - painful, but rewarding if you could prevail. I wrote a prototype that introduced the behaviour_info/1 callback. It was accepted by OTP, with typos and all (I had used the American spelling, and it took a while before someone pointed this out; by this time, the easiest remedy was to accept that the erlang linter had become bi-lingual). There were discussions now and then about whether it was ok to mix behaviors in a single module. The OTP stance was that it was frowned upon, but their case was weakened by the fact that this is done in a number of places in OTP (e.g. kernel.erl). Nowadays, the linter accepts multiple behaviors, as long as they don't lay claim to the same callbacks (checking Fun/Arity only). With the new -callback() types, it should be possible to accept overlapping callbacks, as long as their type signatures are compatible (preferrably the same). Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From zerthurd@REDACTED Thu Jun 7 13:06:23 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 7 Jun 2012 18:06:23 +0700 Subject: [erlang-questions] Automatic nodes discovering Message-ID: Hello How I can make automatically discovering new nodes in cluster? When new node started up it is not listed in output of nodes() on other nodes without explicit ping of them. There is way to do this using RabbitMQ or ZeroMQ, but it is some redundant. Yet another way is ping of one known node in cluster, but it is constrained by nodes startup order. May be some interaction with epmd can helps? Thank you -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From siddharth.karandikar@REDACTED Thu Jun 7 13:07:50 2012 From: siddharth.karandikar@REDACTED (Siddharth Karandikar) Date: Thu, 7 Jun 2012 16:37:50 +0530 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: <4FCF835A.4080004@eonblast.com> References: <4FC86EE6.3050509@gmail.com> <4FCF835A.4080004@eonblast.com> Message-ID: This is cool! So kind of you to offer a domain name. notanswered.org and www.notanswered.org now go to correct page. Thanks, Siddharth On Wed, Jun 6, 2012 at 9:50 PM, Henning Diedrich wrote: > On 6/6/12 5:40 PM, CGS wrote: > > more "energy drinks". > > I like it. I'll donate www.notanswered.org > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From paperless@REDACTED Thu Jun 7 13:17:12 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 7 Jun 2012 23:17:12 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> <4FD07D1E.3060309@eonblast.com> <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> Message-ID: The reason that it is "not neat" is your focus is on side issues. The real issue is how to make the contributing as simple as possible, thus leading to better documentation. Expecting people to download the entire source distribution to fix minor grammar mistakes is far less neat. Concerns about column widths are really not that significant. "patch-1" is good enough for a branch name, merely because it basically implies that it is a minor change that came through the GitHub web UI. No one else would use such a convention. Branch names are far less significant than commit messages, which are strongly promoted within GitHub's interface. Pull request policies are just policies. If the OTP team would prefer a patch file, they simply change the file extension of the URL and they have a patch file. On 7 June 2012 22:18, Anthony Ramine wrote: > Editing directly from GitHub makes you unable to check that you wrote lines > too long (under 80 columns). > Also it produces nondescript branch names like "patch-1". > > Furthermore, Erlang/OTP explicitely forbid pull requests for contribution. > > For all these points, I would say it's not neat at all. > > -- > Anthony Ramine > > Le 7 juin 2012 ? 12:06, H. Diedrich a ?crit : > > I think github-online editing worked, quite nicely even. > > The process seems tedious but what you get for free is quite nice. > > https://github.com/erlang/otp/pull/24 > > You? click "edit this file", it creates a github fork for you. > When you change and click "save" the changes are saved to your fork. > Then you are automatically put on the screen where you send a pull request. > As for example the link above. > > Pretty neat in my view. Absolutely usable so far. A fully documented and > repository-based, click-and-point patching process. > > Henning > > Magnus Henoch schrieb: > > ----- Original Message ----- > > > On 7 June 2012 12:23, OvermindDL1 wrote: > > > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner > wrote: > > > and nobody > bothers to correct the mistakes. > > > Not "nobody". For example, I'm one of those weird people who will > spend 30 seconds to sign up for almost any wiki I see a typo in. > > > For what it is worth, I am one as well. ?Some sort of user-editable > documentation (even something that I could comment on like the > Haskell > one) would be nice. > > > One very easy to implement solution would be to put the sources of > the > document into GitHub and use pull requests. This is especially handy > these days, given that GitHub supports in-browser editing. > > > So I decided to try this out. I went to > https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , > clicked "Edit this file", removed the stray apostrophe on line 43, > typed in a commit message, and clicked "Propose file change". I got > a very fancy 500 error page that changes perspective when I move the > mouse; no sign of anything being committed to my otp fork on Github. > > Is there a workaround for that? > > Regards, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Thu Jun 7 13:20:30 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 7 Jun 2012 13:20:30 +0200 Subject: [erlang-questions] Automatic nodes discovering In-Reply-To: References: Message-ID: On 7 Jun 2012, at 13:06, Maxim Treskin wrote: > How I can make automatically discovering new nodes in cluster? When new node started up it is not listed in output of nodes() on other nodes without explicit ping of them. There is way to do this using RabbitMQ or ZeroMQ, but it is some redundant. Yet another way is ping of one known node in cluster, but it is constrained by nodes startup order. The old-school way is indeed to have the new node ping one of the known nodes. It used to be judged acceptable to have a couple of "master nodes", one of which had to be up in order for the system to be operational. I think this model is a bit too easily discarded these days? not everyone needs so many nodes that perfect symmetry is really called for from a handling perspective. Even looking at e.g. riak_core, a new node needs to be told which node to talk to in order to join the ring. You can also look into the net_adm module in kernel. It has some functions for auto-discovery. http://www.erlang.org/doc/man/net_adm.html BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Thu Jun 7 13:20:54 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 20:20:54 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: As Ulf's blow-by-blow demonstrates, getting from "I want to add a period to the end of a sentence that doesn't have one" in the Erlang/OTP docs to actually adding that period, starting from scratch, is a tortuous path, with diversions like FOP, the problem of installing Erlang/OTP, building the docs, etc. And that's not even the whole story: you're not actually making a change. You're submitting a patch, and people at Ericsson have to review your "append a period" operation. You might not see it online until the next release. I'm now going to time myself on adding a sentence to an embryonic wiki I started not long ago. Tick, tick, tick ... OK, about 35 seconds -- several of which were spent identifying the URL of the wiki and waiting for it to come up. And here's the change. http://beam.referata.com/w/index.php?title=Main_Page&diff=10&oldid=9 I'll time myself on reverting that change: http://beam.referata.com/w/index.php?title=Main_Page&diff=11&oldid=10 About 10 seconds. Some of which were spent waiting because referata.com can be a little slow. Wikis are, of course, utterly retarded, a completely stupid idea. I don't know why anybody ever thought they could work for any purpose whatsoever, because they could only work when an intelligent and devoted community of users contributes to them, and in the Erlang/OTP community, there's an incredible shortage of ... Um, wait, I think maybe there's something wrong with one of my premises. -michael turner On Thu, Jun 7, 2012 at 7:13 PM, Ulf Wiger wrote: > > On 7 Jun 2012, at 10:06, Gustav Simonsson wrote: > >> Today the OTP team have dedicated resources for reviewing patches. This means that >> if you write a documentation patch it's almost guaranteed to be included following a review, >> provided it improves the documentation. The time to review a documentation change is also >> typically much shorter compared to reviews of code changes, and can often be done by >> more team members. > > As Richard pointed out, the wiki page on submitting patches > could be extended with some helpful tips on how to make > documentation patches. > > At the very least, the github wiki should have a link to the OTP > installation guide, > > http://www.erlang.org/doc/installation_guide/INSTALL.html > > which among many other things covers what to do when you > run into problems with FOP. For those just wanting to patch > the docs of a single app, it would of course be better to learn > how to avoid FOP entirely, and perhaps even a sneaky way > not to have to build *all* the docs. > > Here's a practical result: > > Now, the installation manual above says that `make` takes > about 5 minutes on a fast machine. I'm not sure what kind of > machine that is, but as an alternative data point, it takes > 21 minutes on my fairly new MacBook Air with a 1.7 GHz > Core i5, 4GB RAM and an SSD. It saddens me to find out > how 'unfast' it is. > > Running `make doc` with fakefop on the same machine > takes 7 minutes. > > (I had FOP installed before, but uninstalled it since it > drove me nuts). > > Once all this is done, stepping down into e.g. sasl > and running `make docs` there takes less than a second. > > Unfortunately, this is not enough to verify the docs > entirely, since I don't get the frame layout. > > (The wiki says to use `make release_docs`, which > is good advice. The INSTALL guide starts out with > `make doc`, and then enters into a discussion about > targets, which doesn't necessarily help me understand > which one will make the html docs render right in my > browser). > > However, running `make release_docs` afterwards > (45 secs, since docs are already made), produces the > full HTML docs ?under $ERL_TOP/release/doc/index.html > > Once at this point, making a change, running > `make release_docs` at the app level, and refreshing > the web page, is very quick. > > BR, > Ulf W > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Thu Jun 7 13:22:54 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 7 Jun 2012 13:22:54 +0200 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> Message-ID: Hi! If you do not care about certificates you can use an anonymous chiper suite. You have to explicitly give such suites as arguments as they are not supported by default for security reasons. This of course will save you some memory as there are no own or peer certificate that will be saved. We are also curious to know what you think is an acceptable memory footprint and why? Regards Ingela Erlang/OTP team Ericsson AB 2012/6/7 Morgan Segalis : > Hi everyone, > > As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. > > I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. > Using SSL with this kinda feels like using a bazooka for smashing a fly. > > The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. > > Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) > > I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ > > Thanks for your help. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Thu Jun 7 13:27:42 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 20:27:42 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> <4FD07D1E.3060309@eonblast.com> <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> Message-ID: On Thu, Jun 7, 2012 at 7:18 PM, Anthony Ramine wrote: > Editing directly from GitHub makes you unable to check that you wrote lines > too long (under 80 columns). The Hollerith Limit seems to be honored mainly in the breach these days, though I still try to observe it. > Also it produces nondescript branch names like "patch-1". If you can get them listed with the edit summaries, maybe that's no so bad. > Furthermore, Erlang/OTP explicitely forbid pull requests for contribution. > > For all these points, I would say it's not neat at all. If they could make it work with fewer problems like these, it might be the simplest alternative to wiki-style editing, especially since github manages it for you. The major drawback I see in an Erlang/OTP documentation wiki is that there *is* significant management overhead, especially if it's hard to automate a effective access control. -michael turner > Le 7 juin 2012 ? 12:06, H. Diedrich a ?crit : > > I think github-online editing worked, quite nicely even. > > The process seems tedious but what you get for free is quite nice. > > https://github.com/erlang/otp/pull/24 > > You? click "edit this file", it creates a github fork for you. > When you change and click "save" the changes are saved to your fork. > Then you are automatically put on the screen where you send a pull request. > As for example the link above. > > Pretty neat in my view. Absolutely usable so far. A fully documented and > repository-based, click-and-point patching process. > > Henning > > Magnus Henoch schrieb: > > ----- Original Message ----- > > > On 7 June 2012 12:23, OvermindDL1 wrote: > > > On Wed, Jun 6, 2012 at 7:50 AM, Michael Turner > wrote: > > > and nobody > bothers to correct the mistakes. > > > Not "nobody". For example, I'm one of those weird people who will > spend 30 seconds to sign up for almost any wiki I see a typo in. > > > For what it is worth, I am one as well. ?Some sort of user-editable > documentation (even something that I could comment on like the > Haskell > one) would be nice. > > > One very easy to implement solution would be to put the sources of > the > document into GitHub and use pull requests. This is especially handy > these days, given that GitHub supports in-browser editing. > > > So I decided to try this out. I went to > https://github.com/erlang/otp/blob/maint/lib/et/doc/src/et_intro.xml , > clicked "Edit this file", removed the stray apostrophe on line 43, > typed in a commit message, and clicked "Propose file change". I got > a very fancy 500 error page that changes perspective when I move the > mouse; no sign of anything being committed to my otp fork on Github. > > Is there a workaround for that? > > Regards, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From n.oxyde@REDACTED Thu Jun 7 13:31:18 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 7 Jun 2012 13:31:18 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <336ca0d8-0c5b-447d-940a-6b9f1d4dcbe4@knuth> <4FD07D1E.3060309@eonblast.com> <7EC5BEC3-3B2D-445C-92F3-E3C21F416459@gmail.com> Message-ID: <1BA236F1-D172-40B0-AF76-88D88A6A1127@gmail.com> Le 7 juin 2012 ? 13:17, Tim McNamara a ?crit : > Branch names > are far less significant than commit messages, which are strongly > promoted within GitHub's interface. Again, there is no easy from the GitHub interface to know for sure that the commit messages are correctly formatted. Branch names may be insignificant but commit messages aren't. From zerthurd@REDACTED Thu Jun 7 13:40:18 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 7 Jun 2012 18:40:18 +0700 Subject: [erlang-questions] Automatic nodes discovering In-Reply-To: References: Message-ID: Thank you, Ulf net_adm:names() is almost exactly what i need On 7 June 2012 18:20, Ulf Wiger wrote: > > On 7 Jun 2012, at 13:06, Maxim Treskin wrote: > > How I can make automatically discovering new nodes in cluster? When new > node started up it is not listed in output of nodes() on other nodes > without explicit ping of them. There is way to do this using RabbitMQ or > ZeroMQ, but it is some redundant. Yet another way is ping of one known node > in cluster, but it is constrained by nodes startup order. > > > The old-school way is indeed to have the new node ping one > of the known nodes. It used to be judged acceptable to have > a couple of "master nodes", one of which had to be up in order > for the system to be operational. I think this model is a bit too > easily discarded these days? not everyone needs so many > nodes that perfect symmetry is really called for from a handling > perspective. > > Even looking at e.g. riak_core, a new node needs to be told > which node to talk to in order to join the ring. > > You can also look into the net_adm module in kernel. > It has some functions for auto-discovery. > > http://www.erlang.org/doc/man/net_adm.html > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Thu Jun 7 13:41:16 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 20:41:16 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: I'll admit that what I wrote below is not quite fair. It could take longer to make a small change to a wiki, starting from scratch. Depending on access control policies, I might even have to wait a day for a human being to process my request to become an editor for a wiki, if that's how the administrator qualifies people. I've suggested one heuristic for automated approval that's still a reasonable fine - and tunable - filter: require an e-mail address, match it against contributions to this mailing list, and only do an automatic grant if the time between the first contribution and the most recent one is above some threshold. Others are possible. Some wiki packages allow you to submit one or more changes along with an application for editorship. The changes will be approved and made to the article after you've been approved as an editor; all your changes after that point will be immediate. At any rate, any wiki and any wiki administrative policy would mean less time and less to learn, for people who want to make simple changes. Github itself hosts wikis about repositories, and I'm sure at least some of them are useful. I hope I'm not preaching to a choir here about the virtues of wikis, but from Joe's and Richard's responses, I suspect not. -michael turner On Thu, Jun 7, 2012 at 8:20 PM, Michael Turner wrote: > As Ulf's blow-by-blow demonstrates, getting from "I want to add a > period to the end of a sentence that doesn't have one" in the > Erlang/OTP docs to actually adding that period, starting from scratch, > is a tortuous path, with diversions like FOP, the problem of > installing Erlang/OTP, building the docs, etc. And that's not even the > whole story: you're not actually making a change. You're submitting a > patch, and people at Ericsson have to review your "append a period" > operation. You might not see it online until the next release. > > I'm now going to time myself on adding a sentence to an embryonic wiki > I started not long ago. > > Tick, tick, tick ... > > OK, about 35 seconds -- several of which were spent identifying the > URL of the wiki and waiting for it to come up. And here's the change. > > ?http://beam.referata.com/w/index.php?title=Main_Page&diff=10&oldid=9 > > I'll time myself on reverting that change: > > ?http://beam.referata.com/w/index.php?title=Main_Page&diff=11&oldid=10 > > About 10 seconds. Some of which were spent waiting because > referata.com can be a little slow. > > Wikis are, of course, utterly retarded, a completely stupid idea. I > don't know why anybody ever thought they could work for any purpose > whatsoever, because they could only work when an intelligent and > devoted community of users contributes to them, and in the Erlang/OTP > community, there's an incredible shortage of ... > > Um, wait, I think maybe there's something wrong with one of my premises. > > -michael turner > > On Thu, Jun 7, 2012 at 7:13 PM, Ulf Wiger wrote: >> >> On 7 Jun 2012, at 10:06, Gustav Simonsson wrote: >> >>> Today the OTP team have dedicated resources for reviewing patches. This means that >>> if you write a documentation patch it's almost guaranteed to be included following a review, >>> provided it improves the documentation. The time to review a documentation change is also >>> typically much shorter compared to reviews of code changes, and can often be done by >>> more team members. >> >> As Richard pointed out, the wiki page on submitting patches >> could be extended with some helpful tips on how to make >> documentation patches. >> >> At the very least, the github wiki should have a link to the OTP >> installation guide, >> >> http://www.erlang.org/doc/installation_guide/INSTALL.html >> >> which among many other things covers what to do when you >> run into problems with FOP. For those just wanting to patch >> the docs of a single app, it would of course be better to learn >> how to avoid FOP entirely, and perhaps even a sneaky way >> not to have to build *all* the docs. >> >> Here's a practical result: >> >> Now, the installation manual above says that `make` takes >> about 5 minutes on a fast machine. I'm not sure what kind of >> machine that is, but as an alternative data point, it takes >> 21 minutes on my fairly new MacBook Air with a 1.7 GHz >> Core i5, 4GB RAM and an SSD. It saddens me to find out >> how 'unfast' it is. >> >> Running `make doc` with fakefop on the same machine >> takes 7 minutes. >> >> (I had FOP installed before, but uninstalled it since it >> drove me nuts). >> >> Once all this is done, stepping down into e.g. sasl >> and running `make docs` there takes less than a second. >> >> Unfortunately, this is not enough to verify the docs >> entirely, since I don't get the frame layout. >> >> (The wiki says to use `make release_docs`, which >> is good advice. The INSTALL guide starts out with >> `make doc`, and then enters into a discussion about >> targets, which doesn't necessarily help me understand >> which one will make the html docs render right in my >> browser). >> >> However, running `make release_docs` afterwards >> (45 secs, since docs are already made), produces the >> full HTML docs ?under $ERL_TOP/release/doc/index.html >> >> Once at this point, making a change, running >> `make release_docs` at the app level, and refreshing >> the web page, is very quick. >> >> BR, >> Ulf W >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From samuelrivas@REDACTED Thu Jun 7 13:47:54 2012 From: samuelrivas@REDACTED (Samuel) Date: Thu, 7 Jun 2012 13:47:54 +0200 Subject: [erlang-questions] multiple behaviours In-Reply-To: References: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> Message-ID: > Wow, someone answered a question sent 7 days ago :-) Actually, it was motivated by Joe's mail about questions lingering unanswered in the list, I just scan back a little to find one that I could surface again :) Regards -- Samuel From erlang@REDACTED Thu Jun 7 13:54:25 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Jun 2012 13:54:25 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: On Thu, Jun 7, 2012 at 1:41 PM, Michael Turner wrote: > I'll admit that what I wrote below is not quite fair. It could take > longer to make a small change to a wiki, starting from scratch. > Depending on access control policies, I might even have to wait a day > for a human being to process my request to become an editor for a > wiki, if that's how the administrator qualifies people. > > I've suggested one heuristic for automated approval that's still a > reasonable fine - and tunable - filter: require an e-mail address, > match it against contributions to this mailing list, and only do an > automatic grant if the time between the first contribution and the > most recent one is above some threshold. Others are possible. > > Some wiki packages allow you to submit one or more changes along with > an application for editorship. The changes will be approved and made > to the article after you've been approved as an editor; all your > changes after that point will be immediate. > > At any rate, any wiki and any wiki administrative policy would mean > less time and less to learn, for people who want to make simple > changes. Github itself hosts wikis about repositories, and I'm sure at > least some of them are useful. I hope I'm not preaching to a choir > here about the virtues of wikis, but from Joe's and Richard's > responses, I suspect not. The problem isn't the wiki. It's the entire tool chain. A wiki is just part of this. We have 1335 XML files containing "the documentation" currently the DTDs are too complex. Many XML files don't validate and the tool chain is difficult to install and run externally. If we reformat the XML into a wiki we will have another problem - back propagating the wiki changes into the XML tree. /Joe > > -michael turner > > On Thu, Jun 7, 2012 at 8:20 PM, Michael Turner > wrote: >> As Ulf's blow-by-blow demonstrates, getting from "I want to add a >> period to the end of a sentence that doesn't have one" in the >> Erlang/OTP docs to actually adding that period, starting from scratch, >> is a tortuous path, with diversions like FOP, the problem of >> installing Erlang/OTP, building the docs, etc. And that's not even the >> whole story: you're not actually making a change. You're submitting a >> patch, and people at Ericsson have to review your "append a period" >> operation. You might not see it online until the next release. >> >> I'm now going to time myself on adding a sentence to an embryonic wiki >> I started not long ago. >> >> Tick, tick, tick ... >> >> OK, about 35 seconds -- several of which were spent identifying the >> URL of the wiki and waiting for it to come up. And here's the change. >> >> ?http://beam.referata.com/w/index.php?title=Main_Page&diff=10&oldid=9 >> >> I'll time myself on reverting that change: >> >> ?http://beam.referata.com/w/index.php?title=Main_Page&diff=11&oldid=10 >> >> About 10 seconds. Some of which were spent waiting because >> referata.com can be a little slow. >> >> Wikis are, of course, utterly retarded, a completely stupid idea. I >> don't know why anybody ever thought they could work for any purpose >> whatsoever, because they could only work when an intelligent and >> devoted community of users contributes to them, and in the Erlang/OTP >> community, there's an incredible shortage of ... >> >> Um, wait, I think maybe there's something wrong with one of my premises. >> >> -michael turner >> >> On Thu, Jun 7, 2012 at 7:13 PM, Ulf Wiger wrote: >>> >>> On 7 Jun 2012, at 10:06, Gustav Simonsson wrote: >>> >>>> Today the OTP team have dedicated resources for reviewing patches. This means that >>>> if you write a documentation patch it's almost guaranteed to be included following a review, >>>> provided it improves the documentation. The time to review a documentation change is also >>>> typically much shorter compared to reviews of code changes, and can often be done by >>>> more team members. >>> >>> As Richard pointed out, the wiki page on submitting patches >>> could be extended with some helpful tips on how to make >>> documentation patches. >>> >>> At the very least, the github wiki should have a link to the OTP >>> installation guide, >>> >>> http://www.erlang.org/doc/installation_guide/INSTALL.html >>> >>> which among many other things covers what to do when you >>> run into problems with FOP. For those just wanting to patch >>> the docs of a single app, it would of course be better to learn >>> how to avoid FOP entirely, and perhaps even a sneaky way >>> not to have to build *all* the docs. >>> >>> Here's a practical result: >>> >>> Now, the installation manual above says that `make` takes >>> about 5 minutes on a fast machine. I'm not sure what kind of >>> machine that is, but as an alternative data point, it takes >>> 21 minutes on my fairly new MacBook Air with a 1.7 GHz >>> Core i5, 4GB RAM and an SSD. It saddens me to find out >>> how 'unfast' it is. >>> >>> Running `make doc` with fakefop on the same machine >>> takes 7 minutes. >>> >>> (I had FOP installed before, but uninstalled it since it >>> drove me nuts). >>> >>> Once all this is done, stepping down into e.g. sasl >>> and running `make docs` there takes less than a second. >>> >>> Unfortunately, this is not enough to verify the docs >>> entirely, since I don't get the frame layout. >>> >>> (The wiki says to use `make release_docs`, which >>> is good advice. The INSTALL guide starts out with >>> `make doc`, and then enters into a discussion about >>> targets, which doesn't necessarily help me understand >>> which one will make the html docs render right in my >>> browser). >>> >>> However, running `make release_docs` afterwards >>> (45 secs, since docs are already made), produces the >>> full HTML docs ?under $ERL_TOP/release/doc/index.html >>> >>> Once at this point, making a change, running >>> `make release_docs` at the app level, and refreshing >>> the web page, is very quick. >>> >>> BR, >>> Ulf W >>> >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Thu Jun 7 13:59:48 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 7 Jun 2012 20:59:48 +0900 Subject: [erlang-questions] Java error making docs in R15B01 In-Reply-To: References: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> Message-ID: "By the way, are the DTDs used in building Erlang/OTP documentation *documented* anywhere?" Didn't Ulf just point them out on another thread? Here's what he turned up: http://www.erlang.org/doc/apps/erl_docgen/users_guide.html -michael turner On Thu, Jun 7, 2012 at 4:35 PM, Joe Armstrong wrote: > On Thu, Jun 7, 2012 at 6:24 AM, Richard O'Keefe wrote: >> I downloaded R15B01. >> ./configure --prefix=$HOME/local >> reported that fop was missing, >> so I downloaded Apache FOP (fop-1.0-bin.tar), >> set that up (but apparently not quite right), >> went back to otp_src_R15B01, and did >> >> ./configure --prefix=$HOME/local >> make >> >> I was a little surprised that the make step >> apparently did not build the documentation. >> So I then did >> make docs >> >> All went well up to this step: >> >> date=`date +"%B %e %Y"`; \ >> ? ? ? ?xsltproc \ >> ? ? ? ? --stringparam docgen "/home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen" \ >> ? ? ? ? --stringparam gendate "$date" \ >> ? ? ? ? --stringparam appname "stdlib" \ >> ? ? ? ? --stringparam appver "1.18.1" \ >> ? ? ? ? --xinclude --stringparam specs_file "`pwd`/specs.xml" \ >> ? ? ? ? -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd \ >> ? ? ? ? -path /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/dtd_html_entities \ >> ? ? ? ? ? /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/erl_docgen/priv/xsl/db_pdf.xsl book.xml > ../pdf/stdlib-1.18.1.fo >> fop -fo ../pdf/stdlib-1.18.1.fo -pdf ../pdf/stdlib-1.18.1.pdf >> Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/fop/cli/Main >> Caused by: java.lang.ClassNotFoundException: org.apache.fop.cli.Main >> ? ? ? ?at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> ? ? ? ?at java.security.AccessController.doPrivileged(Native Method) >> ? ? ? ?at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> ? ? ? ?at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> ? ? ? ?at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> ? ? ? ?at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> make[3]: *** [../pdf/stdlib-1.18.1.pdf] Error 1 >> make[2]: *** [docs] Error 2 >> make[1]: *** [docs] Error 2 >> make: *** [docs] Error 2 >> >> >> By the way, are the DTDs used in building Erlang/OTP documentation >> *documented* anywhere? > > No > > /Joe > > >> >> >> Y'know, generating LaTeX looks better and better... >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From msegalis@REDACTED Thu Jun 7 14:15:20 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 7 Jun 2012 14:15:20 +0200 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> Message-ID: <70DCD697-AFB3-4A54-8B7B-59A2934EA2BF@gmail.com> Hi Angela ! Actually, anonymous cyphers are, as described in the Erlang doc, not quite good for security? (vulnerable to man in the middle) I should explain more about the idea I had for good encryption without SSL (kinda, since we need it for the gateway) ? 1) The client should connect WITH SSL to a gateway (With Certificate and strong Cipher) 2) Client and Server will negotiate a key together along with a sessionID (not sure we need Diffie-Hellman since it is a secure connection) 3) The sessionID + key will be stored in a cassandra database (With a TimeToLive of 2min) 4) The server will tell to which node the client should connect 5) Client disconnects from gateway 6) Client connects to the node specified by the gateway, and gives the SessionID to the node (in clear) 7) The Server node retrieve the Key from the cassandra db 8) If it exist, it sends an accept to the client with the encryption, if not it disconnects the client and we are back to square 1. 9) Both client & server can communicate over AES using the key they now both have. Only the sessionID is sent without encryption, but without the key, the sessionID is useless. Once retrieved by the server node, the sessionID will be deleted from cassandra. This will give the gateway all the hassle to deal with SSL and it's memory footprint, but since the client won't stay connected to the gateway, it doesn't matter, it will be removed as soon as the client disconnects. > We are also curious to know what you think is an acceptable memory > footprint and why? Actually, I do not have any acceptable level per say? I would like to handle a maximum client on a single server. I'm actually trying to reach 1 millions connections (mostly idle) on a server with 16 GB of RAM with nothing else on it that the erlang node. Right now, memory speaking I can "only" handle 200 000 clients on the same server afterwards it start swapping (on a SSD so stay pretty fast, but still), using ssl_hibernate AND gen_server hibernate. Thanks! Morgan. Le 7 juin 2012 ? 13:22, Ingela Andin a ?crit : > Hi! > > If you do not care about certificates you can use an anonymous chiper > suite. You have to > explicitly give such suites as arguments as they are not supported by > default for security reasons. This of course will save you some > memory as there are no own or peer certificate that will be saved. We > are also curious to know what you think is an acceptable memory > footprint and why? > > Regards Ingela Erlang/OTP team Ericsson AB > > 2012/6/7 Morgan Segalis : >> Hi everyone, >> >> As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. >> >> I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. >> Using SSL with this kinda feels like using a bazooka for smashing a fly. >> >> The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. >> >> Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) >> >> I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ >> >> Thanks for your help. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From lambdadmitry@REDACTED Thu Jun 7 14:22:44 2012 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Thu, 7 Jun 2012 16:22:44 +0400 Subject: [erlang-questions] [ANN] tempo: strptime/strftime bindings Message-ID: Hello fellow erlangers, As we all know, there is a substantial amount of pain in the field of time parsing/formatting in Erlang -- strings instead binaries, different time formats and so on. So we've written a binding to libc's strptime/strftime that takes and returns binaries and almost all common formats of time. https://github.com/selectel/tempo (tempo_dev@REDACTED)1> {ok, Bin} = tempo:format(iso8601, {now, now()}). {ok,<<"2012-06-01T19:06:420000">>} (tempo_dev@REDACTED)2> tempo:parse(iso8601, {datetime, Bin}). {ok,{{2012,6,1},{19,6,42}}} Any feedback is appreciated, Dmitry From dmkolesnikov@REDACTED Thu Jun 7 14:29:15 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Thu, 7 Jun 2012 15:29:15 +0300 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> Message-ID: <-5124868496751168372@unknownmsgid> Hello, I guess you can use crypto:dh_compute_key for DH key exchange procedure. Your application cannot trust server w/o certificates. Therefore number of attacks are possible. If you are fine with all related threats then yes this is good option to save memory. On another hand, if you need to use multiple transport e.g. Tcp + udp then it is good as well. Btw, C ssl API allows to perform SSL:read/write directly to memory. It would be cool if such feature is available in Erlang. I suppose it might help to reduce memory footprint for some applications Best Regards, Dmitry >-|-|-*> On 7.6.2012, at 13.35, Morgan Segalis wrote: > Hi everyone, > > As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. > > I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. > Using SSL with this kinda feels like using a bazooka for smashing a fly. > > The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. > > Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) > > I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ > > Thanks for your help. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From cgsmcmlxxv@REDACTED Thu Jun 7 15:31:46 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 7 Jun 2012 15:31:46 +0200 Subject: [erlang-questions] Erlang and the TDS protocol In-Reply-To: References: <015601cd4406$cdf59a10$69e0ce30$@com> Message-ID: Hi Marcel, I suppose the reason for using C wrappers is coming from the compatibility with the native language in which that particular SQL application is developed. I do agree that a solution based on TCP server would be at least more readable and maybe more suitable for clustering SQL servers. Nevertheless, that's the choice of developers. Speaking for myself only, I would go with your option because I see no downfall to that and it would provide a less dependable interface. This is my 2c opinion. If someone has a better explanation/suggestion, please, by all means, correct me. CGS On Wed, Jun 6, 2012 at 8:22 PM, Marcel Meyer wrote: > Hi David, > No replies yet. I was wanted to see how other db wire protocols were > implemented, and saw that 'dizzyd' did the mysql driver natively. I spoke > to a colleague of mine and we wondered if there might be performance issues > with bit syntax and that the ODBC app wanted to go straight to the metal. > The reason I was asking this is that we're talking about how you would > build a proxy to a cluster of Sql Servers (not a Sql Server Cluster). > Jan Kneschke of lighttpd fame is working on a MySql proxy, with > failover/load balancing properties. Could be interesting for Sql Server... > > Regards, > M > > On Wed, Jun 6, 2012 at 1:07 PM, David Mercer wrote: > >> Just checking to see if anyone replied to you on this and did not >> reply-all the list. I would be interested in hearing the answer, too, if >> one is available.**** >> >> ** ** >> >> Cheers,**** >> >> ** ** >> >> DBM**** >> >> ** ** >> >> *From:* erlang-questions-bounces@REDACTED [mailto: >> erlang-questions-bounces@REDACTED] *On Behalf Of *Marcel Meyer >> *Sent:* Sunday, June 03, 2012 4:39 PM >> *To:* erlang-questions >> *Subject:* [erlang-questions] Erlang and the TDS protocol**** >> >> ** ** >> >> Dear list,**** >> >> ** ** >> >> I am curious why there isn't a native Erlang TDS protocol implementation. >> Even /otp/lib wraps a C ODBC implementation using a port.**** >> >> Would this not be simpler and more concise using bit syntax, since the >> TCP datagram unpacks in 1 line of code (more for readability, of course)? >> **** >> >> It just seems like extra work: C wraps the msg, port wraps C, ODBC app >> wraps port etc. There is a lot of 'wrapping' going on.**** >> >> ** ** >> >> Any input would be much appreciated.**** >> >> ** ** >> >> Kind regards,**** >> >> Marcel**** >> >> ** ** >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Jun 7 16:06:05 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 7 Jun 2012 16:06:05 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> Message-ID: <20120607140605.GA3444@erix.ericsson.se> On Wed, Jun 06, 2012 at 05:17:32PM +0530, Siddharth Karandikar wrote: > Hi all, > > I have put together a program for this. > > This program fetches email archives from erlang-questions and uses > information available in email headers to identify unanswered mails. > It then generates pages to browse through these mails, view and reply. > > This program is currently running on my server. Its currently > configured to keep track of mails within last 120 days. I am fetching > new content and updating the pages twice a day. > > Please have a look - http://178.79.140.33/index.html > Comments/suggestions are welcome! > > On the side note, Can anyone tell me how frequently does these mail > archives get updated? So that I will tune the schedule on my server > accordingly. I can not find anything about that in the Mailman documentation. It may be "instant". The internal Pipermail archiver is used and right now the date of the index file is the same as on the latest post. > > Thanks, > Siddharth > : -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jared@REDACTED Thu Jun 7 16:28:44 2012 From: jared@REDACTED (Jared Morrow) Date: Thu, 7 Jun 2012 08:28:44 -0600 Subject: [erlang-questions] R15B01 SmartOS Compile Error Message-ID: <6B996CD7-B550-4233-8E4C-16F31BFB2FD3@basho.com> Witeman, I also had that problem before installing GNU's version of m4. Try installing it, clearing out your OTP directory (make clean is not enough), reconfiguring and building. My `pkgin list` command shows: m4-1.4.16 GNU version of UNIX m4 macro language processor My m4 shows up as: $m4 --version m4 (GNU M4) 1.4.16 Hope that helps, Jared P.S. Thanks to Siddharth for his notanswered.org site -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Thu Jun 7 19:04:56 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 7 Jun 2012 11:04:56 -0600 Subject: [erlang-questions] [ANN] tempo: strptime/strftime bindings In-Reply-To: References: Message-ID: On Thu, Jun 7, 2012 at 6:22 AM, Dmitry Groshev wrote: > Hello fellow erlangers, > > As we all know, there is a substantial amount of pain in the field of > time parsing/formatting in Erlang -- strings instead binaries, > different time formats and so on. So we've written a binding to libc's > strptime/strftime that takes and returns binaries and almost all > common formats of time. > > https://github.com/selectel/tempo > > (tempo_dev@REDACTED)1> {ok, Bin} = tempo:format(iso8601, {now, now()}). > {ok,<<"2012-06-01T19:06:420000">>} > (tempo_dev@REDACTED)2> tempo:parse(iso8601, {datetime, Bin}). > {ok,{{2012,6,1},{19,6,42}}} > > Any feedback is appreciated, > Dmitry Looks to be useful, may use it in my next project that needs such formatting (which is quite common) to see how it compares. From janburse@REDACTED Thu Jun 7 22:26:54 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 07 Jun 2012 22:26:54 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <1339049265.5968.8.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> References: <1339049265.5968.8.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: <4FD10E8E.4010401@fastmail.fm> Bengt Kleberg schrieb: > My first job allowed me to work with 2 technical writers. They had spent > 3 years learning how to do technical writing. They where much better at > this than the people that had spent 4 years learning engineering. > > If this was solely because of the education, or a result of wanting to > do the job so much that they took the time to learn it, I do not know. Down with classical technical writing. Most valuable source for a programmer are the comments inside the code written by the original engineers. If these comments were extensively written, precicely showing preconditions, postconditions, motivations and pitfalls, then everything that comes after it is just shelfware. Bye From samuelrivas@REDACTED Thu Jun 7 22:44:00 2012 From: samuelrivas@REDACTED (Samuel) Date: Thu, 7 Jun 2012 22:44:00 +0200 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <4FD0588B.5070000@gmail.com> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> <4FD0588B.5070000@gmail.com> Message-ID: > It seems like the eterm interface is the older/slower interface, and the ei interface is the newer/faster interface (where you have better control over memory usage). ?That is just based on experience, and is unlikely to be found in the documentation. A better way to see it is that the ETERM abstraction used by erl_interface builds on top of ei. ei contains functions to decode and encode the erlang binary format, but how to represent the data in your program is up to you, ei will only provide pointers and integers. Also you have to manage the memory yourself (well, there is a little aid called ei_x_buff, but still quite low level). ETERM buils on that so that you get c terms representing lists, tuples and the rest of non-recursive types. That comes with the tradeoff of letting erl_interface manage the memory for you which may or may not work depending on your needs. It is good because it saves time and is supposed to prevent errors, but is bad because you loose flexibility and optimisation opportunities. Using ei directly gives you more freedom, but requires more coding usually. In my experience, erl_interface works well for fairly simple ports, but is a bit difficult to keep memory and performance under control as soon as you start using more complex data types. In any case, if you end up managing fairly complex data types, the serialisation/deserialisation code is going to be a pain, no matter which option you use. When I was working for Interoud we implemented a code generator that given type specifications generated code to serialise and deserialise erlang terms from/to C types in a fairly compact format. You can use the same idea using either library to support it if you don't want to use your own serialisation algorithm, but in this case I'm pretty sure that the best option would be ei. There are also more sophisticated ways of doing it as UBF [1,2] and other frameworks. Also, I remember reading at some point that erl_interface was deprecated, but I cannot find the reference now. The only thing that I see is that the code for erl_interface lives in a directory called legacy ... [1] http://www.sics.se/~joe/ubf/site/home.html [2] https://github.com/norton/ubf Best -- Samuel From watson.timothy@REDACTED Thu Jun 7 15:57:09 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 7 Jun 2012 14:57:09 +0100 Subject: [erlang-questions] Erlang and the TDS protocol In-Reply-To: References: <015601cd4406$cdf59a10$69e0ce30$@com> Message-ID: <4FBD9C92-1D25-4902-8EFD-7E08D74CF52E@gmail.com> On 7 Jun 2012, at 14:31, CGS wrote: > Hi Marcel, > > I suppose the reason for using C wrappers is coming from the compatibility with the native language in which that particular SQL application is developed. > > I do agree that a solution based on TCP server would be at least more readable and maybe more suitable for clustering SQL servers. Nevertheless, that's the choice of developers. Speaking for myself only, I would go with your option because I see no downfall to that and it would provide a less dependable interface. > I prefer a wire protocol implementation on the whole. The existing FreeTDS library, for example, doesn't support MARS which is pretty useful for connecting to MSSQL. If you decide to go native and open source, let me know as I'll be interested in participating, even if only it's to do some testing. > This is my 2c opinion. If someone has a better explanation/suggestion, please, by all means, correct me. > > CGS > > > > > On Wed, Jun 6, 2012 at 8:22 PM, Marcel Meyer wrote: > Hi David, > No replies yet. I was wanted to see how other db wire protocols were implemented, and saw that 'dizzyd' did the mysql driver natively. I spoke to a colleague of mine and we wondered if there might be performance issues with bit syntax and that the ODBC app wanted to go straight to the metal. > The reason I was asking this is that we're talking about how you would build a proxy to a cluster of Sql Servers (not a Sql Server Cluster). > Jan Kneschke of lighttpd fame is working on a MySql proxy, with failover/load balancing properties. Could be interesting for Sql Server... > > Regards, > M > > On Wed, Jun 6, 2012 at 1:07 PM, David Mercer wrote: > Just checking to see if anyone replied to you on this and did not reply-all the list. I would be interested in hearing the answer, too, if one is available. > > > > Cheers, > > > > DBM > > > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Marcel Meyer > Sent: Sunday, June 03, 2012 4:39 PM > To: erlang-questions > Subject: [erlang-questions] Erlang and the TDS protocol > > > > Dear list, > > > > I am curious why there isn't a native Erlang TDS protocol implementation. Even /otp/lib wraps a C ODBC implementation using a port. > > Would this not be simpler and more concise using bit syntax, since the TCP datagram unpacks in 1 line of code (more for readability, of course)? > > It just seems like extra work: C wraps the msg, port wraps C, ODBC app wraps port etc. There is a lot of 'wrapping' going on. > > > > Any input would be much appreciated. > > > > Kind regards, > > Marcel > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jun 8 00:08:08 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 10:08:08 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: Message-ID: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> On 7/06/2012, at 5:27 PM, Michael Turner wrote: >> Non sequitur. It demonstrates that the money _has_ not been spent >> on good technical writers, not that the money _could_ not have been >> so spent. > > I'm imagining a reasonable assumption: that glaring typos and > grammatical errors, at the very least, really do seem like the kind of > thing they'd fix if they could get the money for it. Recall that Erlang comes from Sweden. Things that are "glaring typos and grammatical errors" to a native speaker of English might very well be not so apparent to a native speaker of Swedish. As for people associated with Erlang who might be native speakers of English, from the messages of one of them I've gleaned "nieve", "ie", "asked be", "nastyness", number agreement errors, TAM errors, swapping "you" and "your", "over" for "other", "of" for "on", "lot's", "starred" for "stared", "the" for "they", punctuation errors in abundance, in every message there's _something_. I strongly suspect that many of the people associated with the development of Erlang don't normally *notice* such things. I have a colleague here who had part of his education in England at a time when the Initial Teaching Alphabet was in vogue. To this day he is not just bad at spelling but bad at knowing where he's bad at it. That's quite consistent with him being an excellent programmer (if only I could cope with his indentation style I would call him an exceptional programmer) and a very intelligent innovator in his field. > Richard, I'm not aiming for anything near perfection, but rather "not > embarrassing." It's embarrassing for errors to sit around for years > while new ones accumulate. It's embarrassing *if you notice*. > > [To be clear: it's not me writing this next line:] >> Never let an engineer write the documentation. > > *Never*? Not even one who's proven his/her worth for years as a > technical writer? Such people do exist, you know. >> Our first technical writer, Kennita Watson, was technically competent. Did you read that sentence? That means that I *know* that there are people with programming skills who have proven worth as technical writers because I worked with one. (And there were others later.) "Engineer" in this context means "someone who implemented/maintained the specific system being documented". These people know too much. They have forgotten what it was like *not* knowing. They are keen to explain things, but they explain the wrong aspects of those things, or by reference to other things that they forget people haven't heard of. The ideal is someone very like Kennita: someone with a general technical background who is *able* to learn the information customers need but does not know it *yet*, and writes up what she's learned right at the cusp where she now understands it but vividly remembers not understanding it. >> On more than one occasion her "how am I expected >> to document _that_?" resulted in changes to designs we "engineers" >> were perfectly happy with but that customers would not have been. > > There's no excuse for not knowing your intended audience. I am not sure what that sentence is directed at. Let me give you a specific example of what I was talking about. One aspect of Edinburgh Prolog which David Warren had thought well enough of from the beginning was its "totally defined semantics". Things that might have been exceptions in other languages simply said 'no solution' and triggered backtracking. This was the way Edinburgh Prolog had *always* been. It had been clearly and explicitly documented for a long time. The audience Quintus were writing for was "people who already know about Prolog and now want to use an efficient Prolog on machines other than DEC-10s, like VAXen, 68ks, /370s, Xerox D-machines, ..." We were happy enough that "functor(T, 37, 37)" failed quietly; that was the way it had always been, that was the way it had always been documented, that was the way it was explained in the tutorial material, that was the way people coming from DEC-10 Prolog or C-Prolog _expected_ it to be. It was Kennita who persuaded us that this wasn't good enough, because _she_ found it confusing while learning Prolog, and argued that reporting obvious errors as errors rather than appealing to "the totally defined semantics" would be better for customers and for our reputation. This ultimately led to my development of the error handling system and the error classification in Quintus Prolog which can still be traced in the ISO Prolog standard, although they changed the names of the operations. > But for a > wiki, the editing is largely done *by* the intended audience, so > there's at least less chance of disconnect, even if there's more room > for (technical) error. Most of the intended audience will never touch any of the wiki pages. That will be done by a self-selected minority who *think* they are good. Remember, the issue is not "disconnect". The problem is - people selected for programming skill, not writing skill (or even the ability to notice the grammatical errors and typos that bother you and me), - who know too much about the system (if they didn't, would they have the confidence to edit the documentation?) to understand what most needs to be documented. Wikis do not solve this problem, they exacerbate it. > > What the existing docs need right now is editing. There are vast > swathes of text calling out for minor fixes. Why let those problems > sit around for yet more years? Why set the bar for improving them so > high? How high *is* the bar? Have you ever submitted a documentation patch and had it rejected? > My beef is primarily with what's already out there, and secondarily > with how it's packaged online - searching is still a pain, the docs > are still seriously underlinked. With searching and linking, help is at hand. There are plenty of free information retrieval systems out there, like Zettair and Atire (formerly Ant). It's dead simple to take a bunch of HTML files and tell one of these systems "index that lot", and get good searching. I am actually rather surprised that the Erlang documentation site has not done this. As for linking, the INEX consortium has seen a fair bit of research on automatic linking. The reference task had been to try to match the linking in the Wikipedia; that's now understood as having too many links that are not very useful. But we do have some algorithms now that, while not wonderful, are better than nothing. From marcel.meyer@REDACTED Fri Jun 8 00:35:53 2012 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Thu, 7 Jun 2012 18:35:53 -0400 Subject: [erlang-questions] What to use for calling COM on Windows? In-Reply-To: References: Message-ID: Hi there Aleksandr, Tim Watson from the list mentioned such a thing in this post: http://erlang.org/pipermail/erlang-questions/2011-February/056181.html I've not played around with this at all and would be interested in your progress. My current opinion is that if I was to interface with anything, like MSMQ mentioned in that post, I'd do it with message passing, like zero mq and a thin CLR wrapper. Regards, Marcel On Wed, May 30, 2012 at 8:29 AM, Aleksandr Vinokurov < aleksandr.vin@REDACTED> wrote: > > > Hi all, > > What I can use for subj? > > NIF? :) > > -- Aleksandr, from #euc2012 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jun 8 00:42:33 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 10:42:33 +1200 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> Message-ID: On 7/06/2012, at 7:19 PM, Zhemzhitsky Sergey wrote: > Hi Richard, > > Thanks a lot for answer. It is rather helpful. > >>> Why was it hard to see in the documentation? > As I'm kinda a newbie at erlang I understand your indignation. There was no indignation there. There has been a lot of discussion here about documentation and what to do to improve it, and I really wanted to know what went wrong here. Would it be sufficient to have a document "Exchanging data between Erlang and C : Overview" with links between it and the libraries? What would you like to see in examples? From ok@REDACTED Fri Jun 8 00:46:03 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 10:46:03 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: On 7/06/2012, at 8:40 PM, Michael Turner wrote: > Occasionally accepting documentation patches whenever offered doesn't > seem to be resulting in cleanup of errors that have been hanging > around in the documentation for years. Since the barrier to getting corrections in is in fact low, it would appear that people aren't *sending* patches for those errors. When I am able to build the documentation I will start sending patches. From ok@REDACTED Fri Jun 8 00:46:47 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 10:46:47 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> <4FD06988.6090701@ninenines.eu> Message-ID: <000D3A24-2070-41C3-9D60-F06282EB1BC4@cs.otago.ac.nz> > > BECAUSE THAT WOULD BE A DOCUMENTATION PATCH, AND I DON'T KNOW WHERE TO > START IN SUBMITTING DOCUMENTATION PATCHES. There is a patches mailing list. It's documented. From ok@REDACTED Fri Jun 8 01:06:16 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 11:06:16 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> References: <30A9F1B4-4801-438B-B6FE-21E4D6C552CC@feuerlabs.com> Message-ID: On 7/06/2012, at 9:15 PM, Ulf Wiger wrote: > The DTDs *are* documented: > > http://www.erlang.org/doc/apps/erl_docgen/users_guide.html Great! Many thanks! Could someone please change the graphics example? Figure 6.1 does NOT show "A Silly Man" but a man with hyperelastic skin, so he is suffering from some kind of connective tissue disease, *possibly* a form of Ehlers-Danlos Syndrome. A similar condition produces winged cats (http://www.messybeast.com/winged-cats.htm). From ok@REDACTED Fri Jun 8 01:33:56 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 11:33:56 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <4FD060EE.4020902@erlang.org> Message-ID: <8114D2CD-0B18-457F-9ED5-036AA04D21E6@cs.otago.ac.nz> On 7/06/2012, at 10:13 PM, Ulf Wiger wrote: > At the very least, the github wiki should have a link to the OTP > installation guide, > > http://www.erlang.org/doc/installation_guide/INSTALL.html Section 3.8 starts with the non-sentence "GS now Tcl/Tk 8.4." Presumably the word "uses" or "requires" has dropped out here. > > which among many other things covers what to do when you > run into problems with FOP. We must be looking at different pages somehow. I clicked on that link, and searched for FOP. Just two things turned up: The "Building Documentation" part of section 3.4 says that FOP is needed and where to get it. The "Build Issues" part of section 3.6 mentions problems with Java running out of memory, suggests a way to cope, and directs you to an Apache web page. There is *nothing* about any *other* problems with FOP. In particular, the problem I ran into yesterday and reported in this mailing list is *not* there. Let me show once more what I get when I try 'make docs': fop -fo ../pdf/stdlib-1.18.1.fo -pdf ../pdf/stdlib-1.18.1.pdf Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/fop/cli/Main Caused by: java.lang.ClassNotFoundException: org.apache.fop.cli.Main at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) make[3]: *** [../pdf/stdlib-1.18.1.pdf] Error 1 I have now found the cause: a copy-and-paste error in setting up $FOP_HOME. That fixed, I ran straight into the out-of-memory error. The default heap space for Java is just 64MiB; if this is such a well known problem with such an easy fix, why not just patch the relevant Makefile(s) to set FOP_OPTS suitably? From ok@REDACTED Fri Jun 8 01:54:36 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 11:54:36 +1200 Subject: [erlang-questions] Java error making docs in R15B01 In-Reply-To: References: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> Message-ID: On 7/06/2012, at 11:59 PM, Michael Turner wrote: > "By the way, are the DTDs used in building Erlang/OTP documentation > *documented* anywhere?" > > Didn't Ulf just point them out on another thread? E-mail is a *distributed* system. Ulf's message saying where to find the documentation arrived in my mailbox the day after I sent my question. The right thing, of course, is for each of the DTDs to contain a comment saying where to find the documentation, or better still, for the documentation to be automatically extracted from comments in the DTDs. There are some issues that are not documented. When I've set up DTDs and had a use for something that was also found in HTML, I have generally use the HTML names for things. Thus when I have a table, I use , , , I want to know why. Perhaps some other template is being copied? DocBook maybe? It would be useful to know, because I might find useful background in the thing that was copied. Similarly, the HTML 3.2 character entities are copied, well and good, but why not the HTML 4 ones? I'm thinking of section 24.3 "Character entity references for symbols, mathematical symbols, and Greek letters" with pubid "-//W3C//ENTITIES Symbols//EN//HTML". If we're using xsltproc and FOP, which use Unicode, what's wrong with these Unicode characters? Can I use numeric references to get them? If there's some basic limit here, if, for example, it is still thought important to generate pure HTML 3.2, I'd like to know. Or is it a limitation of UNIX man pages? Need that limit apply to documents that won't be processed by troff? From ok@REDACTED Fri Jun 8 02:16:24 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 12:16:24 +1200 Subject: [erlang-questions] Trouble with 'make docs' Message-ID: <242A68FD-469F-4B31-959C-D8F0566F0415@cs.otago.ac.nz> fop -fo ../pdf/asn1-1.7.fo -pdf ../pdf/asn1-1.7.pdf Jun 8, 2012 11:33:42 AM org.apache.fop.events.LoggingEventListener processEvent WARNING: Line 1 of a paragraph overflows the available area by more than 50 points. (See position 3001:226) Lots more of those. (cd ../../java_src;javadoc -sourcepath . -d ../doc/html/java \ -windowtitle 'Java-Erlang Interface Library' com.ericsson.otp.erlang) Creating destination directory: "../doc/html/java/" Loading source files for package com.ericsson.otp.erlang... Constructing Javadoc information... ./com/ericsson/otp/erlang/._OtpCookedConnection.java:1: illegal character: \0 Mac OS X 2??ATTR'?2??&?&com.apple.quarantine0000;4fd01bab;Safari;|com.apple.SafariThis resource fork intentionally left blank ?? ^ ./com/ericsson/otp/erlang/._OtpCookedConnection.java:1: illegal character: \5 Mac OS X 2??ATTR'?2??&?&com.apple.quarantine0000;4fd01bab;Safari;|com.apple.SafariThis resource fork intentionally left blank ?? ^ Lots more of those. When you download and unpack something, Mac OS X helpfully creates a resource fork that basically says "don't trust this". Just opening a file creates a long slow bounce that eventually pops up a dialogue box saying "this is an application that came from the net, do you really want to run it" when all you are doing is trying to view a text file with an editor. The convention is that foo.bar gets an associated "._foo.bar" which must for most purposes be ignored. If there's any way to stop these files being created, I don't know what it is. Some Makefile could do find . -name '[.]_*' -exec rm "{}" + Standard Doclet version 1.6.0_31 Building tree for all the packages and classes... Generating ../doc/html/java/com/ericsson/otp/erlang//AbstractConnection.html... /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/jinterface/java_src/./com/ericsson/otp/erlang/AbstractConnection.java:276: warning - @param argument "msg" is not a parameter name. Generating ../doc/html/java/com/ericsson/otp/erlang//OtpConnection.html... /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/jinterface/java_src/./com/ericsson/otp/erlang/OtpConnection.java:414: warning - @param argument "msg" is not a parameter name. A couple more like that. Generating ../doc/html/java/com/ericsson/otp/erlang//OtpErlangString.html... /home/cshome/o/ok/Downloads/otp_src_R15B01/lib/jinterface/java_src/./com/ericsson/otp/erlang/OtpErlangString.java:53: warning - Tag @return cannot be used in constructor documentation. It can only be used in the following types of documentation: method. All of these resulted in 'make docs' being derailed part way through. I don't believe anyone can ever have built the docs on a Mac from a download without running into these problems. From paperless@REDACTED Fri Jun 8 02:20:18 2012 From: paperless@REDACTED (Tim McNamara) Date: Fri, 8 Jun 2012 12:20:18 +1200 Subject: [erlang-questions] Documentation tooling Message-ID: Would the Erlang community be open to using Sphinx? It is a Python tool, but produces excellent documentation. In addition, the reStructuredText format is supported natively with GitHub. You can see some example output: http://readthedocs.org/ From robert.virding@REDACTED Fri Jun 8 02:55:29 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 08 Jun 2012 01:55:29 +0100 (BST) Subject: [erlang-questions] Data structures are strange things In-Reply-To: <4a0b389f-5cb3-4768-8eba-acf95d03cce3@knuth> Message-ID: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> I was finally going to release some alternate modules with the dict interface using tree structures, either AA trees or Red Black trees. These are relatively well-known binary tree structures which are binary tree versions of 2-3 trees and 2-3-4 trees respectively. These contain codes with either 2, 3 or 4 sub-nodes. Generally the binary versions are recommended as being better/easier/more space efficient. Before I released my modules I thought it would be interesting to try the non-binary versions to see how they behaved and what it was lie to program them. I was lucky to find a very good description of 2-3 trees and the rules for inserting/deleting elements; it was literally just translate the rules straight to Erlang and it worked. The code was a little more complex than working with the binary versions, not much. In some cases actually simpler. The I decided to measure speed, and the fun started. All tests were done with sequences of random numbers as keys, the same for each implementation. What I found was that the lookup times were about the same as for the binary versions, which was expected, but the insertion/deletion times were about 20% faster. This compared to my AA and RB tree versions. To check that it wasn't just bad coding on my part (not bloody likely :-)) I compared it with gb_trees and got the similar results. Also checking the data sizes I found that the 2-3 and 2-3-4 trees actually used *less* space than their binary counterparts. Here are the figures. I have also included dict, which is the slowest on insertion/deletion but significantly faster lookup times. All times are in usecs for the whole set: Operation ? ? ? ? ? ?Load ? ? ? ?Fetch ? ? ? Load ? ? ? ?Fetch # of elements? ? ? ? ? ? ? ? ?100000 ? ? ? ? ? ? ? ? ?10000 Overhead time? ? ? ? ? ? ? ? ? 43000 ? ? ? ? ? ? ? ? ? 3700 ttdict 2-3 trees 310000 161000 24000 15500 ttfdict 2-3-4 trees 312000 173000 24200 16500 llrbdict Left Leaning RB trees 390000 164000 27100 14900 rbdict Red Black trees 380000 167000 27100 15000 aadict AA trees 423000 167000 30000 15000 gb_trees using faster insert/3 370000 190000 26500 16200 dict 522000 107000 28000 11000 (I hope the table is readable) Gb_trees are a variant of AA trees with a slightly different algorithm. One should be careful interpreting exact values here but the trend is clear. I don't know why the straight multi-size versions are faster but my guess is that the pattern-matching/data construction is faster on the flat bigger nodes rather than on the nested pairs. My guess is that when they comment on the 2-3(-4) trees using more space it is because they use the same size node for all cases which in the cases with fewer sub-nodes wastes space, while just used different sized tuples which didn't waste space; it also made the pattern matching much easier. It looks like I will be submitting the 2-3 tree version, ttdict, as reasonable choice. Next I will try AVL trees which were discarded long ago and see how they fare. Robert P.S. This is the reason why there was hoped to be many different dict modules with the same API but different data structures. So as it to make it easier to choose the most suitable version for each occasion. There is no generally "best" choice, even among trees. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jun 8 03:21:04 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 13:21:04 +1200 Subject: [erlang-questions] Trouble with 'make docs' In-Reply-To: <242A68FD-469F-4B31-959C-D8F0566F0415@cs.otago.ac.nz> References: <242A68FD-469F-4B31-959C-D8F0566F0415@cs.otago.ac.nz> Message-ID: The setup we are talking about is Erlang R15B01 downloaded yesterday, FOP 1.0 downloaded yesterday, nothing changed whatever, Mac OS X 10.6.8. I looked at one of the generated PDFs, erl-interface-3.7.7.pdf The sources contain (in ei.xml) stuff like intei_encode_ulong(char *buf, int *index, unsig ned long p) intei_x_encode_ulong(ei_x_buff* x, unsigned lon g p) Encode unsigned integer The PDF contains stuff that looks like intei_encode_ulong(char *buf, int *index, unsigned long p) intei_x_encode_ulong(ei_x_buff* x, unsigned long p) In no case do I see a space generated between and . I did notice that ei_reg *ei_reg_open(size) looked OK, but that's because the space is inside the . (I don't understand why some functions in this PDF have their types inside the and some don't; still less do I understand why the same tag is used in both cases.) I'm seeing exactly the same issue in the man pages. For example, .LP .B voidei_set_compat_rel(release_number) .br .RS in ei.3 This suggests a bug somewhere in the XSLT, but I've found XSLT both _amazingly_ bulky (XSLT source : equivalent Scheme source :: 12 : 1) and about as readable as TECO, so at this point I give up. From ok@REDACTED Fri Jun 8 03:32:48 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Jun 2012 13:32:48 +1200 Subject: [erlang-questions] Data structures are strange things In-Reply-To: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> References: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> Message-ID: <27729434-CD15-4082-B0DF-AC99CCFAE34A@cs.otago.ac.nz> If you take a look at a typical 2-3-4 tree implementation in Java http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java >> http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java you'll see nodes containing an integer saying how many keys there are three keys four links to children nodes a link to a parent node Allow one word for "what is the class of this object" and that's 10 words per node. Using Erlang tuples, we'd have {K1,C0,C1} 4 words {K1,K2,C0,C1,C2} 6 words {K1,K2,K3,C0,C1,C2,C3} 8 words WAM-based Prolog systems would take the same space; SML/NJ should be able to shave a word off each (using the bottom 2 bits of a pointer to say which of the three cases applies). In a functional language, when you make a change you create new nodes, so no node ever has to change size/shape. You _could_ do that in Java too, but people often don't think of it. It seems pretty clear that the Prolog/Erlang/SML version of a 2-3-4 tree is going to take *less* space than a binary tree in the same language. Since in these languages there is often a strong correlation between space and time, I find Robert Virding's results pleasant but not that surprising. From michael.eugene.turner@REDACTED Fri Jun 8 06:24:05 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 8 Jun 2012 13:24:05 +0900 Subject: [erlang-questions] Data structures are strange things In-Reply-To: <27729434-CD15-4082-B0DF-AC99CCFAE34A@cs.otago.ac.nz> References: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> <27729434-CD15-4082-B0DF-AC99CCFAE34A@cs.otago.ac.nz> Message-ID: "I don't know why the straight multi-size versions are faster but my guess is that the pattern-matching/data construction is faster on the flat bigger nodes rather than on the nested pairs...." Just a guess here (but at least a guess informed by an obsession with search tree data structures, several decades ago): Before you go looking for the effects of a reduction of space, remember that the conventional time complexity analysis for operations on such data structures is predicated on destructive updates. An O(log n) time complexity for delete or update represents the *worst case*. Some operations will result in no rebalancing at all, just the addition or removal of a node. Without destructive updates, you always need to rebuild/rewrite tree nodes from the point of update up to the root. Even if nothing substantive changes along the way. It'll cost ya. In space, too. How can you reduce these costs? By systematically reducing the distance to the root. You can do that by choosing a structure with a higher (average) branching factor. The higher the branching factor of your search structure, the shallower it will be, and the fewer node-allocate-and-rewrite operations you'll need to do on your way from the updated node to the root. (This probably reduces GC costs as well, though I expect that would be a second-order effect.) Increasing the branching factor might have diminishing returns after some point, but I doubt that a branching factor of 4 is anywhere near that point. 2-3 trees, and 2-3-4 trees, have higher (average) branching factors than binary trees. Leaning RB trees are binary, but there might be something in how they lean that effectively reduces the average distance from updated node to root. I don't know anything about them. "P.S. This is the reason why there was hoped to be many different dict modules with the same API but different data structures. So as it to make it easier to choose the most suitable version for each occasion. There is no generally "best" choice, even among trees." Sign me up for that. -michael turner On Fri, Jun 8, 2012 at 10:32 AM, Richard O'Keefe wrote: > If you take a look at a typical 2-3-4 tree implementation in Java > ? ?http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java >>> ?http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java > you'll see nodes containing > > ? ? ? ?an integer saying how many keys there are > ? ? ? ?three keys > ? ? ? ?four links to children nodes > ? ? ? ?a link to a parent node > > Allow one word for "what is the class of this object" and that's > 10 words per node. > > Using Erlang tuples, we'd have > ? ? ? ?{K1,C0,C1} ? ? ? ? ? ? ? ? ? ? ?4 words > ? ? ? ?{K1,K2,C0,C1,C2} ? ? ? ? ? ? ? ?6 words > ? ? ? ?{K1,K2,K3,C0,C1,C2,C3} ? ? ? ? ?8 words > WAM-based Prolog systems would take the same space; > SML/NJ should be able to shave a word off each (using the bottom > 2 bits of a pointer to say which of the three cases applies). > > In a functional language, when you make a change you create new nodes, > so no node ever has to change size/shape. ?You _could_ do that in Java > too, but people often don't think of it. > > It seems pretty clear that the Prolog/Erlang/SML version of a 2-3-4 > tree is going to take *less* space than a binary tree in the same > language. ?Since in these languages there is often a strong correlation > between space and time, I find Robert Virding's results pleasant but > not that surprising. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Fri Jun 8 09:19:21 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 8 Jun 2012 16:19:21 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: On Fri, Jun 8, 2012 at 7:08 AM, Richard O'Keefe wrote: [snip] > Recall that Erlang comes from Sweden. > Things that are "glaring typos and grammatical errors" to a native > speaker of English might very well be not so apparent to a native > speaker of Swedish. Not *as* apparent, no. But I've seen some who aren't native speakers write things on this list that put some native speakers to shame. And the history of Erlang development is littered with names of people who I'm sure are native English speakers. (Admittedly, sometimes Joe sounds like he's been in Sweden too long.) > I strongly suspect that many of the people associated with the > development of Erlang don't normally *notice* such things. Be that as it may, this thread is (or has become) about how those who *do* notice such things might be able to contribute to this aspect of document quality much more easily. Not to mention the dearth of hyperlinks in the text, a task where native English ability is far less important. [snip] >> Richard, I'm not aiming for anything near perfection, but rather "not >> embarrassing." It's embarrassing for errors to sit around for years >> while new ones accumulate. > > It's embarrassing *if you notice*. No, it's embarrassing to me personally, when I rave at someone about how wonderful Erlang/OTP is, and bring up a manual page about some cool package, but there it is, right in front of them, unavoidable: awkward, broken, poorly punctuated English. Especially when I've pointed out that one of Erlang's strengths is that it's a professional, industrial-strength product, not just an open source project. "Well, you know: it's a bunch of Scandinavians." Yes, I'll have to try that excuse next time. Except that I also rave about Joe's book. Not many Swedes named Joe, I think. >> [To be clear: it's not me writing this next line:] >>> ? ? ? ?Never let an engineer write the documentation. >> >> *Never*? Not even one who's proven his/her worth for years as a >> technical writer? Such people do exist, you know. > >>> Our first technical writer, Kennita Watson, was technically competent. > > Did you read that sentence? Yes. I read it as "technically competent *as an engineer*", because (a) she has degrees in computer science (from Stanford and MIT at that -- facts I recalled about her the instant I read her name), and (b) what followed was an anecdote that I didn't quite get, because somebody (you?) wrote "Never let an engineer write the documentation" and I thought you were agreeing with the point -- you certainly didn't contradict it immediately. So at first I read your comment on her work as saying that she'd suggested changes in the documentation to make things more documentable from *her* point of view (as an engineer), but that they were still things that the customers didn't understand. > "Engineer" in this context means "someone who implemented/maintained > the specific system being documented". And to me, "engineer" can ALSO mean "an engineer assigned to do technical writing", which is what your (?) "Never let an engineer write the documentation" (now followed with any immediate contradiction of the claim) primed me to think. Reading involves perception. Unclear writing can distort perception, by activating the wrong concepts. It happens in speech too. You are rarely unclear, so I assumed that the assumption I made about your sentiment, and my misperception that followed, were both correct. A nice little lesson in itself about writing, come to think of it. > The ideal is someone very like Kennita: ?someone with a general technical > background who is *able* to learn the information customers need but does > not know it *yet*, and writes up what she's learned right at the cusp > where she now understands it but vividly remembers not understanding it. I'd like to suggest a better ideal: an intelligent teacher who never stops thinking about how to clearly explain things to newbies, no matter how deep he/she gets into a subject. And who never stops practicing on the newbies. This might help explain why some of the better lower-division undergraduate textbooks are written by professors who win high marks from their students for effective teaching of the subject. These professors might be able to solve research-level problems in the same field with ease, but they stay intrigued with the perennial (and perhaps for them, more challenging) problem of how to make the field more intuitive and manageable for those completely new to it. Such people are admittedly as rare as hen's teeth in the more quotidian livelihood of tech writing. The Kennitas of that world are rare enough. (I think she was only sidelined because of some injury -- I'm not sure. I'd kill to make use of her on a project myself, if she were still available.) Which takes me back to my agenda: what's the *practical*, *dirt-cheap* way to *eventually* get better quality docs for Erlang/OTP? Maybe it's only because wikis are what I know best, but I keep coming back to wikis. [snip] > I am not sure what that sentence is directed at. I hope you now understand how I misunderstood your understanding of what I understood to be your ... um .... never mind. [beautiful story about how a need to clarify things for beginners resulted in a better design, even a better Prolog standard - snipped.] I agree. In fact, this should be part of the case for Erlang/OTP to hire tech writers: that a good one can actually provide valuable feedback to engineers for whom some (initially) counterintuitive, user-hostile aspect of a system has become second nature. As some UI guru once said, "The intuitive is the familiar. Period." Anybody - and especially any *engineer* - who claims they've got something that's "intuitive" should always be confronted with: "to whom?" > Most of the intended audience will never touch any of the wiki pages. > That will be done by a self-selected minority who *think* they are good. And their edits can easily be reviewed. If they don't pass muster, they will quickly be reverted by those who think they are good enough because they actually *are* good -- who tend to be much more devoted to the wiki as well (And who tend to slightly underestimate their abilities, according to the Dunning-Kruger Effect.) If someone contests a revert, it can go to Talk page discussion. If reasoned discussion doesn't work with the incompetent, the conflict can be escalated until it comes to the attention of someone whose judgment no sensible person would question - especially after that arbitrator explains the judgment. If the problematic editor persists in the face of that, you ban them. Wikis are not anarchy. Wikis are not "waterfall" development either. Wikis are an entirely new medium for collective effort on text. The successful wikis have (or evolve) intelligent governance structures. People will say a lot of silly things about wikis and their governance until they gain more direct experience of maintaining them. In that, wikis are like anything in life. > Remember, the issue is not "disconnect". ?The problem is > ?- people selected for programming skill, not writing skill (or even > ? the ability to notice the grammatical errors and typos that bother > ? you and me), > ?- who know too much about the system (if they didn't, would they > ? have the confidence to edit the documentation?) to understand what > ? most needs to be documented. > Wikis do not solve this problem, they exacerbate it. You're prone to sweeping generalizations about wikis based on no direct personal experience of editing them, from what I can tell. You're appealing to the Dunning-Kruger Effect http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect as an argument against wikis without realizing that you yourself lack the competence to make these judgments. In fact, every single objection you make here is something that every experienced editor of wikis has long since understood. All of these problems have long since been addressed with various strategies, subject to various trade-offs. [Me:] >> What the existing docs need right now is editing. There are vast >> swathes of text calling out for minor fixes. Why let those problems >> sit around for yet more years? Why set the bar for improving them so >> high? [Richard:] > How high *is* the bar? ?Have you ever submitted a documentation patch > and had it rejected? I've never submitted a documentation patch. But on the code itself, yes. It was rejected as badly formatted. But then Akira Kitada banged it into better shape, only to see rejected AGAIN -- by someone who couldn't figure out how he missed it. http://erlang.org/pipermail/erlang-patches/2009-September/000472.html Not exactly earth-shaking, but Akira found it useful, so I count it toward my good-deeds-done. You hate amateurs. I can understand that. But the experts fail sometimes too. If I submitted a patch to the seq_trace documentation so that it mentioned that seq_trace implements Lamport clocks, and it got rejected with "we're still thinking about how to work that fact in, maybe we'll have something in the next release," I think I'd end up writing Lamport to say, "Can you *believe* these people?" But I'm used to being able to fix problems like that in under a minute, because I edit tech wikis. Call me spoiled. Call me an amateur, even. But this is one that the pros are getting wrong. [Good stuff about Zettaiir/Atire and INEX work on autolinking] > ...?But we do have some algorithms > now that, while not wonderful, are better than nothing. Good ideas, I'm sure. I'm using Erlang to implement a computational linguistics framework, so I'm definitely keen on those sorts of solutions myself. But automating a task to get "better than nothing" shouldn't be the goal here. This is supposed to be documentation by professionals, for professionals. As your experience with Kennita shows, the more properly-primed human intelligence you apply to documentation, the better. An intelligently administered wiki can funnel the right kinds of eyes to the right places, at a price-point that Erlang/OTP can afford. I happen to hate the price constraint. I'd like to sell them on getting a proper tech writer myself. How likely is that, though? I mean, look at your own labors of love: you've submitted a number of EEPs, with excellent ideas, beautifully written. Ericsson haven't gotten around to a fraction of them. In a decade, in some cases. When are they going to split the atom already? Who knows? That's got to be a lot more important than my nitpicks on the documentation. Yet there's no movement on it. And what does that tell you about their resource constraints? -michael turner From ulf@REDACTED Fri Jun 8 09:47:09 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 09:47:09 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <8114D2CD-0B18-457F-9ED5-036AA04D21E6@cs.otago.ac.nz> References: <4FD060EE.4020902@erlang.org> <8114D2CD-0B18-457F-9ED5-036AA04D21E6@cs.otago.ac.nz> Message-ID: <4562BBA9-4129-47EA-BC1E-935A02DAD2C9@feuerlabs.com> On 8 Jun 2012, at 01:33, Richard O'Keefe wrote: > We must be looking at different pages somehow. > I clicked on that link, and searched for FOP. [?] > The "Build Issues" part of section 3.6 mentions problems > with Java running out of memory, suggests a way to cope, > and directs you to an Apache web page. Yes, that was what I meant. Apologies for my terseness. I should perhaps have written something along the lines of "that page does explain some of the tricky aspects of the OTP build system, including some of the problems *I* have encountered. such as the out-of-heap memory problems with FOP. Due to time pressures, that was somehow reduced to "at the very least, the github wiki should have a link?". I still think it should, but more is definitely needed. > There is *nothing* about any *other* problems with FOP. > In particular, the problem I ran into yesterday and reported > in this mailing list is *not* there. I can imagine that many people who run into problems don't think to go to that page, and so are less likely to give any feedback on it. > fop -fo ../pdf/stdlib-1.18.1.fo -pdf ../pdf/stdlib-1.18.1.pdf > Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/fop/cli/Main > Caused by: java.lang.ClassNotFoundException: org.apache.fop.cli.Main > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > make[3]: *** [../pdf/stdlib-1.18.1.pdf] Error 1 > > I have now found the cause: a copy-and-paste error in setting > up $FOP_HOME. I think the wiki and Erlang docs should clearly discourage people from running FOP - and 'fakefop' should be the default - *unless* you want to produce PDFs. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From avinash@REDACTED Fri Jun 8 10:24:24 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Fri, 08 Jun 2012 13:54:24 +0530 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell Message-ID: Where may I look further for the basic understanding of spatio-temporal relations among the "things" named as ERTS, Emulator and Shell? The question occurred while I was toying with "erl" as Unix-filter shell-command and couldn't get over the options (to erl) like "-noshell", "-s", "-detached", init:stop(), and so on in the documentation and in the FAQ. Thanks From gordon@REDACTED Fri Jun 8 10:47:10 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 8 Jun 2012 09:47:10 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema Message-ID: Folks Having some problems with mnesia backups. I have servers running with lots of tables for lots of site, so we use a naming convention for the tables to separate the data: site_1_table_1, ..., site_1_table_n ... site_m_table_1, ..., site_m_table_n I want to be able to backup individual sites and restore them - maybe on the original machine, maybe on another one. So I setup a checkpoint across a set of tables (site_1_table_1 to site_1_table_n) and back it up. The tables can be quite big so mnesia:restore/2 isn't practical and I have to use mnesia:install_fallback - this is where I hit problems. I get the error message: {error,{"Cannot install fallback", {"Cannot lookup",schema}}} Ok, I figured it needs the schema to be backed up as well, so I then build a checkpoint with the schema and the set of tables I want to backup. But when I run that I get a different error: {error,{"Cannot install fallback", {already_exists,'hndev@REDACTED'}}} So I am at a bit of a loss. Does install_fallback only work with whole table backups? Gordon From rtrlists@REDACTED Fri Jun 8 10:53:08 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 8 Jun 2012 09:53:08 +0100 Subject: [erlang-questions] What to use for calling COM on Windows? In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 1:29 PM, Aleksandr Vinokurov < aleksandr.vin@REDACTED> wrote: > > Hi all, > > What I can use for subj? > > NIF? :) > > -- Aleksandr, from #euc2012 > > One way would be to use one of the Lua wrappers for Erlang and then use LuaCOM to call into COM. Not sure if that helps you at all, though. In a pinch, you could use the code from LuaCOM as inspiration to help you write a more direct connection. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Fri Jun 8 10:53:44 2012 From: janburse@REDACTED (Jan Burse) Date: Fri, 08 Jun 2012 10:53:44 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: <4FD1BD98.5070004@fastmail.fm> Michael Turner schrieb: > Which takes me back to my agenda: what's the*practical*,*dirt-cheap* > way to*eventually* get better quality docs for Erlang/OTP? Since years there are spelling checkers and grammar checkers around, even on unix command level. Run your docs through it and you can turn a swenglish text into an english text. http://www.linfo.org/spell.html http://www.gnu.org/software/diction/ From chandrashekhar.mullaparthi@REDACTED Fri Jun 8 11:33:10 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 8 Jun 2012 10:33:10 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: Hi Gordon, The node name is in the schema. If you want to install_fallback on another node, you have to change the node name in the backup file. Attached is a module which might be of use to you. We use it quite a lot in our setup. The code here is based on an example in the Mnesia user guide. cheers Chandru On 8 June 2012 09:47, Gordon Guthrie wrote: > Folks > > Having some problems with mnesia backups. > > I have servers running with lots of tables for lots of site, so we use > a naming convention for the tables to separate the data: > site_1_table_1, ..., site_1_table_n > ... > site_m_table_1, ..., site_m_table_n > > I want to be able to backup individual sites and restore them - maybe > on the original machine, maybe on another one. > > So I setup a checkpoint across a set of tables (site_1_table_1 to > site_1_table_n) and back it up. > > The tables can be quite big so mnesia:restore/2 isn't practical and I > have to use mnesia:install_fallback - this is where I hit problems. > > I get the error message: > > {error,{"Cannot install fallback", > {"Cannot lookup",schema}}} > > Ok, I figured it needs the schema to be backed up as well, so I then > build a checkpoint with the schema and the set of tables I want to > backup. But when I run that I get a different error: > > {error,{"Cannot install fallback", > {already_exists,'hndev@REDACTED'}}} > > So I am at a bit of a loss. > > Does install_fallback only work with whole table backups? > > Gordon > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cnm.erl Type: application/octet-stream Size: 3479 bytes Desc: not available URL: From gordon@REDACTED Fri Jun 8 11:41:00 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 8 Jun 2012 10:41:00 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: Chandru I have built a code changer/backup traverser based on the same code. I am trying to get the full cycle to work backing up and restoring on the same node first before I move to the two-node cycle. I can't restore a partial backup on the same node with install_fallback at the moment. Gordon On 8 June 2012 10:33, Chandru wrote: > Hi Gordon, > > The node name is in the schema. If you want to install_fallback on another > node, you have to change the node name in the backup file. Attached is a > module which might be of use to you. We use it quite a lot in our setup. > > The code here is based on an example in the Mnesia user guide. > > cheers > Chandru > > On 8 June 2012 09:47, Gordon Guthrie wrote: >> >> Folks >> >> Having some problems with mnesia backups. >> >> I have servers running with lots of tables for lots of site, so we use >> a naming convention for the tables to separate the data: >> site_1_table_1, ..., site_1_table_n >> ... >> site_m_table_1, ..., site_m_table_n >> >> I want to be able to backup individual sites and restore them - maybe >> on the original machine, maybe on another one. >> >> So I setup a checkpoint across a set of tables (site_1_table_1 to >> site_1_table_n) and back it up. >> >> The tables can be quite big so mnesia:restore/2 isn't practical and I >> have to use mnesia:install_fallback - this is where I hit problems. >> >> I get the error message: >> >> ? ? {error,{"Cannot install fallback", >> ? ? ? ? ? ? ?{"Cannot lookup",schema}}} >> >> Ok, I figured it needs the schema to be backed up as well, so I then >> build a checkpoint with the schema and the set of tables I want to >> backup. But when I run that I get a different error: >> >> ? ? {error,{"Cannot install fallback", >> ? ? ? ? ? ? ?{already_exists,'hndev@REDACTED'}}} >> >> So I am at a bit of a loss. >> >> Does install_fallback only work with whole table backups? >> >> Gordon >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ulf@REDACTED Fri Jun 8 11:55:05 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 11:55:05 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> On 8 Jun 2012, at 09:19, Michael Turner wrote: > If I submitted a patch to the seq_trace documentation so that it > mentioned that seq_trace implements Lamport clocks, and it got > rejected with "we're still thinking about how to work that fact in, > maybe we'll have something in the next release," I think I'd end up > writing Lamport to say, "Can you *believe* these people?" Up to this point, I thought you were on a roll with clear, insightful and respectful discussion not least about the problems of understanding each other in writing, and conveying ideas through documentation. I can understand that certain points can get lost in a long and heated thread, so let me clarify a few things: - I don't work for Ericsson (as also apparent from my sig) - I don't vet patches on OTP. I have to submit patches like everyone else; sometimes they are rejected, mostly not. - Whatever changes *I* would like to see to seq_trace, are my own ideas; It's been quite a while since the topic of seq_trace came up in any discussions I've had with OTP, and then only in passing (although I *did* push for changes to TTB a while back that would make sequence tracing, and other things, easier, and was glad to finally see the changes, most of which not made by me, accepted). In this particular case, the most relevant passage of our long discussion should be Gustav Simonsson (who *does* represent OTP) saying that your patch was welcome: [Gustav:] > +1 > > Especially under the "Advanced examples" section there > is less need for brevity since users clicking on that section > likely are prepared for multiple, possibly long, examples > of using the Event Tracer. Most doc- and other patches are unsolicited. I have a number of times asked for advance feedback on the list, to get some indication of whether it would be worth my while to put in the time and effort to make a patch (in those cases, changes that affect the API and documentation). Usually, such queries have gone unanswered, and I'm left to decide for myself whether I should embark on a project of writing code, test suites and documentation and making an argument that (1) the change is an improvement, (2) doesn't break anything, (3) doesn't confuse users who don't need the new feature, etc. *That* can easily amount to a week's worth of work, with no guarantee that it will be accepted. In the most recent case, it was much, much more than that: a fairly large addition to mnesia that has been many years in the making, and was brought to patchworthy status only through the sponsorship of Klarna, and the help of Roland Karlsson at Erlang Solutions, since I was no longer in a position to complete it. That change was actually rejected a few times in the early stages. For the larger project, we visited the OTP team to present our ideas and get an early indication that it was a plausible approach - no guarantees that it would finally be accepted, but as much encouragement as was reasonable at the time. http://www.erlang-factory.com/upload/presentations/601/EUC2012-009.pdf Personally, I think that's ok. It's my idea, and others may not agree, or otherwise find it important. However, for everyone (not least on the OTP team) who spots such requests for early feedback: it really means a lot to get comments early on. As for feedback from me, I'm just another user on the list. I have never been in a position to decide what goes into OTP (well, except for being on the EEP committee). Kenneth has no problem rejecting my advice, although I believe he values it. The only concrete comments you've gotten this time from OTP have been words of encouragement: submit a patch, and it will be accepted as long as it improves the documentation - not a single hint that they are planning anything else for seq_trace. We have no reason to think that they are, but now would of course be a good time to say so then. :) I reserve the right to give feedback with the intent of being helpful, but it is *not* to be interpreted as an opinion from the Erlang maintainers. Any such opinion must come straight from the OTP team, preferrably framed in such a way that it is clear that they are not just engaging in free and open debate. There are others who are actually on the fence: Joe, Richard Carlsson, Kostis, ? who are not on the OTP team, but deeply enough involved that it can sometimes be hard to make out whether they speak for the Team or not. In case of doubt, it's best to ask for clarification and generally assume they don't. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From desired.mta@REDACTED Fri Jun 8 12:33:32 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Fri, 8 Jun 2012 12:33:32 +0200 Subject: [erlang-questions] Dialyzer, Callback info about the behaviour is not available In-Reply-To: <20120514154742.GA12528@localhost> References: <7D661B20-3D54-4FBC-95EF-EF846C92F48F@gmail.com> <4F057E22.7080509@cs.ntua.gr> <20120514154742.GA12528@localhost> Message-ID: Bump I am really looking forward into this to have 100% clean dialyzer output for my project. ;) Motiejus On Mon, May 14, 2012 at 5:47 PM, Motiejus Jak?tys wrote: > On Thu, Jan 05, 2012 at 10:08:43PM +0100, Magnus Klaar wrote: >> Hi! >> >> We created a development branch for adding this information to this, and >> other behaviors and came to the same conclusion, we'd have to drop R14 >> support, something which we cannot do when we know that there are users who >> still haven't upgraded their production environments to R15. It was a >> rather negative surprise to see the build fail after having put great care >> into making sure the -callback specifications were accurate. ;) > > Hm, do you still have that development branch so I can check my code > with dialyzer for my own sake? > >> Since we're on the subject, is there currently a good way to add >> specifications for optional callback functions? > > If it causes compile failure on this thread), maybe it would be possible to wrap -callback > specifications with Erlang version macros? > > This way R15B dialyzer users would be happy, and R14B and below would > not be forced to upgrade. > > Regards, > Motiejus -- Motiejus Jak?tys From ulf@REDACTED Fri Jun 8 12:36:43 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 12:36:43 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: <75896D09-B408-49A6-A764-C628F3A4F320@feuerlabs.com> One of my main demands on any wiki-style documentation for OTP would be that it supports baselining, and online and offline browsing of older baselines. The current OTP documentation, for all its flaws, is at least version-consistent. I can check out and build an old version of OTP and the docs will be the version that was set in stable revision together with the code. I won't say that it *can't* be done with wikis, but even such wikis as the Github wiki, which are based on a strong VCS, normally don't make this easy. In the case of the github wiki, you could clone it like any other git repository, label it, and do all sorts of things with it, but the only access you have to older versions of the pages online is through the diffs in the history, and viewing github-flavored Markdown off-line is no fun - or at least requires quite a bit of work for rendered viewing. Such a solution will not be much of an improvement over the current situation. If there is a wiki out there that can do this, I'd be thrilled to hear about it. BR, Ulf W On 8 Jun 2012, at 00:08, Richard O'Keefe wrote: > Remember, the issue is not "disconnect". The problem is > - people selected for programming skill, not writing skill (or even > the ability to notice the grammatical errors and typos that bother > you and me), > - who know too much about the system (if they didn't, would they > have the confidence to edit the documentation?) to understand what > most needs to be documented. > Wikis do not solve this problem, they exacerbate it. Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Fri Jun 8 12:46:09 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 8 Jun 2012 19:46:09 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> Message-ID: On Fri, Jun 8, 2012 at 6:55 PM, Ulf Wiger wrote: > > On 8 Jun 2012, at 09:19, Michael Turner wrote: > >> If I submitted a patch to the seq_trace documentation so that it >> mentioned that seq_trace implements Lamport clocks, and it got >> rejected with "we're still thinking about how to work that fact in, >> maybe we'll have something in the next release," I think I'd end up >> writing Lamport to say, "Can you *believe* these people?" > > Up to this point, I thought you were on a roll with clear, > insightful and respectful discussion not least about the > problems of understanding each other in writing, and > conveying ideas through documentation. > > I can understand that certain points can get lost in a long > and heated thread, so let me clarify a few things: > > - I don't work for Ericsson (as also apparent from my sig) Ulf, at no point in that particular paragraph did I mention you by name. We didn't exactly bury the hatchet, so it's not like you're digging it up. But you are picking it up. > - I don't vet patches on OTP. [....] I never suggested you did. > - Whatever changes *I* would like to see to seq_trace, are > ?my own ideas; [...] Nothing I wrote up there about getting seq_trace documentation to mention that it implement Lamport clocks could reasonably be construed as your own idea, particularly since you initially seemed pretty hazy in your response my statement that seq_trace implements Lamport clocks. Specifically: "... many of the inputs that *actually* informed the implementation were either confidential or proprietary enough to be of no interest to the people reading the manuals." You stressed "*actually*". To me, this strongly implies that Lamport clocks -- whether they came in as a recommendation by Erlang/OTP as a "better forlopp tracing", or because of a customers request for them (in the other version of two as-yet-unreconciled versions of events) -- did not "inform the implementation." Are they in there? Then there was some kind of "accident." Or "coincidence." Which seems ridiculous, since it's pretty obvious from citations in the open literature by half a dozen Erlang/OTP people that they already knew what Lamport clocks were. > In this particular case, the most relevant passage of our > long discussion should be Gustav Simonsson (who *does* > represent OTP) saying that your patch was welcome: If you mean specifically the idea of a patch mentioning that seq_trace implements Lamport clocks, I have yet to see that from Gustav. He wrote that documentation patches are (generally) welcome. Obviously, that's no guarantee of the acceptability for any particular one. And I have it (second-hand from you) that he has himself thought of a good place for this mention, in the seq_trace documentation. To which I replied: if he's got a good idea for that, why doesn't he just do it? I mean, c'mon, this is on the web. It's not like you have to do a whole print-run of some dead-tree reference manuals to fix it today. And every day that the Lamport clock pseudocode sits there in the seq_trace documentation is one more day when Erlang/OTP looks like it's plagiarizing the idea of a major computer scientist. As far as I can tell, NOBODY at Ericsson has, in writing, said directly, "We need to fix that." Until then .... > The only concrete comments you've gotten this > time from OTP have been words of encouragement: submit > a patch, and it will be accepted as long as it improves the > documentation - not a single hint that they are planning > anything else for seq_trace. And I've already responded to that: "improvement" is in the eye (and to some extent, the imagination) of the beholder. So long as it's part of some internal Ericsson mythology that seq_trace only reiterates Lamport clocks as a technique invented independently (and concurrently, if not earlier) in "forlopp tracing", the response can always be: "we invented that, probably before Lamport thought of it, we don't have to credit anybody else, and some of our customers still think we invented it anyway, so leaving it unmentioned is what works best for Ericsson AS A BUSINESS." Until I hear directly from Erlang/OTP people otherwise, I have to keep that possibility open in my mind. Especially since both you and Robert Virding seem to have been under this impression, if you aren't still. > We have no reason to think > that they are, but now would of course be a good time > to say so then. :) I'm all ears. > There are others who are actually on the fence: Joe, > Richard Carlsson, Kostis, ? who are not on the OTP > team, but deeply enough involved that it can > sometimes be hard to make out whether they speak > for the Team or not. In case of doubt, it's best to ask > for clarification and generally assume they don't. I hope it doesn't have to go to lawyers for approval. I mean, whether forlopp tracing, at any point in its evolution, did or did not include Lamport clocks ... is that actually something you can't reveal under NDA? The mind boggles. If it's in there, the fact that it's also a technique in the open literature from around the same time would seem to argue that there's no big deal (in NDA terms, anyway) in telling people. Likewise if it's *not* in there -- you don't even have to divulge details of what IS in there -- you just issue a denial. I can think of reasons why a corporation might be cagey on an issue like this. None of them are terribly complimentary to the corporation. -michael From gordon@REDACTED Fri Jun 8 12:52:33 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 8 Jun 2012 11:52:33 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: Ok At another impasse... I was backing up some tables (or so I thought) with mnesia:backup_checkpoint/3 and passing in the table list. Turns out the table list applies to the checkpoint and not the backup which includes all tables... So if I restore this backup I will restore all the tables from the original site, not some of them... Is it possible to selectively restore tables at all? Gordon On 8 June 2012 11:20, Gordon Guthrie wrote: > Chandru > > Made some progress - if you back up with a schema you can install the > fallback... > > BUT if you have crashed during a failed install_fallback from a > schemaless backup mnesia thinks that a fallback is installed and borks > out with this message > > ? ? {error,{"Cannot install fallback", > ? ? ? ? ? ? ?{already_exists,'hndev@REDACTED'}}} > > So you need to run mnesia:uninstall_fallback/0 after all the failed > attempt whilst you are writing your code :( > > Upwards and onwards... > > Gordon > > On 8 June 2012 10:41, Gordon Guthrie wrote: >> Chandru >> >> I have built a code changer/backup traverser based on the same code. >> >> I am trying to get the full cycle to work backing up and restoring on >> the same node first before I move to the two-node cycle. >> >> I can't restore a partial backup on the same node with >> install_fallback at the moment. >> >> Gordon >> >> On 8 June 2012 10:33, Chandru wrote: >>> Hi Gordon, >>> >>> The node name is in the schema. If you want to install_fallback on another >>> node, you have to change the node name in the backup file. Attached is a >>> module which might be of use to you. We use it quite a lot in our setup. >>> >>> The code here is based on an example in the Mnesia user guide. >>> >>> cheers >>> Chandru >>> >>> On 8 June 2012 09:47, Gordon Guthrie wrote: >>>> >>>> Folks >>>> >>>> Having some problems with mnesia backups. >>>> >>>> I have servers running with lots of tables for lots of site, so we use >>>> a naming convention for the tables to separate the data: >>>> site_1_table_1, ..., site_1_table_n >>>> ... >>>> site_m_table_1, ..., site_m_table_n >>>> >>>> I want to be able to backup individual sites and restore them - maybe >>>> on the original machine, maybe on another one. >>>> >>>> So I setup a checkpoint across a set of tables (site_1_table_1 to >>>> site_1_table_n) and back it up. >>>> >>>> The tables can be quite big so mnesia:restore/2 isn't practical and I >>>> have to use mnesia:install_fallback - this is where I hit problems. >>>> >>>> I get the error message: >>>> >>>> ? ? {error,{"Cannot install fallback", >>>> ? ? ? ? ? ? ?{"Cannot lookup",schema}}} >>>> >>>> Ok, I figured it needs the schema to be backed up as well, so I then >>>> build a checkpoint with the schema and the set of tables I want to >>>> backup. But when I run that I get a different error: >>>> >>>> ? ? {error,{"Cannot install fallback", >>>> ? ? ? ? ? ? ?{already_exists,'hndev@REDACTED'}}} >>>> >>>> So I am at a bit of a loss. >>>> >>>> Does install_fallback only work with whole table backups? >>>> >>>> Gordon >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> From dgud@REDACTED Fri Jun 8 13:03:43 2012 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 8 Jun 2012 13:03:43 +0200 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 12:52 PM, Gordon Guthrie wrote: > Is it possible to selectively restore tables at all? Only via mnesia:restore(..) /Dan From msegalis@REDACTED Fri Jun 8 13:06:03 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 8 Jun 2012 13:06:03 +0200 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: <-5124868496751168372@unknownmsgid> References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> <-5124868496751168372@unknownmsgid> Message-ID: <6F18A906-B1D3-4AA8-BC2A-2943C2D7D3AE@gmail.com> Hi ! As I explained answering Angela, Here's my idea for the connection protocol I'm thinking about. 1) The client should connect WITH SSL to a gateway (With Certificate and strong Cipher) 2) Client and Server will negotiate a key together along with a sessionID (not sure we need Diffie-Hellman since it is a secure connection) 3) The sessionID + key will be stored in a cassandra database (With a TimeToLive of 2min or less) 4) The server will tell to which node the client should connect 5) Client disconnects from gateway 6) Client connects to the node specified by the gateway, and gives the SessionID to the node (in clear) 7) The Server node retrieve the Key from the cassandra db 8) If it exist, it sends an accept to the client encrypted, if not, it disconnects the client and we are back to square 1. 9) Both client & server can communicate over AES using the key they now both have. in part 4, the gateway, a trusted SSL server will give the ip of the node the client should connect (encoded of course). This will make the client able to trust the server. in part 8, the node will accept the client by sending him a specific message, that only a trusted server could have sent. This will make the client able to trust the server. I'm confident this protocol is secure, and will allow me to have solid encryption while having a good ratio memory/client. But I might be wrong? Thanks ! Le 7 juin 2012 ? 14:29, dmitry kolesnikov a ?crit : > Hello, > > I guess you can use crypto:dh_compute_key for DH key exchange procedure. > > Your application cannot trust server w/o certificates. Therefore > number of attacks are possible. If you are fine with all related > threats then yes this is good option to save memory. > > On another hand, if you need to use multiple transport e.g. Tcp + udp > then it is good as well. > > Btw, C ssl API allows to perform SSL:read/write directly to memory. It > would be cool if such feature is available in Erlang. I suppose it > might help to reduce memory footprint for some applications > > Best Regards, > Dmitry >-|-|-*> > > > On 7.6.2012, at 13.35, Morgan Segalis wrote: > >> Hi everyone, >> >> As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. >> >> I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. >> Using SSL with this kinda feels like using a bazooka for smashing a fly. >> >> The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. >> >> Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) >> >> I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ >> >> Thanks for your help. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From gordon@REDACTED Fri Jun 8 13:11:47 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 8 Jun 2012 12:11:47 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: The circle completes... I tried this first and hit the problem mentioned in the documentation: `If the database is huge, it may not be possible to restore it online. In such cases, the old database must be restored by installing a fallback and then restart.` Basically the system runs out of memory on restore even when the tables are all swapped to be on disk only... So then I tried to get install_fallback working :( So essentially there is no way to selectively restore large tables? Gordon On 8 June 2012 12:03, Dan Gudmundsson wrote: > On Fri, Jun 8, 2012 at 12:52 PM, Gordon Guthrie wrote: >> Is it possible to selectively restore tables at all? > > Only via mnesia:restore(..) > > /Dan From dgud@REDACTED Fri Jun 8 13:30:25 2012 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 8 Jun 2012 13:30:25 +0200 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 1:11 PM, Gordon Guthrie wrote: > So essentially there is no way to selectively restore large tables? Nope, you will have to restart from a complete system backup. Or you can write if you do something dirty, like take the system offline, down to single node, traverse the backup and and lot of dirty_writes. From gordon@REDACTED Fri Jun 8 13:33:35 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 8 Jun 2012 12:33:35 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: Dan I think what I will do is create a second node on the target system, restore to that, then replicate/copy the tables across... Then I can take down the restored db node. Or mebbies your way is better, simply pass in a fun to the backup traverse to write out the records in tables that I want to bring over... Gordon On 8 June 2012 12:30, Dan Gudmundsson wrote: > On Fri, Jun 8, 2012 at 1:11 PM, Gordon Guthrie wrote: >> So essentially there is no way to selectively restore large tables? > > Nope, you will have to restart from a complete system backup. > > Or you can write if you do something dirty, like take the system > offline, down to single node, > traverse the backup and and lot of dirty_writes. From chandrashekhar.mullaparthi@REDACTED Fri Jun 8 13:54:25 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 8 Jun 2012 12:54:25 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On 8 June 2012 12:33, Gordon Guthrie wrote: > > Or mebbies your way is better, simply pass in a fun to the backup > traverse to write out the records in tables that I want to bring > over... > FWIW, this is what we do on very large tables. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jun 8 14:04:59 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 14:04:59 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> Message-ID: <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> On 8 Jun 2012, at 12:46, Michael Turner wrote: > On Fri, Jun 8, 2012 at 6:55 PM, Ulf Wiger wrote: >> >> On 8 Jun 2012, at 09:19, Michael Turner wrote: >> > Ulf, at no point in that particular paragraph did I mention you by > name. We didn't exactly bury the hatchet, so it's not like you're > digging it up. But you are picking it up. No, actually, I'm not. I complemented you for your tone and interesting content in that thread, but did have some comments on this particular statement. If I mistakenly thought I detected *you* picking up the hatchet, and it was not so, I apologize. > Nothing I wrote up there about getting seq_trace documentation to > mention that it implement Lamport clocks could reasonably be construed > as your own idea, particularly since you initially seemed pretty hazy > in your response my statement that seq_trace implements Lamport > clocks. I suggest we don't revisit that discussion, esp. in this thread. Robert's comment, I think, described it well: [Robert:] > I really think there are two issues here: what they are, or what > they are commonly known as; and the generating idea behind them. I'm sure there is much more that could be said to expand on that, and I could give some other examples of where mixing up purpose and implementation in the crafting of API and docs can lead to problems (the 'ets' module, for one), but I don't think I will, since you and I seem to differ completely on what the purpose of seq_trace is, what the word 'tracing' (and more so 'sequence trace') actually implies, so I think we would just continue to talk past each other. >> In this particular case, the most relevant passage of our >> long discussion should be Gustav Simonsson (who *does* >> represent OTP) saying that your patch was welcome: > > If you mean specifically the idea of a patch mentioning that seq_trace > implements Lamport clocks, I have yet to see that from Gustav. That quote came as a response to me saying: [Ulf:] > In this case, I'm pretty sure [a reference to Lamport clocks] > could be worked in without harming readability, but it is of course > perfectly possible to use seq_trace without understanding, or even > being aware of the existence of, Lamport clocks. > > The OTP team is known to accept patches to the documentation, > so please feel free to contribute to a more helpful way to describe > the tracing support. I'm sure it would be universally appreciated. If Gustav meant something other than the obvious interpretation that you were most welcome to insert an advanced example describing how seq_trace implements Lamport clocks, and how that knowledge can help you make good use of them, he should hasten to clarify that. > And I have it (second-hand from you) that he has himself > thought of a good place for this mention, in the seq_trace > documentation. To which I replied: if he's got a good idea > for that, why doesn't he just do it? No, first-hand from Gustav, as he replied in the thread, cc to the list. Also, his comment made it clear that he invited you to expand on the topic - not just insert a one-liner. > As far as I can tell, NOBODY at Ericsson has, in writing, said > directly, "We need to fix that." Until then ?. What I was trying to convey with my last post was that you have actually received more indication that your patch will be accepted than most people who have submitted documentation patches have. If you want even more confirmation before doing anything, I understand that. That was another thing I tried to convey. Nobody expects you to put in the time and effort to improve Ericsson's documentation. I have decided to not make a number of patches, since I didn't have the time, and didn't feel confident that they would be accepted. But that is your decision. It was suggested in another comment that you have spent more time debating the issue than it would have taken you to prepare the patch and submit it. While that is probably true, well, so have I. Meta-discussions are also important, perhaps sometimes even more important than what the docs actually say. Really, Michael, it would be great if you could write up a good advanced example for seq_trace, with references to relevant research etc. Whether you do it as a blog post or as a documentation patch, really doesn't matter much. You can do both, even. Sequence trace deserves more use, and any tutorials or additions to the docs that can help that are a great addition. > And I've already responded to that: "improvement" is in the eye (and > to some extent, the imagination) of the beholder. So long as it's part > of some internal Ericsson mythology that seq_trace only reiterates > Lamport clocks as a technique invented independently (and > concurrently, if not earlier) in "forlopp tracing", the response can > always be: "we invented that, probably before Lamport thought of it, > we don't have to credit anybody else, and some of our customers still > think we invented it anyway, so leaving it unmentioned is what works > best for Ericsson AS A BUSINESS." To me, this is a big strawman you're building, but let's try not to revisit that. For those interested, the debate, and more, is in the list archive. > I hope it doesn't have to go to lawyers for approval. Nothing ever goes to the lawyers for approval, since the only answer you ever get is "no". ;-) > I mean, whether > forlopp tracing, at any point in its evolution, did or did not include > Lamport clocks ... is that actually something you can't reveal under > NDA? The mind boggles. Again, an amazing misrepresentation of what I (or someone else?) have said. What I did try to explain was that there is only so much I can tell, as a former employee, of what went on at Ericsson. Even when I worked there, I was not allowed to reveal internal project or product details without approval. When I worked there, I could of course have picked up the phone and asked some old-timers what the real story was. Now, I can't. Why you find this either disturbing or strange is beyond me. Besides, it has no bearing on whether the seq_trace docs can say that they implement Lamport clocks - Kenneth has already said they do. If you do write a patch, and it's rejected, it cannot possibly be for that reason. > I can think of reasons why a corporation might be cagey on > an issue like this. None of them are terribly complimentary > to the corporation. I suggested a reason earlier in this mail: the people at the OTP team have lots on their plate and, right or not, don't see this as a terribly important issue. Chances are, they are even too busy to follow this thread in any detail. (Come to think of it, that goes for me too. Whatever more there is to say, I have exceeded my time quota, and I'm sure many on the list would like to see this discussion end.) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From dafgar@REDACTED Fri Jun 8 15:35:01 2012 From: dafgar@REDACTED (Sergei Fomin) Date: Fri, 08 Jun 2012 17:35:01 +0400 Subject: [erlang-questions] enotconn from gen_tcp socket Message-ID: <4FD1FF85.7010500@yandex-team.ru> Hello. There is a server which gen_tcp:accept new connections and immediately logs inet:peername. But still, "Accepted {error,enotconn}" can be seen in the log files. One possible reason of such error is described in http://erlang.org/pipermail/erlang-questions/2012-May/066802.html , i.e. a client does 3-way handshake and sends RST packet immediately. Though, this scenario seems uncommon. Wonder, what other causes can lead to enotconn on just opened socket? -- Sergei Fomin From attila.r.nohl@REDACTED Fri Jun 8 15:50:58 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 8 Jun 2012 15:50:58 +0200 Subject: [erlang-questions] enotconn from gen_tcp socket In-Reply-To: <4FD1FF85.7010500@yandex-team.ru> References: <4FD1FF85.7010500@yandex-team.ru> Message-ID: 2012/6/8 Sergei Fomin : > Hello. > > There is a server which gen_tcp:accept new connections and immediately logs > inet:peername. But still, "Accepted {error,enotconn}" can be seen in the log > files. One possible reason of such error is described in > http://erlang.org/pipermail/erlang-questions/2012-May/066802.html , i.e. a > client does 3-way handshake and sends RST packet immediately. Though, this > scenario seems uncommon. > > Wonder, what other causes can lead to enotconn on just opened socket? I think it doesn't have to be an RST, a normal connection closing would also lead to this error. I've seen a very similar error a couple of months ago when the OTP SFTP client did something similar as you, i.e. it had a {ok, PeerAddr} = inet:peername(Socket), call soon after the Socket was created and sometimes this crashed with a badmatch. The SFTP client has been fixed in R15B01, so there's a case and error handling instead of the matching. From michael.eugene.turner@REDACTED Fri Jun 8 17:05:50 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 00:05:50 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <75896D09-B408-49A6-A764-C628F3A4F320@feuerlabs.com> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <75896D09-B408-49A6-A764-C628F3A4F320@feuerlabs.com> Message-ID: On Fri, Jun 8, 2012 at 7:36 PM, Ulf Wiger wrote: > > One of my main demands on any wiki-style documentation > for OTP would be that it supports baselining, and online > and offline browsing of older baselines. http://en.wikipedia.org/wiki/Wikipedia:Database_download > The current OTP documentation, for all its flaws, is at least > version-consistent. I can check out and build an old version > of OTP and the docs will be the version that was set in stable > revision together with the code. MediaWiki makes it fairly easy to establish (perhaps with a single template invocation in the article footer) a way to see every previous "official release" version of a given article. Here, for example http://en.wikipedia.org/w/index.php?title=Erlang_(programming_language)&oldid=465840644 is the version of the Wikipedia article about Erlang from particular instant in time. I picked it out of the History page for the article. http://en.wikipedia.org/w/index.php?title=Erlang_(programming_language)&action=history See the links in the tabs of any Wikipedia article, if you've never poked at them before. The tab called "edit" is particularly powerful, I find. > I won't say that it *can't* be done with wikis, but even such > wikis as the Github wiki, which are based on a strong VCS, > normally don't make this easy. "Even"? The Github wiki strikes me as kind of crude, to be honest. Probably because it's based on a "strong VCS", which isn't necessarily the the right substrate. (But good enough for Github purposes. I admit I haven't looked very closely at it yet.) > In the case of the github wiki, you could clone it like any other > git repository, label it, and do all sorts of things with it, but > the only access you have to older versions of the pages online > is through the diffs in the history, and viewing github-flavored > Markdown off-line is no fun - or at least requires quite a bit of > work for rendered viewing. Such a solution will not be much > of an improvement over the current situation. "Improvement" is in the eyes of the beholder. To me, just being able to fix a grammatical error in under a minute, and have the rest of the world see it shortly after I hit "submit", is a vast improvement over the status quo. > If there is a wiki out there that can do this, I'd be thrilled to > hear about it. I don't see the value-added in cloning an entire wiki, except in the standard model of revision flow where everything leads to a release date. Wikis are: fix and release a single page, right now. Right now there's a page that makes Erlang/OTP look like it's plagiarizing Leslie Lamport. I want to fix it and release it right now. If you want release dates for the wiki, you can warn people of the release date, start locking down pages and even whole namespaces, and letting in only a few emergency edits as required, then declare what a snapshot made on the release day the "version" of the wiki for a given *software* release. Right after the release, you can start loosening up again, not least so that individual wiki articles can reflect fresh reports of the more important new bugs in the new releases of the modules they document. We all want the perfect, zero-cost system, right now. What you can get in reality: good, cheap, soon - pick any two. Admittedly, "good" is in the eye of the beholder. My "good" might not be yours. But beware premature judgments of "bad". I'm hearing a lot of "wikis are trash", "wikis can't do this," "wikis can't do that." Not to get all Mitt Romney on you, but wikis are people. Including people with designated powers and responsibilities. Wikipedia can be edited by anybody, and tries to cover everything. For an Erlang/OTP wiki that's totally inappropriate. You might instead parcel out editing rights only to a few dozen people outside Ericsson, and limit the range of topics to articles on released software plus methodology outlines like OTP Design Principles. MediaWiki doesn't have the most advanced access control system in the world. But it can do that. So can other wiki packages. The older ones still in use have been evolving for about 17 years now, since Ward Cunningham put up the Portland Pattern Repository http://en.wikipedia.org/wiki/Portland_Pattern_Repository in 1995, in the first wiki. So if you say, with very little experience editing and managing wikis, "they can't do X", there's a very good chance you're wrong. -michael turner > On 8 Jun 2012, at 00:08, Richard O'Keefe wrote: > >> Remember, the issue is not "disconnect". ?The problem is >> - people selected for programming skill, not writing skill (or even >> ? the ability to notice the grammatical errors and typos that bother >> ? you and me), >> - who know too much about the system (if they didn't, would they >> ? have the confidence to edit the documentation?) to understand what >> ? most needs to be documented. >> Wikis do not solve this problem, they exacerbate it. > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From Sergey_Zhemzhitsky@REDACTED Fri Jun 8 17:15:53 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Fri, 8 Jun 2012 15:15:53 +0000 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CC41DE@exch-mbx2.msk.trd.ru> Hello Richard, I'm very interested in the information and answers to the following points: 1. If there is ei API why there are no samples of how to use it here http://www.erlang.org/doc/tutorial/users_guide.html? 2. Why samples here http://www.erlang.org/doc/tutorial/erl_interface.html use ETERM API although the page is about erl interface (ei API?)? 3. I'd like to see more complex samples in the official docs, for examples how to parse a list of tagged tuples (proplist, etc.). 4. I'd like to know how to use ei API correctly. For example 1) ei_get_type(const char *buf, const int *index, int *type, int *size) decodes type of the term. There is no documentation about possible values of the *type* parameter except for ei.h file. 2) if ei_get_type returns ERL_INTEGER_EXT as a third parameter which of the following functions I have to use ei_decode_long, ei_decode_ulong, ei_decode_longlong, ei_decode_ulonglong to decode the term? 5. I'm also interested in what type of API is legal and recommended to use in c nodes, ports, port drivers. Is it possible to use ei, eterm, etc.? Is there any difference between usage of different api in c nodes, ports, port drivers? 6. I'd like see among the docs for memory, term, etc.-allocating functions information about corresponding disposing functions and whether it's necessary to call them or not. Best Regards, Sergey -----Original Message----- From: Richard O'Keefe [mailto:ok@REDACTED] Sent: Friday, June 08, 2012 2:43 AM To: Zhemzhitsky Sergey Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? On 7/06/2012, at 7:19 PM, Zhemzhitsky Sergey wrote: > Hi Richard, > > Thanks a lot for answer. It is rather helpful. > >>> Why was it hard to see in the documentation? > As I'm kinda a newbie at erlang I understand your indignation. There was no indignation there. There has been a lot of discussion here about documentation and what to do to improve it, and I really wanted to know what went wrong here. Would it be sufficient to have a document "Exchanging data between Erlang and C : Overview" with links between it and the libraries? What would you like to see in examples? _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From michael.eugene.turner@REDACTED Fri Jun 8 17:51:09 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 00:51:09 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> Message-ID: On Fri, Jun 8, 2012 at 9:04 PM, Ulf Wiger wrote: > > On 8 Jun 2012, at 12:46, Michael Turner wrote: > >> On Fri, Jun 8, 2012 at 6:55 PM, Ulf Wiger wrote: >>> >>> On 8 Jun 2012, at 09:19, Michael Turner wrote: >> Nothing I wrote up there about getting seq_trace documentation to >> mention that it implement Lamport clocks could reasonably be construed >> as your own idea, particularly since you initially seemed pretty hazy >> in your response my statement that seq_trace implements Lamport >> clocks. > > I suggest we don't revisit that discussion, esp. in this thread. And then what do you do? You start revisiting that discussion. > Robert's comment, I think, described it well: > > [Robert:] >> I really think there are two issues here: what they are, or what >> they are commonly known as; and the generating idea behind them. "Generating idea" isn't a well-defined term. So I disagree: this doesn't really say much of anything, because there's apparently no way (under your NDAs) to distinguish the supposed "generating idea" from what Lamport clocks are "commonly known as": Lamport clocks. Lamport clocks are integral to the seq_trace API. The wording of the documentation makes that very, very clear. The only "beta" caveat is about possible *interface* changes. End of story. > I'm sure there is much more that could be said to expand on > that, and I could give some other examples of where mixing > up purpose and implementation in the crafting of API and > docs can lead to problems (the 'ets' module, for one), but > I don't think I will, since you and I seem to differ completely > on what the purpose of seq_trace is, what the word 'tracing' > (and more so 'sequence trace') actually implies, so I think > we would just continue to talk past each other. You say we "differ completely", but you base your differences on things you say that I can't know because you can't tell me: customer requirements generated under NDA. This makes no sense. If seq_trace actually directly reflects customer requirements, I'd say it's pretty obvious that the customer wanted (among other things) Lamport clocks. Perhaps they expressed them as, "we have clock skew problems in our distributed system, and we need some way to correctly sequence our traces in spite of that." Lamport clocks are a simple, classical solution to that problem. If that customer (and others) are using that solution, all YOUR talk of possible alternative implementations is moot: you'd be introducing an incompatibility. You might opine otherwise, but since you can't tell me the difference between the customer requirement and what the seq_trace documentation says (two things that, at times, you've equated, more confusing still), you might as well be talking about some divine inspiration not communicable to mere mortals like me. > If Gustav meant something other than the obvious interpretation > that you were most welcome to insert an advanced example > describing how seq_trace implements Lamport clocks, and > how that knowledge can help you make good use of them, he > should hasten to clarify that. That's your idea of "the most obvious interpretation"? I don't know how I could have made myself clearer: I've REPEATEDLY suggested adding four words - "seq_trace implements Lamport clocks." There's no need to "insert an advanced example describing how seq_trace implements Lamport clocks." The documentation ALREADY DESCRIBES HOW IT IMPLEMENTS LAMPORT CLOCKS. I want to see credit where it's due, because (a) that's the right thing to do, and (b) it gives people who already know what Lamport clocks are the advantage of instant familiarity. Which I've said at least twice before. >> And I have it (second-hand from you) that he has himself >> thought of a good place for this mention, in the seq_trace >> documentation. To which I replied: if he's got a good idea >> for that, why doesn't he just do it? > > No, first-hand from Gustav, as he replied in the thread, > cc to the list. A bland and vague solicitation for "improvements" still leaves room for interpretation of the term "improvement." (As I've already said probably three times now.) If giving credit to Lamport is finally judged to not be an improvement, then Gustav's solicitation is NOT a specific, explicit request for that crediting of Lamport. > Also, his comment made it clear that he invited > you to expand on the topic - not just insert a one-liner. First things first: stop making the seq_trace documentation look like plagiarism. In four words. Gustav replied with dozens of words when he himself could have just added four to the documentation. >> As far as I can tell, NOBODY at Ericsson has, in writing, said >> directly, "We need to fix that." Until then ?. > > What I was trying to convey with my last post was that you > have actually received more indication that your patch > will be accepted than most people who have submitted > documentation patches have. No, what I've received is a lot of words with no definite commitment. What is so hard about being direct and saying, "Yes, the documentation should say that seq_trace implements Lamport clocks"? [snip long passage] The issues here are so basic. -michael turner From ulf@REDACTED Fri Jun 8 17:38:38 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 17:38:38 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <75896D09-B408-49A6-A764-C628F3A4F320@feuerlabs.com> Message-ID: <283C741C-7263-4B23-85DB-1D7F883C4918@feuerlabs.com> On 8 Jun 2012, at 17:05, Michael Turner wrote: > So if you say, with very little experience editing and managing wikis, > "they can't do X", there's a very good chance you're wrong. ?sure, as with everything. And indeed I explicitly said "I won't say that it *can't* be done with wikis, ?" As it happens, I do have experience editing and managing wikis, and while I was perfectly able to view the history of individual pages, and could also bring up an older revision of a particular page, I found the lack of *baseline support* a distinct disadvantage for some uses. For example, when we were doing iterative development, and using wiki pages to reflect the architecture, usage and installation instructions, as well as the state of the project, the wikis usually ended up reflecting only the most recent state - anything else was simply too hard. We could reflect milestones across modules in the code and in the issue tracker, but the wiki pages kept floating on top. That's sometimes fine, but in the case of OTP, I personally want to be able to browse around in a consistent set of pages for the particular release I'm using. The wiki used in that particular case was Redmine's wiki. I have experienced use of other wikis, e.g. MediaWiki, in past development projects. They were great for keeping up with peripheral stuff - how to set up test environments, useful tricks, etc. I think OTP could use the wiki on github more. The reason why I brought up the github wiki was that it obviously should be able to do this, since the underlying storage is a git repository, and github offers the ability to browse a tagged version of source code repositories. If they did the same for the wiki, I'd have my baseline support. (I document my own code with edown, which lets github render edoc generated as markdown. It has the advantage that I can browse documentation online that's consistent with the code in the same view - not at all revolutionary, of course, but still convenient.) There used to be a wiki for Erlang: Trapexit. It is currently down. Some people found it extremely useful, and others didn't. Some, who would have been able to correct mistakes and add useful insight, didn't want to create an account, log in and do the edits, so we had to contact them, ask for their opinion, and try to relate their knowledge on the wiki and forums. Others abused the service and had to be locked out. ESL has said that they are working on a new community site. Hopefully, we will all get to see it soon. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From michael.eugene.turner@REDACTED Fri Jun 8 18:19:34 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 01:19:34 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <283C741C-7263-4B23-85DB-1D7F883C4918@feuerlabs.com> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <75896D09-B408-49A6-A764-C628F3A4F320@feuerlabs.com> <283C741C-7263-4B23-85DB-1D7F883C4918@feuerlabs.com> Message-ID: On Sat, Jun 9, 2012 at 12:38 AM, Ulf Wiger wrote: > > On 8 Jun 2012, at 17:05, Michael Turner wrote: > [snip] > There used to be a wiki for Erlang: Trapexit. It is currently down. > Some people found it extremely useful, and others didn't. Some, > who would have been able to correct mistakes and add useful > insight, didn't want to create an account, log in and do the edits, ... Or couldn't. And were entirely blameless for that. I should know. I don't really care who I offend with the following comment: if TrapExit is dead now, it killed itself. I wanted to fix things on it immediately. I tried to create an account. I couldn't figure out how. There was a sign-up page with a form with a header that said something like, "If you want to create a new account, start here." Followed by some form fields, which I filled out -- but without getting a new account notification. Here's what the problem was: Somebody had decided that it was totally cool webby design to not have links indicated graphically. You had to move your mouse around the text and IF it happened to intersect linked text, *then* the underscoring (or whatever) would make it evident there was a link. And somebody had made the above "here" in the vague instructions the link to the *real* way to make an account. But when you issue a vague "here's how" command to the reader, with some form-fields to fill in immediately following, you're saying that "start here" means "fill out the form fields below," not "click on this link which doesn't even look like a link unless you are lucky enough happen to move the mouse across it." Those who were lucky enough wouldn't remember having any trouble signing up. Those who weren't lucky enough would have had nothing but trouble. And who needs trouble? Dumb, dumb, dumb. It took me forever to convince the one web admin contact I could get a response from that this was confusing as hell. Actually, I'm not sure he ever quite got it. People: go with the defaults. Follow basic web usability guidelines as you start departing from the installation defaults. Familiarize yourself with those guidelines if you haven't yet. Most of those guidelines will tell you that linking the word "here" is at best very informal style. Most of them will also tell you that links should be obvious. This signup-page screwup was far from the only confusing thing about TrapExit, but in terms of bringing in new users to edit the wiki into shape, it was certainly the worst. Who knows how many people tried several times, as I did, to make a new account, only to say, "screw it, something's broken here, these people don't know what they're doing, it's not worth the effort." And moved on. As I remember, I finally figured it out by viewing the HTML source, after an admin insisted there was absolutely nothing wrong with the page -- he'd just tested it himself. Laugh if you like, but these kinds of snags can add up to more frustration than any volunteer wants to deal with. -michael turner >> So if you say, with very little experience editing and managing wikis, >> "they can't do X", there's a very good chance you're wrong. > > ?sure, as with everything. And indeed I explicitly said "I won't > say that it *can't* be done with wikis, ?" > > As it happens, I do have experience editing and managing wikis, > and while I was perfectly able to view the history of individual pages, > and could also bring up an older revision of a particular page, I found > the lack of *baseline support* a distinct disadvantage for some uses. > > For example, when we were doing iterative development, and using > wiki pages to reflect the architecture, usage and installation > instructions, as well as the state of the project, the wikis usually > ended up reflecting only the most recent state - anything else was > simply too hard. We could reflect milestones across modules in > the code and in the issue tracker, but the wiki pages kept floating > on top. That's sometimes fine, but in the case of OTP, I personally > want to be able to browse around in a consistent set of pages for > the particular release I'm using. > > The wiki used in that particular case was Redmine's wiki. > I have experienced use of other wikis, e.g. MediaWiki, in > past development projects. They were great for keeping up > with peripheral stuff - how to set up test environments, useful > tricks, etc. I think OTP could use the wiki on github more. > > The reason why I brought up the github wiki was that it obviously > should be able to do this, since the underlying storage is a git > repository, and github offers the ability to browse a tagged > version of source code repositories. If they did the same for the > wiki, I'd have my baseline support. > > (I document my own code with edown, which lets github render > edoc generated as markdown. It has the advantage that I can > browse documentation online that's consistent with the code > in the same view - not at all revolutionary, of course, but still > convenient.) > > There used to be a wiki for Erlang: Trapexit. It is currently down. > Some people found it extremely useful, and others didn't. Some, > who would have been able to correct mistakes and add useful > insight, didn't want to create an account, log in and do the edits, > so we had to contact them, ask for their opinion, and try to relate > their knowledge on the wiki and forums. Others abused the > service and had to be locked out. > > ESL has said that they are working on a new community site. > Hopefully, we will all get to see it soon. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From heinz@REDACTED Fri Jun 8 18:25:32 2012 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 8 Jun 2012 18:25:32 +0200 Subject: [erlang-questions] Optimization for the ++ operator In-Reply-To: <4FC8BEF5.3000006@erlang.org> References: <460492439358073642@unknownmsgid> <4FC8BEF5.3000006@erlang.org> Message-ID: <00D8B979-9974-435A-872B-375DEBBCC884@licenser.net> The very unlikely is the important part here ;) also imagine mars position causes it to catch a astroid in it's gravitation field and slingshots it right into your computer - now that would screw your benchmark results! -- Heinz N. Gies heinz@REDACTED http://licenser.net On Jun 1, 2012, at 15:09, Gustav Simonsson wrote: > On 2012-06-01 12:07, Bj?rn-Egil Dahlberg wrote: >> >> >> 1 jun 2012 kl. 10:46 skrev "Jos? Valim" : >> >>> The '++' operator does not verify that the second argument is a list. >>> Therefore, the compiler will rewrite the first expression to simply: >>> >>> [$F,$o,$o|Bar] >>> >>> Yes, thanks. For some weird reason I thought Erlang restricted the right side to be a list in such cases. >>> >>> Have you made sure that you run each test in a newly spawned process? >>> Do you run the test for long enough time? >>> >>> Running in a newly spawned process did the trick, the results are now consistent. >>> I was already running it 1000 times, here is the code (without using spawn): >>> >>> https://www.refheap.com/paste/2954 >>> >>> I assume having a "clean" heap (reducing the chance of garbage collection) is the reason why it is a good idea to run benchmarks in a newly spawned process? >> >> If you want to vary one parameter in a test, make sure all other parameters stays the same otherwise you have no idea what you are measuring! >> >> Code loading, heaps, position of mars .. Everything has to be taken into account. > > The effect of Mars magnetic field on that of Earth, even if Earth's aphelion and the perihelion of Mars were to coincide and the interplanetary distance were to be at its theoretical minimum, is negligible compared to the normal variations in Earth's magnetic field. > > Therefore it's very unlikely that the relative position of Mars would affect the probability of a memory bitflip occurring due to cosmic radiation and thus affecting the benchmark. > > Regards, > Gustav Simonsson > Erlang/OTP team > >> >> >>> >>> Thanks Bj?rn! >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Fri Jun 8 18:34:11 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Fri, 8 Jun 2012 17:34:11 +0100 Subject: [erlang-questions] More a comment that a question Message-ID: Hi folks I've been coding/working with Erlang (and Chicago Boss) for a while now, and I've recently had to do some work in Python. I find myself looking at some code I'm writing that looks like this: html = html + "some markup" and getting nervous as I know this is a "naughty" that Erlang would not let me away with. If only there was a "make everything immutable" switch for Python! ;-) Paul. -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From ulf@REDACTED Fri Jun 8 18:50:16 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 18:50:16 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> Message-ID: <6DE6B5C1-7F7F-41C7-BB78-1554B834741A@feuerlabs.com> On 8 Jun 2012, at 17:51, Michael Turner wrote: > Perhaps they expressed them as, "we have clock skew problems in our > distributed system, and we need some way to correctly sequence our > traces in spite of that." Lamport clocks are a simple, classical > solution to that problem. No (or, if they asked for that, I'm not aware of it. The OTP team can correct me if I'm wrong, but that was not the impetus to seq_trace.) I will try one more time. Tracing a sequence, as in "our system handles a thousand call setups per second, and if we turn on a trace on all of them, we will not only learn nothing - we will kill the system. We need a way to trigger trace output on *one* session in the midst, and have that trigger 'contaminate' processes as the request is passed around in the system, and then, obviously, turn off, so we get only what we asked for, and nothing more". Effectively, automatically selecting trace output so that it looks as if we traced everything and ran only one single call through the system (which is what most people resort to). If we call it "session trace", does that make it clearer? Obviously ordering (sequencing) is *one* part of the problem, for which Lamport clocks are a great solution. But the part where tokens act as "probes" whizzing through the system activating trace output selectively, is part of trying to reduce the amount of trace data generated. A large part of the complexity of the tracing subsystem in Erlang comes from the need of the user to be able to define, ad-hoc, just the right filters so that one can get useful trace output without killing the system. While you can accomplish a "session-specific" trace just using pattern-matching on function calls, this quickly becomes unwieldy. Usually, you just want to enable a wide trace, to include all important calls, but *only* for the one session you decide to trace - not for the perhaps hundreds or thousands of other sessions that may touch the same process. For this, the trace patterns in the tracing subsystem can match on function call parameters and dynamically set and clear trace tokens. Rather than 'independent' of the standard tracing, I would say that sequence tracing is 'orthogonal'. The standard trace is great for tracing on a small set of functions or modules, or showing all activity in one or a few processes. But in large systems under commercial load, doing any kind of tracing is really scary. Some Erlang old-timers are known to explain how they took down entire mobile networks by carelessly setting up a wrong trace. This is my take on this area. In most practical uses, the ordering one gets from timestamps is perfectly fine (for tracing - *not* if one really wants to ensure that the trace reflects the exact causal order. The hardest problem in the scenario I describe is avoiding killing the node or at least getting so much trace data that any analysis of it becomes prohibitively hard. That seq_trace makes use of Lamport clocks is an added bonus, and at least the tools in OTP, like e.g. ttb and et, should take advantage of this whenever possible. As it is now, they don't (or I missed it, which is also a possibility). Is this all clear from the seq_trace manual? No. It's easy to get the idea that it was created for a different purpose entirely, and even people who seek it out wanting to do exactly what I describe above, tend to turn away frustrated. But it's a hard problem to solve, and I'm not saying that the current support is sub-par. If I had a clear idea of how to improve it, I would have submitted patches long ago. What I did to try to improve things was work to get ttb's support for saving useful trace patterns and replaying them later, more stable and better documented. This doesn't relate to seq_trace as much as to the ability to manage trace tokens through trace patterns. It's badly named. It should definitely not speak of "sequential tracing". I'm pretty sure "sequence" came out of "f?rlopp", which basically means "a sequence of events". The function of a "forlopp" in AXE is that it the smallest single unit of failure. It can be aborted and re-run, like a transaction. Does all this preclude adding a reference to lamport clocks in the seq_trace manual? Obviously not. If it bugs you so much, write a patch and submit it. I agree that it will cost OTP as much to vet your patch as it would for them to put the sentence in themselves, but if it's not a priority to them, and it is to you, you know what to do. But it sounds as if you are putting seq_trace to good use, in a way that is different from what I describe above. Boiling this down to an example would be a great contribution. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From bchesneau@REDACTED Fri Jun 8 18:56:22 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 8 Jun 2012 18:56:22 +0200 Subject: [erlang-questions] [ANN] mochicow 0.1 Message-ID: Hi all, I just released the version 1 of mochicow, a mochiweb adapter for cowboy. Use your mochiweb code d with cowboy. There are 2 ways to use mochicow: - as a cowboy protocol: It will use the socket acceptor pool of cowboy instead of the mochiweb one. - as a protocol upgrade. Like websockets you can upgrade a cowboy handler to use a mochiweb loop. It allows you to use both your new code with cowboy and old code with mochiweb. See the examples in the examples for the usage. The source is on github: https://github.com/benoitc/mochicow Enjoy :) - beno?t From bob@REDACTED Fri Jun 8 19:21:57 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 8 Jun 2012 10:21:57 -0700 Subject: [erlang-questions] More a comment that a question In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry wrote: > > I've been coding/working with Erlang (and Chicago Boss) for a while > now, and I've recently had to do some work in Python. I find myself > looking at some code I'm writing that looks like this: > > html = html + "some markup" > > and getting nervous as I know this is a "naughty" that Erlang would > not let me away with. If only there was a "make everything > immutable" switch for Python! ;-) Strictly speaking, there's no mutation at all here, because str in Python *are* immutable. What's happening is that the variable html is rebound to the result. This is roughly equivalent to something like this in Erlang [1]: HTML1 = <>. You really wouldn't want to have to write Python in single assignment form, especially without tail call optimization. Python IO would benefit from something like Erlang's iolist, but for use cases like the above people tend to append to a list and then join it with the empty string, or use an instance of cStringIO.StringIO. [1] Some versions of CPython do have some hacks that can detect this x += s where x is a string with a reference count of 1 and strategically do these updates in-place with over-allocation... so it's possible that it's not quite as bad as you think it is (amortized). -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Jun 8 19:25:50 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 8 Jun 2012 10:25:50 -0700 Subject: [erlang-questions] [ANN] mochicow 0.1 In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 9:56 AM, Benoit Chesneau wrote: > > I just released the version 1 of mochicow, a mochiweb adapter for > cowboy. Use your mochiweb code d with cowboy. > > There are 2 ways to use mochicow: > > - as a cowboy protocol: It will use the socket acceptor pool of cowboy > instead of the mochiweb one. > - as a protocol upgrade. Like websockets you can upgrade a cowboy > handler to use a mochiweb loop. It allows you to use both your new > code with cowboy and old code with mochiweb. > > See the examples in the examples for the usage. > > The source is on github: > > https://github.com/benoitc/mochicow > > Enjoy :) > This is fantastic, I had considered doing this myself but I don't have any legacy code to worry about at the moment :) -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Fri Jun 8 19:37:58 2012 From: eric@REDACTED (Eric Moritz) Date: Fri, 8 Jun 2012 13:37:58 -0400 Subject: [erlang-questions] More a comment that a question In-Reply-To: References: Message-ID: Where the danger comes in is mutating object, lists and dicts that other threads or functions have references to. It is possible to have a reference to a dict in a module or up the call stack that gets changed in an unexpected way and things crash. Erlang has made me a very paranoid Python developer. On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito wrote: > On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry wrote: > >> >> I've been coding/working with Erlang (and Chicago Boss) for a while >> now, and I've recently had to do some work in Python. I find myself >> looking at some code I'm writing that looks like this: >> >> html = html + "some markup" >> >> and getting nervous as I know this is a "naughty" that Erlang would >> not let me away with. If only there was a "make everything >> immutable" switch for Python! ;-) > > > Strictly speaking, there's no mutation at all here, because str in Python > *are* immutable. What's happening is that the variable html is rebound to > the result. This is roughly equivalent to something like this in Erlang [1]: > > HTML1 = <>. > > You really wouldn't want to have to write Python in single assignment > form, especially without tail call optimization. Python IO would benefit > from something like Erlang's iolist, but for use cases like the above > people tend to append to a list and then join it with the empty string, or > use an instance of cStringIO.StringIO. > > [1] Some versions of CPython do have some hacks that can detect this x += > s where x is a string with a reference count of 1 and strategically do > these updates in-place with over-allocation... so it's possible that it's > not quite as bad as you think it is (amortized). > > -bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.rutkowski@REDACTED Fri Jun 8 19:47:10 2012 From: adam.rutkowski@REDACTED (Adam Rutkowski) Date: Fri, 8 Jun 2012 19:47:10 +0200 Subject: [erlang-questions] More a comment that a question In-Reply-To: References: Message-ID: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> On Jun 8, 2012, at 7:37 PM, Eric Moritz wrote: > Where the danger comes in is mutating object, lists and dicts that other threads or functions have references to. It is possible to have a reference to a dict in a module or up the call stack that gets changed in an unexpected way and things crash. Erlang has made me a very paranoid Python developer. I let my Python code crash and I always regret it. Not to mention trivial indecisions like 'how should I deal with this without pattern matching?'. Thanks Erlang! > > On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito wrote: > On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry wrote: > > I've been coding/working with Erlang (and Chicago Boss) for a while > now, and I've recently had to do some work in Python. I find myself > looking at some code I'm writing that looks like this: > > html = html + "some markup" > > and getting nervous as I know this is a "naughty" that Erlang would > not let me away with. If only there was a "make everything > immutable" switch for Python! ;-) > > Strictly speaking, there's no mutation at all here, because str in Python *are* immutable. What's happening is that the variable html is rebound to the result. This is roughly equivalent to something like this in Erlang [1]: > > HTML1 = <>. > > You really wouldn't want to have to write Python in single assignment form, especially without tail call optimization. Python IO would benefit from something like Erlang's iolist, but for use cases like the above people tend to append to a list and then join it with the empty string, or use an instance of cStringIO.StringIO. > > [1] Some versions of CPython do have some hacks that can detect this x += s where x is a string with a reference count of 1 and strategically do these updates in-place with over-allocation... so it's possible that it's not quite as bad as you think it is (amortized). > > -bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- AR From michael.eugene.turner@REDACTED Fri Jun 8 20:01:52 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 03:01:52 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <6DE6B5C1-7F7F-41C7-BB78-1554B834741A@feuerlabs.com> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> <6DE6B5C1-7F7F-41C7-BB78-1554B834741A@feuerlabs.com> Message-ID: Ulf, I've been using it seq_trace for months. I know what it does. It's pretty clear what it's *typically* for. That's not to say there shouldn't be a tutorial. If Lamport clocks are (part of) what the customer asked for, there's no mystery hiding behind the NDA. What they asked for has already been been capital-D Disclosed -- in the seq_trace document. It just hasn't been given its proper name there. > Rather than 'independent' of the standard tracing, I would > say that sequence tracing is 'orthogonal'. I'm glad I wasn't drinking anything when I read that, because it would have spurted out of my nose. But I guess I get it anyway. [snip a long explication of what I already thought was pretty obvious] > Is this all clear from the seq_trace manual? No. If someone wants to write detailed use-cases for seq_trace, I'm all in favor of that. I just want to get Lamport the credit he's due, while also helping to inform those who already know of his work that they can get Lamport clocks out of Erlang/OTP, if they need them. That can be done in four added words, words which nobody seems to willing to explicitly commit to, much less actually add themselves. No, I have to write a patch. Otherwise it won't happen. Weird. As if the *ethical* onus here is not on the provider of the documentation. > That seq_trace makes use of Lamport clocks is an added > bonus, and at least the tools in OTP, like e.g. ttb and et, > should take advantage of this whenever possible. As it > is now, they don't (or I missed it, which is also a possibility). Or everybody missed it, because when they searched the documentation on "Lamport clocks", under the very reasonable assumption that someone writing them up would call them what they're actually called, nothing turned up. And I fear it's because of ego: people at Ericsson still think they invented this everything in seq_trace first, maybe somewhere in forlopp tracing .... > Is this all clear from the seq_trace manual? No. > It's easy to get the idea that it was created for a different > purpose entirely ... I have no idea why you think that. Among my first thoughts, upon encountering seq_trace, was, "hey, that's cool: if I've got zillions of messages flying around, and I'm only interested in those that originate from one process (or a few of them) and kick off yet others in a kind of chain reaction, seq_trace helps me reduce overhead. Yes, I have uses for that." Then I discovered that it implements Lamport clocks. I was even happier. In fact, it implements almost nothing *but* Lamport clocks -- which implicitly include tokens being passed between processes that might be looking at skewed clocks - so almost everything you describe falls out as the simplest default choice of what messages to record. But so what? We agree that it has a bad name -- I think if you're going to call it a trace package, call it "par_trace" or something. But so what? Computer technology is full of misnomers. > But it sounds as if you are putting seq_trace to good use, > in a way that is different from what I describe above. > Boiling this down to an example would be a great contribution. Um, no. I'll be trying to use seq_trace to verify a partial ordering of multiple default inheritance operations driven by spreading activation in (natural) language network descriptions. Somewhat as described by Richard Hudson in his last two books on Word Grammar. Except that he's a little scared of doing the inheritance in a spreading-activation style (which is already very important in his theory), and I'm trying to find out whether he's actually right to be scared. I don't think he is - I think the win is always much bigger than any possible loss. I've convinced him of other (less ambitious) operational simplifications for his theory, and maybe I can nail this one, too. It's kind of an annoying testing problem, because you don't care about the order or precise timing for some events, but the order definitely has to be right in other ways - i.e., your basic partial order, which is something Lamport clocks ("happened before" relation) gives you. It's not like, "I sent this message, I should see the following exact sequence of resulting messages." Then it would be easy. Or easiER, anyway. I *could* write these tests as simply a process that sends activation messages to some language-network concept nodes (each of which is a process), then send an "inherit" message to the concept node that's supposed to inherit, and wait around to see what it inherits. The problem with that: if it doesn't get the right traits, I'll have no idea why. seq_trace is cool because it can tell me what happened. It's also cool for regression tests because if the right traits are inherited (and other things happen in the right order) the test silently passes. It's also pretty scalable from what I can tell. I can parallelize my tests. That's important because human languages are big and will require a lot of test cases. But I understand the need for a tutorial for more typical uses of seq_trace. This whole thread is, amazingly, is just a lot of topic drift from somebody asking "What's the equivalent of Hello World for Erlang tracing?" For seq_trace, at least in a testing role, the Hello World should be a kind of unnaturally simple problem like this: Make sure that when process A sends a message X to process B, B sends a message Y to process C as a result, and C receives it. So you provide a tracer that sends A an acknowledgement if it sees C get Y after A sends X to B. Process A waits in a "receive/until". The test fails if that receive/until times out. And in fact a lot of my tests so far are just a process sending "inherit" up the inheritance chain(s) in the network of concept processes and then waiting for the tracer to report that a certain message arrived at some concept node. I'll need to go beyond this, soon enough. But it's good enough for where I am now. -michael turner On Sat, Jun 9, 2012 at 1:50 AM, Ulf Wiger wrote: > > On 8 Jun 2012, at 17:51, Michael Turner wrote: > >> Perhaps they expressed them as, "we have clock skew problems in our >> distributed system, and we need some way to correctly sequence our >> traces in spite of that." Lamport clocks are a simple, classical >> solution to that problem. > > No (or, if they asked for that, I'm not aware of it. The OTP team > can correct me if I'm wrong, but that was not the impetus to seq_trace.) > > I will try one more time. > > Tracing a sequence, as in "our system handles a thousand call > setups per second, and if we turn on a trace on all of them, we > will not only learn nothing - we will kill the system. We need a way > to trigger trace output on *one* session in the midst, and have > that trigger 'contaminate' processes as the request is passed > around in the system, and then, obviously, turn off, so we get > only what we asked for, and nothing more". > > Effectively, automatically selecting trace output so that it looks > as if we traced everything and ran only one single call through > the system (which is what most people resort to). > > If we call it "session trace", does that make it clearer? > > Obviously ordering (sequencing) is *one* part of the problem, > for which Lamport clocks are a great solution. But the part where > tokens act as "probes" whizzing through the system activating > trace output selectively, is part of trying to reduce the amount > of trace data generated. > > A large part of the complexity of the tracing subsystem in Erlang > comes from the need of the user to be able to define, ad-hoc, > just the right filters so that one can get useful trace output without > killing the system. While you can accomplish a "session-specific" > trace just using pattern-matching on function calls, this quickly > becomes unwieldy. Usually, you just want to enable a wide trace, > to include all important calls, but *only* for the one session you > decide to trace - not for the perhaps hundreds or thousands of > other sessions that may touch the same process. > > For this, the trace patterns in the tracing subsystem can match > on function call parameters and dynamically set and clear > trace tokens. > > Rather than 'independent' of the standard tracing, I would > say that sequence tracing is 'orthogonal'. The standard trace > is great for tracing on a small set of functions or modules, or > showing all activity in one or a few processes. But in large > systems under commercial load, doing any kind of tracing is > really scary. Some Erlang old-timers are known to explain > how they took down entire mobile networks by carelessly > setting up a wrong trace. > > This is my take on this area. In most practical uses, the > ordering one gets from timestamps is perfectly fine (for > tracing - *not* if one really wants to ensure that the trace > reflects the exact causal order. The hardest problem in > the scenario I describe is avoiding killing the node or > at least getting so much trace data that any analysis > of it becomes prohibitively hard. > > That seq_trace makes use of Lamport clocks is an added > bonus, and at least the tools in OTP, like e.g. ttb and et, > should take advantage of this whenever possible. As it > is now, they don't (or I missed it, which is also a possibility). > > Is this all clear from the seq_trace manual? No. > It's easy to get the idea that it was created for a different > purpose entirely, and even people who seek it out wanting > to do exactly what I describe above, tend to turn away > frustrated. But it's a hard problem to solve, and I'm not > saying that the current support is sub-par. If I had a clear > idea of how to improve it, I would have submitted patches > long ago. What I did to try to improve things was work to > get ttb's support for saving useful trace patterns and > replaying them later, more stable and better documented. > This doesn't relate to seq_trace as much as to the ability > to manage trace tokens through trace patterns. > > It's badly named. It should definitely not speak of > "sequential tracing". I'm pretty sure "sequence" > came out of "f?rlopp", which basically means "a > sequence of events". The function of a "forlopp" in AXE > is that it the smallest single unit of failure. It can be aborted > and re-run, like a transaction. > > Does all this preclude adding a reference to lamport > clocks in the seq_trace manual? Obviously not. > > If it bugs you so much, write a patch and submit it. > I agree that it will cost OTP as much to vet your patch as > it would for them to put the sentence in themselves, but > if it's not a priority to them, and it is to you, you know what > to do. > > But it sounds as if you are putting seq_trace to good use, > in a way that is different from what I describe above. > Boiling this down to an example would be a great contribution. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From ulf@REDACTED Fri Jun 8 20:23:50 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 8 Jun 2012 20:23:50 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <10C85F65-8180-4D14-9AB9-4F321ADBD6DA@feuerlabs.com> <4FDFF3A8-BE43-43D8-826D-C57C4F717EE0@feuerlabs.com> <6DE6B5C1-7F7F-41C7-BB78-1554B834741A@feuerlabs.com> Message-ID: On 8 Jun 2012, at 20:01, Michael Turner wrote: > But I understand the need for a tutorial for more typical uses of > seq_trace. Sure, but if you could condense what you're doing into a simple example, I think that would be very, very cool. It's not a typical problem for telecoms systems, but in a way that's the best kind of help community members can give. Ericsson should manage to explain the mainstream problems in telecoms, but Erlang is used for many other things nowadays, and the docs should reflect that. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From paul.james.barry@REDACTED Fri Jun 8 20:43:59 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Fri, 8 Jun 2012 19:43:59 +0100 Subject: [erlang-questions] More a comment that a question In-Reply-To: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> References: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> Message-ID: Thanks for all those comments, folks. I guess my main motivation for starting this thread was that I find single assignment in Erlang somewhat liberating, in that I don't have to worry about a variable's value changing under me at some later point in my code. Like Eric, I'm looking at my current Python code in a very different (but good) way thanks to my exposure to Erlang... which can only be a good thing. Regards. Paul. On 8 June 2012 18:47, Adam Rutkowski wrote: > > On Jun 8, 2012, at 7:37 PM, Eric Moritz wrote: > >> Where the danger comes in is mutating object, lists and dicts that other threads or functions have references to. ?It is possible to have a reference to a dict in a module or up the call stack that gets changed in an unexpected way and things crash. ?Erlang has made me a very paranoid Python developer. > > I let my Python code crash and I always regret it. > Not to mention trivial indecisions like 'how should I deal with this without pattern matching?'. > > Thanks Erlang! > >> >> On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito wrote: >> On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry wrote: >> >> I've been coding/working with Erlang (and Chicago Boss) for a while >> now, and I've recently had to do some work in Python. ?I find myself >> looking at some code I'm writing that looks like this: >> >> ? html = html + "some markup" >> >> and getting nervous as I know this is a "naughty" that Erlang would >> not let me away with. ? If only there was a "make everything >> immutable" switch for Python! ?;-) >> >> Strictly speaking, there's no mutation at all here, because str in Python *are* immutable. What's happening is that the variable html is rebound to the result. This is roughly equivalent to something like this in Erlang [1]: >> >> ? ? HTML1 = <>. >> >> You really wouldn't want to have to write Python in single assignment form, especially without tail call optimization. Python IO would benefit from something like Erlang's iolist, but for use cases like the above people tend to append to a list and then join it with the empty string, or use an instance of cStringIO.StringIO. >> >> [1] Some versions of CPython do have some hacks that can detect this x += s where x is a string with a reference count of 1 and strategically do these updates in-place with over-allocation... so it's possible that it's not quite as bad as you think it is (amortized). >> >> -bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > AR > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From bchesneau@REDACTED Fri Jun 8 20:58:31 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 8 Jun 2012 20:58:31 +0200 Subject: [erlang-questions] [ANN] mochicow 0.1 In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 7:25 PM, Bob Ippolito wrote: > On Fri, Jun 8, 2012 at 9:56 AM, Benoit Chesneau wrote: >> >> >> I just released the version 1 of mochicow, a mochiweb adapter for >> cowboy. Use your mochiweb code d with cowboy. >> >> There are 2 ways to use mochicow: >> >> - as a cowboy protocol: It will use the socket acceptor pool of cowboy >> instead of the mochiweb one. >> - as a protocol upgrade. Like websockets you can upgrade a cowboy >> handler to use a mochiweb loop. It allows you to use both your new >> code with cowboy and old code with mochiweb. >> >> See the examples in the examples for the usage. >> >> The source is on github: >> >> https://github.com/benoitc/mochicow >> >> Enjoy :) > > > This is fantastic, I had considered doing this myself but I don't have any > legacy code to worry about at the moment :) > Thanks :) I just commited a simple example with webmachine: https://github.com/benoitc/mochicow/tree/master/examples/webmachine_demo The most interesting part is the `webmachine_mochicow` module: https://github.com/benoitc/mochicow/blob/master/examples/webmachine_demo/src/webmachine_mochicow.erl Also I'm doing some tests on top of the couchdb API with my fork couch_core/rcouch in the mochicow branch, and it seems to work enough well: https://github.com/refuge/couch_core/compare/master...mochicow - beno?t From hobson42@REDACTED Fri Jun 8 21:16:03 2012 From: hobson42@REDACTED (Ian) Date: Fri, 08 Jun 2012 20:16:03 +0100 Subject: [erlang-questions] Learning Erlang Message-ID: <4FD24F73.1030903@gmail.com> Hi All, Is there a book, on-line resource, or even a well written open-source project, that would help me understand how to write good clean Erlang code? I taught myself PHP about 4 years ago, and have recently had to go back and modify the first large system I wrote, and I find the coding of that system is dreadful! Now that's good in the sense that I have improved, but bad in that the code is still bad and its costing me effort while I change it now. Its also bad because I should have enough programming experience to avoid writing bad code. I've been programming since 74. I am about to build two web based applications in Erlang, and I don't want to wince too much when I come to enhance them. Thanks for any tips Ian From zabrane3@REDACTED Fri Jun 8 21:23:19 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 8 Jun 2012 21:23:19 +0200 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD24F73.1030903@gmail.com> References: <4FD24F73.1030903@gmail.com> Message-ID: <7928E5CF-ED57-420E-AF9D-FCC45D616673@gmail.com> Don't spent a penny: http://learnyousomeerlang.com/ written by a very smart Erlang coder: Fred Regards, Zabrane On Jun 8, 2012, at 9:16 PM, Ian wrote: > Hi All, > > Is there a book, on-line resource, or even a well written open-source project, that would help me understand how to write good clean Erlang code? > > I taught myself PHP about 4 years ago, and have recently had to go back and modify the first large system I wrote, and I find the coding of that system is dreadful! Now that's good in the sense that I have improved, but bad in that the code is still bad and its costing me effort while I change it now. Its also bad because I should have enough programming experience to avoid writing bad code. I've been programming since 74. > > I am about to build two web based applications in Erlang, and I don't want to wince too much when I come to enhance them. > > Thanks for any tips > > Ian > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Jun 8 21:24:11 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 8 Jun 2012 12:24:11 -0700 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD24F73.1030903@gmail.com> References: <4FD24F73.1030903@gmail.com> Message-ID: http://learnyousomeerlang.com is the best out there IMO. On Fri, Jun 8, 2012 at 12:16 PM, Ian wrote: > Hi All, > > Is there a book, on-line resource, or even a well written open-source > project, that would help me understand how to write good clean Erlang code? > > I taught myself PHP about 4 years ago, and have recently had to go back > and modify the first large system I wrote, and I find the coding of that > system is dreadful! Now that's good in the sense that I have improved, but > bad in that the code is still bad and its costing me effort while I change > it now. Its also bad because I should have enough programming experience to > avoid writing bad code. I've been programming since 74. > > I am about to build two web based applications in Erlang, and I don't want > to wince too much when I come to enhance them. > > Thanks for any tips > > Ian > > > > > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Fri Jun 8 21:25:03 2012 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 8 Jun 2012 21:25:03 +0200 Subject: [erlang-questions] [ANN] mochicow 0.1 In-Reply-To: References: Message-ID: <5C695AE0-472C-4548-958B-C45D71F23B28@dmitriid.com> Wow. Just wow !! On Jun 8, 2012, at 20:58 , Benoit Chesneau wrote: > On Fri, Jun 8, 2012 at 7:25 PM, Bob Ippolito wrote: >> On Fri, Jun 8, 2012 at 9:56 AM, Benoit Chesneau wrote: >>> >>> >>> I just released the version 1 of mochicow, a mochiweb adapter for >>> cowboy. Use your mochiweb code d with cowboy. >>> >>> There are 2 ways to use mochicow: >>> >>> - as a cowboy protocol: It will use the socket acceptor pool of cowboy >>> instead of the mochiweb one. >>> - as a protocol upgrade. Like websockets you can upgrade a cowboy >>> handler to use a mochiweb loop. It allows you to use both your new >>> code with cowboy and old code with mochiweb. >>> >>> See the examples in the examples for the usage. >>> >>> The source is on github: >>> >>> https://github.com/benoitc/mochicow >>> >>> Enjoy :) >> >> >> This is fantastic, I had considered doing this myself but I don't have any >> legacy code to worry about at the moment :) >> > > Thanks :) > > I just commited a simple example with webmachine: > > https://github.com/benoitc/mochicow/tree/master/examples/webmachine_demo > > The most interesting part is the `webmachine_mochicow` module: > > https://github.com/benoitc/mochicow/blob/master/examples/webmachine_demo/src/webmachine_mochicow.erl > > > Also I'm doing some tests on top of the couchdb API with my fork > couch_core/rcouch in the mochicow branch, and it seems to work enough > well: > > https://github.com/refuge/couch_core/compare/master...mochicow > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Fri Jun 8 21:44:25 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 8 Jun 2012 20:44:25 +0100 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD24F73.1030903@gmail.com> References: <4FD24F73.1030903@gmail.com> Message-ID: <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> The O'Reilly by Francesco Cesarini and Simon Thompson called Erlang Programming is a very well-written solid introduction. Covers all the basics thoroughly and gives a firm foundation of OTP. It has useful exercises at the end of each chapter. Ivan Sent from my iPhone On 8 Jun 2012, at 20:16, Ian wrote: > Hi All, > > Is there a book, on-line resource, or even a well written open-source project, that would help me understand how to write good clean Erlang code? > > I taught myself PHP about 4 years ago, and have recently had to go back and modify the first large system I wrote, and I find the coding of that system is dreadful! Now that's good in the sense that I have improved, but bad in that the code is still bad and its costing me effort while I change it now. Its also bad because I should have enough programming experience to avoid writing bad code. I've been programming since 74. > > I am about to build two web based applications in Erlang, and I don't want to wince too much when I come to enhance them. > > Thanks for any tips > > Ian > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From overminddl1@REDACTED Fri Jun 8 21:47:05 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Fri, 8 Jun 2012 13:47:05 -0600 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On Fri, Jun 8, 2012 at 5:54 AM, Chandru wrote: > On 8 June 2012 12:33, Gordon Guthrie wrote: >> >> >> Or mebbies your way is better, simply pass in a fun to the backup >> traverse to write out the records in tables that I want to bring >> over... > > > FWIW, this is what we do on very large tables. Since you do use very large tables, have you thought about something like Riak, which does handle this fine? From wallentin.dahlberg@REDACTED Fri Jun 8 22:09:38 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Fri, 8 Jun 2012 22:09:38 +0200 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CC41DE@exch-mbx2.msk.trd.ru> References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> <06139A918ACCA041BF46A0F36940C7FA39CC41DE@exch-mbx2.msk.trd.ru> Message-ID: First, I would like to say.:Yes, the erl_interface documentation is lacking in details and examples. Secondly, I would like to reframe your question. Should I use erl_interface to communicate with my port driver in the first place? Many examples point to that fact but I would disagree. erl_interface, (ei), should be used for c-nodes where we want to talk to our c-program over erlang distribution. When I want my c-program to simulate an erlang node. This is, I think, a rare case though. In most situations I would construct a packet protocol instead and handle this protocol on the erlang side with bitsyntax. Handling "c-data" in erlang is much easier than using erlang terms in c. Look at efile_drv and inet_drv, none of those use erl_interface. The protocol becomes far simpler and probably faster. We have only one other case where it makes sense to encode and decode terms in c and that is when our data are in the erlang process heap, i.e. when we use NIFs. But, in this case we have an entirely different API so we wouldn't use ei here either. So, while I agree that ei could be better documented, I question if you really need it. I think it complicates more things than it solves. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri Jun 8 22:21:38 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 8 Jun 2012 21:21:38 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On 8 June 2012 20:47, OvermindDL1 wrote: > On Fri, Jun 8, 2012 at 5:54 AM, Chandru > wrote: > > On 8 June 2012 12:33, Gordon Guthrie wrote: > >> > >> > >> Or mebbies your way is better, simply pass in a fun to the backup > >> traverse to write out the records in tables that I want to bring > >> over... > > > > > > FWIW, this is what we do on very large tables. > > Since you do use very large tables, have you thought about something > like Riak, which does handle this fine? > Yes, we keep meaning to, but haven't found the time yet. Hopefully soon. Though only the commercial version of Riak handles multi site replication, not the open source version. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.rutkowski@REDACTED Fri Jun 8 22:58:44 2012 From: adam.rutkowski@REDACTED (Adam Rutkowski) Date: Fri, 8 Jun 2012 22:58:44 +0200 Subject: [erlang-questions] Learning Erlang In-Reply-To: <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> References: <4FD24F73.1030903@gmail.com> <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> Message-ID: <6FA47693-AF62-4B58-8DEA-57A92E6B8C04@jtendo.com> >> Is there a book, on-line resource, or even a well written open-source project, that would help me understand how to write good clean Erlang code? Hey Ian, I'd say Joe Armstrong's "Programming Erlang" (http://pragprog.com/book/jaerlang/programming-erlang) is arguably the best introductory resource. It takes a labor-intensive weekend to get through it and weeks of flashbacks to get your head around it. A great starting point and often (I believe I'm quoting Ferd on this one) - "the ultimate answer". Apart from the books already mentioned in this thread, I can think of two: 1) As soon as you get the "Erlang assembly", that is, semantics and all the basic concepts, I would definitely recommend "Erlang Programming" (http://www.erlangprogramming.org). 2) If you're interested in web apps, the most recent one (and probably the only one?) is "Building Web Applications with Erlang" (http://shop.oreilly.com/product/0636920021452.do) (I haven't read that one yet though). Good luck :) -- AR From mfidelman@REDACTED Fri Jun 8 23:02:19 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Fri, 08 Jun 2012 17:02:19 -0400 Subject: [erlang-questions] Learning Erlang In-Reply-To: <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> References: <4FD24F73.1030903@gmail.com> <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> Message-ID: <4FD2685B.4090003@meetinghouse.net> The best one I've found is "Erlang and OTP in Action" by Logan, Merritt, and Carlsson. It's the only book I've found that goes seriously into the OTP platform, and into building distributed production systems with Erlang. Ivan Uemlianin wrote: > The O'Reilly by Francesco Cesarini and Simon Thompson called Erlang Programming is a very well-written solid introduction. Covers all the basics thoroughly and gives a firm foundation of OTP. > > It has useful exercises at the end of each chapter. > > Ivan > > Sent from my iPhone > > On 8 Jun 2012, at 20:16, Ian wrote: > >> Hi All, >> >> Is there a book, on-line resource, or even a well written open-source project, that would help me understand how to write good clean Erlang code? >> >> I taught myself PHP about 4 years ago, and have recently had to go back and modify the first large system I wrote, and I find the coding of that system is dreadful! Now that's good in the sense that I have improved, but bad in that the code is still bad and its costing me effort while I change it now. Its also bad because I should have enough programming experience to avoid writing bad code. I've been programming since 74. >> >> I am about to build two web based applications in Erlang, and I don't want to wince too much when I come to enhance them. >> >> Thanks for any tips >> >> Ian >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From eric@REDACTED Fri Jun 8 23:30:03 2012 From: eric@REDACTED (Eric Moritz) Date: Fri, 8 Jun 2012 17:30:03 -0400 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD2685B.4090003@meetinghouse.net> References: <4FD24F73.1030903@gmail.com> <2F3EB363-5699-4242-BD7B-6F5B2823D135@llaisdy.com> <4FD2685B.4090003@meetinghouse.net> Message-ID: +1 for OTP in Action. On Fri, Jun 8, 2012 at 5:02 PM, Miles Fidelman wrote: > The best one I've found is "Erlang and OTP in Action" by Logan, Merritt, > and Carlsson. It's the only book I've found that goes seriously into the > OTP platform, and into building distributed production systems with Erlang. > > Ivan Uemlianin wrote: > >> The O'Reilly by Francesco Cesarini and Simon Thompson called Erlang >> Programming is a very well-written solid introduction. Covers all the >> basics thoroughly and gives a firm foundation of OTP. >> >> It has useful exercises at the end of each chapter. >> >> Ivan >> >> Sent from my iPhone >> >> On 8 Jun 2012, at 20:16, Ian wrote: >> >> Hi All, >>> >>> >>> Is there a book, on-line resource, or even a well written open-source >>> project, that would help me understand how to write good clean Erlang code? >>> >>> I taught myself PHP about 4 years ago, and have recently had to go back >>> and modify the first large system I wrote, and I find the coding of that >>> system is dreadful! Now that's good in the sense that I have improved, but >>> bad in that the code is still bad and its costing me effort while I change >>> it now. Its also bad because I should have enough programming experience to >>> avoid writing bad code. I've been programming since 74. >>> >>> I am about to build two web based applications in Erlang, and I don't >>> want to wince too much when I come to enhance them. >>> >>> Thanks for any tips >>> >>> Ian >>> >>> >>> >>> >>> >>> >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Fri Jun 8 23:42:31 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Fri, 8 Jun 2012 23:42:31 +0200 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD24F73.1030903@gmail.com> References: <4FD24F73.1030903@gmail.com> Message-ID: 2012/6/8 Ian > Hi All, > > Is there a book, on-line resource, or even a well written open-source > project, that would help me understand how to write good clean Erlang code? > My "A guide to the Erlang Style"[1] was/is meant to be just that: An attempt to put into words what I've learnt about style in Erlang. It's not about how to program in Erlang, but how to design in Erlang (the small and the large stuff); i.e. not about how to express some functionality at all, but how it can be expressed in different ways and the trade-offs involved (including such things as convention). Even so, I've come to feel that I ought to know more of L.Y.S.E -- the free Erlang introduction which is often mentioned --, lest I just end up duplicating parts of work, inferiorly... However, reading L.Y.S.E. is yet another thing to spend my 'all-too-plentiful' spare time on, and thorough as it may be, I don't expect to learn much Erlang from it (most of the chapters, anyway). One thing experience has told me is that I can waste a lot of time duplicating things which others have the time and resources to do better. My pendulum is, these years, at the extreme where I assume most things have already been done, so that I need to make sure something is worth doing - and that I'm going about it the right way - before putting much work into it. That isn't too productive either; hopefully I'll get to a more balanced point. Anyway, long story short, I'll probably be doing other things until I have a better idea of whether I'm on the right track, and what questions I should focus on answering. (When I announced the project on Twitter, the only feedback I got was in the form of a number of people starting watching the Github repo. Encouraging, to be sure, but not very useful as guidance; I don't even know whether they saw something useful there or just hoped that there would eventually be something useful :-)) [1] https://github.com/eriksoe/AGttES -------------- next part -------------- An HTML attachment was scrubbed... URL: From parseerror@REDACTED Sat Jun 9 00:23:35 2012 From: parseerror@REDACTED (Ryan Flynn) Date: Fri, 8 Jun 2012 18:23:35 -0400 Subject: [erlang-questions] What is shutdown_error? In-Reply-To: References: Message-ID: Thank you :) On Wed, Jun 6, 2012 at 4:25 AM, Siri Hansen wrote: > sorry, forgot to copy the list > > > 2012/6/6 Siri Hansen > >> shutdown_error is used by supervisor (and application_master) to inform >> at something went a bit wrong during shutdown or a child. When shutting >> down a child, supervisor will normally do exit(ChildPid,shutdown) and then >> monitor the child to check the actual exit reason. If it terminates with >> Reason /= shutdown, supervisor will report a shutdown_error. >> >> Also, if a child does not respond to supervisor's attempt of graceful >> shutdown within the shutdown time specified in the child spec, then the >> supervisor will kill it brutally and report a shutdown_error. This might be >> what happened in your case, since the reason reported is 'killed'. >> >> Regards >> /siri >> >> 2012/6/5 Ryan Flynn >> >>> What is shutdown_error? I can't figure out what to make of >>> shutdown_error; all I can find is a brief mention that it exists in sasl >>> docs. Full error: >>> >>> Supervisor foo_sup had child foo_server started with >>> foo_server:start_link() at <0.379.0> exit with reason killed in context >>> shutdown_error >>> >>> Thanks in advance. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat Jun 9 01:51:54 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 09 Jun 2012 00:51:54 +0100 (BST) Subject: [erlang-questions] Data structures are strange things In-Reply-To: <27729434-CD15-4082-B0DF-AC99CCFAE34A@cs.otago.ac.nz> Message-ID: <4fec23e3-fdd9-44e4-8453-836d734c4f66@knuth> No, it is not surprising when you think about it. I just found it a bit surprising that the conventions which seem to apply to imperative languages have been more less imported into the functional/declarative world where they don't apply to the same degree. It seems that in the functional world going "red-black" seems to be accepted even though there aren't many benefits of it. The code is a little simpler, but for 2-3 trees the difference is very small. And it is faster for insertion and deletion. ----- Original Message ----- > If you take a look at a typical 2-3-4 tree implementation in Java > http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java > >> http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java > you'll see nodes containing > > an integer saying how many keys there are > three keys > four links to children nodes > a link to a parent node > > Allow one word for "what is the class of this object" and that's > 10 words per node. > > Using Erlang tuples, we'd have > {K1,C0,C1} 4 words > {K1,K2,C0,C1,C2} 6 words > {K1,K2,K3,C0,C1,C2,C3} 8 words > WAM-based Prolog systems would take the same space; > SML/NJ should be able to shave a word off each (using the bottom > 2 bits of a pointer to say which of the three cases applies). > > In a functional language, when you make a change you create new > nodes, > so no node ever has to change size/shape. You _could_ do that in > Java > too, but people often don't think of it. > > It seems pretty clear that the Prolog/Erlang/SML version of a 2-3-4 > tree is going to take *less* space than a binary tree in the same > language. Since in these languages there is often a strong > correlation > between space and time, I find Robert Virding's results pleasant but > not that surprising. > > > From robert.virding@REDACTED Sat Jun 9 02:03:19 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 09 Jun 2012 01:03:19 +0100 (BST) Subject: [erlang-questions] Data structures are strange things In-Reply-To: Message-ID: <9d3f3131-04b1-4cb5-9fc1-c40d4121a64c@knuth> The flatter trees do work better, but the difference for lookup should be small as you end up doing the same comparisons as in a binary tree. One problem I found was that code size grows as there become more cases to handle. Unless I can find/come up with some really smart compact handling then 2-3 trees seems to be a reasonable trade-off with slightly larger code but faster. Code grew with 2-3-4 trees as there was a new node representation to handle which is more complex than the other ones. So something smart is needed here. Anyway there should soon be a tree alternative for the dict API. Robert ----- Original Message ----- > "I don't know why the straight multi-size versions are faster but my > guess is that the pattern-matching/data construction is faster on the > flat bigger nodes rather than on the nested pairs...." > > Just a guess here (but at least a guess informed by an obsession with > search tree data structures, several decades ago): > > Before you go looking for the effects of a reduction of space, > remember that the conventional time complexity analysis for > operations > on such data structures is predicated on destructive updates. An > O(log > n) time complexity for delete or update represents the *worst case*. > Some operations will result in no rebalancing at all, just the > addition or removal of a node. > > Without destructive updates, you always need to rebuild/rewrite tree > nodes from the point of update up to the root. Even if nothing > substantive changes along the way. It'll cost ya. In space, too. > > How can you reduce these costs? By systematically reducing the > distance to the root. You can do that by choosing a structure with a > higher (average) branching factor. The higher the branching factor of > your search structure, the shallower it will be, and the fewer > node-allocate-and-rewrite operations you'll need to do on your way > from the updated node to the root. (This probably reduces GC costs as > well, though I expect that would be a second-order effect.) > > Increasing the branching factor might have diminishing returns after > some point, but I doubt that a branching factor of 4 is anywhere near > that point. 2-3 trees, and 2-3-4 trees, have higher (average) > branching factors than binary trees. Leaning RB trees are binary, but > there might be something in how they lean that effectively reduces > the > average distance from updated node to root. I don't know anything > about them. > > "P.S. This is the reason why there was hoped to be many different > dict > modules with the same API but different data structures. So as it to > make it easier to choose the most suitable version for each occasion. > There is no generally "best" choice, even among trees." > > Sign me up for that. > > -michael turner > > On Fri, Jun 8, 2012 at 10:32 AM, Richard O'Keefe > wrote: > > If you take a look at a typical 2-3-4 tree implementation in Java > > ? ?http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java > >>> ?http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java > > you'll see nodes containing > > > > ? ? ? ?an integer saying how many keys there are > > ? ? ? ?three keys > > ? ? ? ?four links to children nodes > > ? ? ? ?a link to a parent node > > > > Allow one word for "what is the class of this object" and that's > > 10 words per node. > > > > Using Erlang tuples, we'd have > > ? ? ? ?{K1,C0,C1} ? ? ? ? ? ? ? ? ? ? ?4 words > > ? ? ? ?{K1,K2,C0,C1,C2} ? ? ? ? ? ? ? ?6 words > > ? ? ? ?{K1,K2,K3,C0,C1,C2,C3} ? ? ? ? ?8 words > > WAM-based Prolog systems would take the same space; > > SML/NJ should be able to shave a word off each (using the bottom > > 2 bits of a pointer to say which of the three cases applies). > > > > In a functional language, when you make a change you create new > > nodes, > > so no node ever has to change size/shape. ?You _could_ do that in > > Java > > too, but people often don't think of it. > > > > It seems pretty clear that the Prolog/Erlang/SML version of a 2-3-4 > > tree is going to take *less* space than a binary tree in the same > > language. ?Since in these languages there is often a strong > > correlation > > between space and time, I find Robert Virding's results pleasant > > but > > not that surprising. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > From amindfv@REDACTED Sat Jun 9 03:32:34 2012 From: amindfv@REDACTED (Tom Murphy) Date: Fri, 8 Jun 2012 21:32:34 -0400 Subject: [erlang-questions] Fwd: Re: More a comment that a question In-Reply-To: References: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> Message-ID: Forgot the list: ---------- Forwarded message ---------- From: "Tom Murphy" Date: Jun 8, 2012 9:31 PM Subject: Re: [erlang-questions] More a comment that a question To: "Adam Rutkowski" Python has some basic pattern-matching, eg: (a, b) = ("foo", "bar") Tom On Jun 8, 2012 1:47 PM, "Adam Rutkowski" wrote: > > On Jun 8, 2012, at 7:37 PM, Eric Moritz wrote: > > > Where the danger comes in is mutating object, lists and dicts that other > threads or functions have references to. It is possible to have a > reference to a dict in a module or up the call stack that gets changed in > an unexpected way and things crash. Erlang has made me a very paranoid > Python developer. > > I let my Python code crash and I always regret it. > Not to mention trivial indecisions like 'how should I deal with this > without pattern matching?'. > > Thanks Erlang! > > > > > On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito wrote: > > On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry > wrote: > > > > I've been coding/working with Erlang (and Chicago Boss) for a while > > now, and I've recently had to do some work in Python. I find myself > > looking at some code I'm writing that looks like this: > > > > html = html + "some markup" > > > > and getting nervous as I know this is a "naughty" that Erlang would > > not let me away with. If only there was a "make everything > > immutable" switch for Python! ;-) > > > > Strictly speaking, there's no mutation at all here, because str in > Python *are* immutable. What's happening is that the variable html is > rebound to the result. This is roughly equivalent to something like this in > Erlang [1]: > > > > HTML1 = <>. > > > > You really wouldn't want to have to write Python in single assignment > form, especially without tail call optimization. Python IO would benefit > from something like Erlang's iolist, but for use cases like the above > people tend to append to a list and then join it with the empty string, or > use an instance of cStringIO.StringIO. > > > > [1] Some versions of CPython do have some hacks that can detect this x > += s where x is a string with a reference count of 1 and strategically do > these updates in-place with over-allocation... so it's possible that it's > not quite as bad as you think it is (amortized). > > > > -bob > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > AR > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat Jun 9 04:36:33 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 8 Jun 2012 19:36:33 -0700 Subject: [erlang-questions] More a comment that a question In-Reply-To: References: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> Message-ID: That's not really pattern matching, it's just destructuring assignment. On Friday, June 8, 2012, Tom Murphy wrote: > Forgot the list: > ---------- Forwarded message ---------- > From: "Tom Murphy" 'amindfv@REDACTED');>> > Date: Jun 8, 2012 9:31 PM > Subject: Re: [erlang-questions] More a comment that a question > To: "Adam Rutkowski" 'cvml', 'adam.rutkowski@REDACTED');>> > > Python has some basic pattern-matching, eg: > > (a, b) = ("foo", "bar") > > Tom > On Jun 8, 2012 1:47 PM, "Adam Rutkowski" > > wrote: > >> >> On Jun 8, 2012, at 7:37 PM, Eric Moritz wrote: >> >> > Where the danger comes in is mutating object, lists and dicts that >> other threads or functions have references to. It is possible to have a >> reference to a dict in a module or up the call stack that gets changed in >> an unexpected way and things crash. Erlang has made me a very paranoid >> Python developer. >> >> I let my Python code crash and I always regret it. >> Not to mention trivial indecisions like 'how should I deal with this >> without pattern matching?'. >> >> Thanks Erlang! >> >> > >> > On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito > >> wrote: >> > On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry > >> wrote: >> > >> > I've been coding/working with Erlang (and Chicago Boss) for a while >> > now, and I've recently had to do some work in Python. I find myself >> > looking at some code I'm writing that looks like this: >> > >> > html = html + "some markup" >> > >> > and getting nervous as I know this is a "naughty" that Erlang would >> > not let me away with. If only there was a "make everything >> > immutable" switch for Python! ;-) >> > >> > Strictly speaking, there's no mutation at all here, because str in >> Python *are* immutable. What's happening is that the variable html is >> rebound to the result. This is roughly equivalent to something like this in >> Erlang [1]: >> > >> > HTML1 = <>. >> > >> > You really wouldn't want to have to write Python in single assignment >> form, especially without tail call optimization. Python IO would benefit >> from something like Erlang's iolist, but for use cases like the above >> people tend to append to a list and then join it with the empty string, or >> use an instance of cStringIO.StringIO. >> > >> > [1] Some versions of CPython do have some hacks that can detect this x >> += s where x is a string with a reference count of 1 and strategically do >> these updates in-place with over-allocation... so it's possible that it's >> not quite as bad as you think it is (amortized). >> > >> > -bob >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> AR >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat Jun 9 05:48:25 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 8 Jun 2012 20:48:25 -0700 Subject: [erlang-questions] More a comment that a question In-Reply-To: References: <914AB983-0E4D-4000-8833-66114B816791@jtendo.com> Message-ID: That's not really pattern matching, it's just destructuring assignment. On Friday, June 8, 2012, Tom Murphy wrote: > Forgot the list: > ---------- Forwarded message ---------- > From: "Tom Murphy" 'amindfv@REDACTED');>> > Date: Jun 8, 2012 9:31 PM > Subject: Re: [erlang-questions] More a comment that a question > To: "Adam Rutkowski" 'cvml', 'adam.rutkowski@REDACTED');>> > > Python has some basic pattern-matching, eg: > > (a, b) = ("foo", "bar") > > Tom > On Jun 8, 2012 1:47 PM, "Adam Rutkowski" > > wrote: > >> >> On Jun 8, 2012, at 7:37 PM, Eric Moritz wrote: >> >> > Where the danger comes in is mutating object, lists and dicts that >> other threads or functions have references to. It is possible to have a >> reference to a dict in a module or up the call stack that gets changed in >> an unexpected way and things crash. Erlang has made me a very paranoid >> Python developer. >> >> I let my Python code crash and I always regret it. >> Not to mention trivial indecisions like 'how should I deal with this >> without pattern matching?'. >> >> Thanks Erlang! >> >> > >> > On Fri, Jun 8, 2012 at 1:21 PM, Bob Ippolito > >> wrote: >> > On Fri, Jun 8, 2012 at 9:34 AM, Paul Barry > >> wrote: >> > >> > I've been coding/working with Erlang (and Chicago Boss) for a while >> > now, and I've recently had to do some work in Python. I find myself >> > looking at some code I'm writing that looks like this: >> > >> > html = html + "some markup" >> > >> > and getting nervous as I know this is a "naughty" that Erlang would >> > not let me away with. If only there was a "make everything >> > immutable" switch for Python! ;-) >> > >> > Strictly speaking, there's no mutation at all here, because str in >> Python *are* immutable. What's happening is that the variable html is >> rebound to the result. This is roughly equivalent to something like this in >> Erlang [1]: >> > >> > HTML1 = <>. >> > >> > You really wouldn't want to have to write Python in single assignment >> form, especially without tail call optimization. Python IO would benefit >> from something like Erlang's iolist, but for use cases like the above >> people tend to append to a list and then join it with the empty string, or >> use an instance of cStringIO.StringIO. >> > >> > [1] Some versions of CPython do have some hacks that can detect this x >> += s where x is a string with a reference count of 1 and strategically do >> these updates in-place with over-allocation... so it's possible that it's >> not quite as bad as you think it is (amortized). >> > >> > -bob >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> AR >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sat Jun 9 06:35:41 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 9 Jun 2012 16:35:41 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: On 8/06/2012, at 7:19 PM, Michael Turner wrote: >>>> Our first technical writer, Kennita Watson, was technically competent. >> >> Did you read that sentence? > > Yes. I read it as "technically competent *as an engineer*", because > > (a) she has degrees in computer science (from Stanford and MIT at that > -- facts I recalled about her the instant I read her name), and > > (b) what followed was an anecdote that I didn't quite get, because > somebody (you?) wrote "Never let an engineer write the documentation" > and I thought you were agreeing with the point -- you certainly didn't > contradict it immediately. She wasn't an engineer >>>ON OUR PROJECT<<<. And that's what I was agreeing with. > > So at first I read your comment on her work as saying that she'd > suggested changes in the documentation to make things more > documentable from *her* point of view (as an engineer), but that they > were still things that the customers didn't understand. Not a bit of it. Our customers were *also* programmers; her value was precisely that her perspective was their perspective. I did say it was Prolog; what other kind of person uses a Prolog system? > Such people are admittedly as rare as hen's teeth in the more > quotidian livelihood of tech writing. The Kennitas of that world are > rare enough. (I think she was only sidelined because of some injury -- > I'm not sure. I'd kill to make use of her on a project myself, if she > were still available.) "Sidelined"? She moved to a bigger company (which is still around, unlike Quintus) with a larger customer base and more to write about. I never asked, but they could certainly have _afforded_ to pay her more than we could. Everyone I was close to at Quintus wanted her to stay. > > Which takes me back to my agenda: what's the *practical*, *dirt-cheap* > way to *eventually* get better quality docs for Erlang/OTP? Step 1: fix the tool chain so I can build the documents. I've annotated the erl_docgen document; there's something that needs fixing on every page. I am keen to send in patches, but I can't do that until I can build the documents. > And their edits can easily be reviewed. And now we come to the crux of the matter. It's the *review* process. That takes time and effort and management, and it's not at all clear to me what, if anything, makes a wiki easier to manage than the current setup. The mere existence of a wiki is no guarantee of good reviewing. The great thing about the current scheme is that edits are reviewed BEFORE they are adopted, but we're promised that they'll be reviewed quickly. With a Wiki, bad stuff goes in, and if you're lucky it will be reviewed and eventually be taken out, but there _is_ that window where the bad stuff is there. I won't mentioned which Wikipedia page it was, but the one Wikipedia page I ever did anything to was one which contained several gross factual errors and had for months. But there is also the fact that wikis tend to lead to lots of bits all over the place and a hell of clicketyclicketyclickety for almost anything. Recently I've been wandering round the Tizen "documentation". Not a good advertisement for "Web documentation". One thing the current tool chain gets almost right is that it really does generate man pages and HTML and PDF from the same sources. I have a shelf of printed and bound Erlang manuals. > People will say a lot of silly things about wikis and their governance > until they gain more direct experience of maintaining them. In that, > wikis are like anything in life. My comments about wikis are based entirely on my experience of actually trying to get basic information out of wiki "documentation" for several projects. "Like anything in life", wikis obey Sturegon's law. >> Wikis do not solve this problem, they exacerbate it. > > You're prone to sweeping generalizations about wikis based on no > direct personal experience of editing them, from what I can tell. I have edited one page in one wiki, no wait, three pages in two. But editing is simply not the issue. My comments are based on too much painful experience of trying to *USE* wikis edited by others. I don't see why only people who edit wikis count here. I think that people who *USE* documentation matter, and that having tried to *use* about a dozen wiki-"documented" open source projects and finding intense frustration is relevant. I find the suggestion that only experienced editors of Wikis are competent to judge the usability of wikis outrageous. >> How high *is* the bar? Have you ever submitted a documentation patch >> and had it rejected? > > I've never submitted a documentation patch. Ah! You've never tried, but you just _know_ that the barrier is high. > You hate amateurs. No I do not. I *am* an amateur. Let's go back to Quintus. While we had technical writers, the "engineers" often *originated* documentation. It just had to go through technical writers before customers saw it. And it was the same for code: with one notable exception, all changes that affected what customers saw had to be reviewed before final acceptance. And that one exception was the single dumbest thing Quintus ever did, and sadly, one of the most copied. The chief problem with the existing Erlang documentation is that it was released without an adequate review of the language. > I can understand that. But the experts fail sometimes too. It's not professional vs amateur. It is review-before-release vs post-release-review-if-ever. By the way, if you want to see an appalling example of editing, take a look at the ANSI Smalltalk standard. The 5th draft was full of glitches, almost all of which survived in the published standard. Even being-published-by-ANSI was no guarantee of adequate proof-reading. > > If I submitted a patch to the seq_trace documentation so that it > mentioned that seq_trace implements Lamport clocks, and it got > rejected with "we're still thinking about how to work that fact in, > maybe we'll have something in the next release," I think I'd end up > writing Lamport to say, "Can you *believe* these people?" Ulf Wiger is not in charge of reviewing changes to the Erlang documentation. The people who _are_ in charge might well be perfectly happy with adding "Since its first release, seq_trace has relied on Lamport clocks [ref], but it is not guaranteed that it will always do so." Right now, I'm reading someone complaining about the extreme difficulty of putting in a sentence and making up a story about what he'll say to someone when rejected WHO HAS NOT TRIED IT. Try it. *Then* resume this conversation. No, wait. *I've* just posted a change to erlang-patches. > Good ideas, I'm sure. I'm using Erlang to implement a computational > linguistics framework, so I'm definitely keen on those sorts of > solutions myself. But automating a task to get "better than nothing" > shouldn't be the goal here. This is supposed to be documentation by > professionals, for professionals. And professionals use good tools. INEX isn't running "link-the-Wiki" any more because you need to able to compare what you can do with a "gold standard", and it turns out that while the Wikipedia is *copiously* linked, it isn't always *well* linked, and when programs did badly at matching what was there, it was sometimes because the programs were doing better. As for unimplemented EEPs, that comes back to a documentation issue, namely that the BEAM is not (yet) adequately documented. From atmb4u@REDACTED Sat Jun 9 06:44:45 2012 From: atmb4u@REDACTED (Anoop Thomas Mathew) Date: Sat, 9 Jun 2012 10:14:45 +0530 Subject: [erlang-questions] Suggestions for an Erlang Opensource Project to Work On Message-ID: Hi All, I'm a beginner in erlang, and want to work on some opensource projects. Can some one suggest a few projects, which can be handled by beginners (And more specifically, a developer group where beginner's are welcome). Thanks, Anoop Thomas Mathew atm ___ Life is short, Live it hard. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sergey_Zhemzhitsky@REDACTED Sat Jun 9 09:08:32 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Sat, 9 Jun 2012 07:08:32 +0000 Subject: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? In-Reply-To: References: <87A855AF-378B-4C48-9B09-34131C62034C@cs.otago.ac.nz> <06139A918ACCA041BF46A0F36940C7FA39CC17D8@exch-mbx2.msk.trd.ru> <06139A918ACCA041BF46A0F36940C7FA39CC41DE@exch-mbx2.msk.trd.ru> Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CC459D@exch-mbx2.msk.trd.ru> Hi Bj?rn-Egil, Thanks a lot! Things have become slightly clearer. I have been looking for exactly such recommendations. The problem is that on the native-code side I have to translate erlang terms into a rather complex ?++ object that encapsulates the complexity of an underlying protocol format. It seems there is no alternatives to using ei in such a case. The ability to create a binary object in erlang and to use driver just to pass it through to the underlying level is pretty awesome. Best Regards, Sergey From: Bj?rn-Egil Dahlberg [mailto:wallentin.dahlberg@REDACTED] Sent: Saturday, June 09, 2012 12:10 AM To: Zhemzhitsky Sergey Cc: Richard O'Keefe; erlang-questions@REDACTED Subject: Re: [erlang-questions] What is the best way to parse complex erlang terms in port drivers? First, I would like to say.:Yes, the erl_interface documentation is lacking in details and examples. Secondly, I would like to reframe your question. Should I use erl_interface to communicate with my port driver in the first place? Many examples point to that fact but I would disagree. erl_interface, (ei), should be used for c-nodes where we want to talk to our c-program over erlang distribution. When I want my c-program to simulate an erlang node. This is, I think, a rare case though. In most situations I would construct a packet protocol instead and handle this protocol on the erlang side with bitsyntax. Handling "c-data" in erlang is much easier than using erlang terms in c. Look at efile_drv and inet_drv, none of those use erl_interface. The protocol becomes far simpler and probably faster. We have only one other case where it makes sense to encode and decode terms in c and that is when our data are in the erlang process heap, i.e. when we use NIFs. But, in this case we have an entirely different API so we wouldn't use ei here either. So, while I agree that ei could be better documented, I question if you really need it. I think it complicates more things than it solves. // Bj?rn-Egil _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Sat Jun 9 09:30:32 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 16:30:32 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: RIchard: "I did say it was Prolog; what other kind of person uses a Prolog system?" To name one: the linguist I've been working with, Richard Hudson, who tried to implement his theory of language in Prolog in the early 90s. "> "Sidelined"? ?She moved to a bigger company (which is still around, > unlike Quintus) with a larger customer base and more to write about." http://www.linkedin.com/pub/kennita-watson/1/321/a19 Kennita's most recent resume entry there: too much experience Long Term Disability July 2005 ? Present (7 years) Sunnyvale, CA I haven't worked since 2005; the purpose of this entry is to get LinkedIn to stop asking me for a current position. --- Maybe she's only joking about the disability part. > And now we come to the crux of the matter. ?It's the *review* process. > That takes time and effort and management, and it's not at all clear > to me what, if anything, makes a wiki easier to manage than the current > setup. I don't know either, because I don't know Ericsson Erlang/OTP documentation patch review processes. For how it would work in a wiki, try starting at this link http://en.wikipedia.org/w/index.php?title=Richard_O%27Keefe&diff=489842048&oldid=489773645 and keep clicking "Previous edit". I think this interface makes it pretty easy to tell that a small, simple edit (which is most of them, and which would be most of them, in what I propose) is in fact correct. If it's not, note the "undo" links -- very often, "undo" just works without fuss. So if this is the "crux of the matter", maybe a wiki would actually make documentation patch review and acceptance *easier* than it is now at Ericsson: a reviewer would spent most of his time clicking "next" or "previous" links, looking at diffs, deciding whether to "undo" (and mostly deciding not to). Whatever the reviewer has left is the official new revision for that page, for the upcoming release. 90-99%% of edit reviews will take a few seconds, if they are limited to typos/grammar and wikilinking what hadn't been linked before. The more problematic content-related ones might take between minutes and hours. Or, in the case of "seq_trace", perhaps weeks, as it generates panicky questions to legal staff like, "Have we ever charged a customer any technology license fees on the basis of a claim that the algorithm here was somehow our own invention? And what's the company's potential liability if so?" > The mere existence of a wiki is no guarantee of good reviewing. *Eyeroll*. As if I said that it was. As if I haven't, in fact, spoken extensively about issues of administration already. More strawmen like these and I'll be buried under several bales worth of straw. > It's not professional vs amateur. ?It is review-before-release > vs post-release-review-if-ever. A false dichotomy. There's also "this wiki reflects recent changes by volunteers since the last snapshot for release X; these changes haven't necessarily been officially reviewed and/or approved by anyone at Ericsson, though they will be reviewed before the next release, which is planned for around . If you want the official version for release X, here's a link to the snapshot of this article that went out at the same time." > "Like anything in life", wikis obey Sturegon's law. Oh? If all existing Erlang/OTP documentation were put in a wiki, as I propose, would it be crap just by being in a wiki, from day one of the wiki being live? Possibly so, if you consider the current documentation crap. > I find the suggestion that only experienced editors of Wikis are > competent to judge the usability of wikis outrageous. The usability of a *given* wiki? Or the fitness-for-purpose of wiki software for helping maintain Erlang/OTP documentation, from an administrative point of view? It's only the latter issues that concern me here, and it's on those issues that you keep saying things I find either incomprehensible or flatly wrong. -michael turner On Sat, Jun 9, 2012 at 1:35 PM, Richard O'Keefe wrote: > > On 8/06/2012, at 7:19 PM, Michael Turner wrote: >>>>> Our first technical writer, Kennita Watson, was technically competent. >>> >>> Did you read that sentence? >> >> Yes. I read it as "technically competent *as an engineer*", because >> >> (a) she has degrees in computer science (from Stanford and MIT at that >> -- facts I recalled about her the instant I read her name), and >> >> (b) what followed was an anecdote that I didn't quite get, because >> somebody (you?) wrote "Never let an engineer write the documentation" >> and I thought you were agreeing with the point -- you certainly didn't >> contradict it immediately. > > She wasn't an engineer >>>ON OUR PROJECT<<<. > And that's what I was agreeing with. >> >> So at first I read your comment on her work as saying that she'd >> suggested changes in the documentation to make things more >> documentable from *her* point of view (as an engineer), but that they >> were still things that the customers didn't understand. > > Not a bit of it. ?Our customers were *also* programmers; her value was > precisely that her perspective was their perspective. ?I did say it was > Prolog; what other kind of person uses a Prolog system? > >> Such people are admittedly as rare as hen's teeth in the more >> quotidian livelihood of tech writing. The Kennitas of that world are >> rare enough. (I think she was only sidelined because of some injury -- >> I'm not sure. I'd kill to make use of her on a project myself, if she >> were still available.) > > "Sidelined"? ?She moved to a bigger company (which is still around, > unlike Quintus) with a larger customer base and more to write about. > I never asked, but they could certainly have _afforded_ to pay her > more than we could. ?Everyone I was close to at Quintus wanted her to > stay. >> >> Which takes me back to my agenda: what's the *practical*, *dirt-cheap* >> way to *eventually* get better quality docs for Erlang/OTP? > > Step 1: ?fix the tool chain so I can build the documents. > > I've annotated the erl_docgen document; there's something that needs > fixing on every page. ?I am keen to send in patches, but I can't do > that until I can build the documents. > >> And their edits can easily be reviewed. > > And now we come to the crux of the matter. ?It's the *review* process. > That takes time and effort and management, and it's not at all clear > to me what, if anything, makes a wiki easier to manage than the current > setup. ?The mere existence of a wiki is no guarantee of good reviewing. > > The great thing about the current scheme is that edits are reviewed > BEFORE they are adopted, but we're promised that they'll be reviewed > quickly. ?With a Wiki, bad stuff goes in, and if you're lucky it will > be reviewed and eventually be taken out, but there _is_ that window > where the bad stuff is there. > > I won't mentioned which Wikipedia page it was, but the one Wikipedia > page I ever did anything to was one which contained several gross > factual errors and had for months. Yep. It happens. Here's one of my recent corrections: http://en.wikipedia.org/w/index.php?title=Greenland_ice_sheet&diff=prev&oldid=495435286 But that's what you get in *Wikipedia*, which is trying to be an encyclopedia for everything, editable by anybody: breathless idiots making order-of-magnitude numerical errors. I've written on this thread SEVERAL TIMES about how the editor membership of any Erlang/OTP wiki could be sharply limited, to help insure good-faith, well-informed edits. Are you EVER going acknowledge that? > But there is also the fact that wikis tend to lead to lots of bits > all over the place and a hell of clicketyclicketyclickety for almost > anything. ?Recently I've been wandering round the Tizen "documentation". > Not a good advertisement for "Web documentation". I'm not talking about wikis that "just grow". I'm talking about a wiki that starts, on day one, with the current Erlang/OTP documentation. That should have been abundantly clear to you by now. If you're still responding to any other scenario, it can only be out of neglect of the arguments I've already made, and perhaps out of a need to get complaints about some *other* wikis off your chest. If I added my *own* complaints about lousy wikis, we'd be here for months. Can we move on? > One thing the current tool chain gets almost right is that it really > does generate man pages and HTML and PDF from the same sources. ?I > have a shelf of printed and bound Erlang manuals. Gosh, it's too bad you can't do that with MediaWiki, and never will be able to, because it's a big mess written by a bunch of retards who would never even think of doing something that useful. I mean, what if you wanted assemble a PDF out of a bunch of Wikipedia articles and make a book? You can't, because ... um ... oh wait: http://en.wikipedia.org/wiki/Help:Books *Sigh*. > Ulf Wiger is not in charge of reviewing changes to the Erlang > documentation. ?The people who _are_ in charge might well be > perfectly happy with adding "Since its first release, seq_trace > has relied on Lamport clocks [ref], but it is not guaranteed > that it will always do so." If they were "perfectly happy" saying that, I'd find it be pretty disturbing. The "Beta" disclaimer speaks only about possible *minor* changes to the *interface*, not any *major* change to core functionality. And Lamport clocks are unmistakably part of the core functionality of seq_trace. Try actually reading the seq_trace documentation, Richard, if you want to weigh in on these issues. > Right now, I'm reading someone complaining about the extreme > difficulty of putting in a sentence and making up a story about > what he'll say to someone when rejected Right now, I'm reading that someone else's ethical imperative -- giving credit where it's due -- is somehow *my* responsibility, rather than that of the providers of the documentation in which credit isn't currently given. And those who maintain the documentation will only accept my contribution if it "improves" the documentation in their eyes -- eyes that seem still prone to see Lamport clocks as an independent, concurrent invention by Ericsson, with no proof of that belief. I think I can be excused my suspicions. >> People will say a lot of silly things about wikis and their governance >> until they gain more direct experience of maintaining them. In that, >> wikis are like anything in life. > > My comments about wikis are based entirely on my experience of actually > trying to get basic information out of wiki "documentation" for several > projects. ?"Like anything in life", wikis obey Sturegon's law. > >>> Wikis do not solve this problem, they exacerbate it. >> >> You're prone to sweeping generalizations about wikis based on no >> direct personal experience of editing them, from what I can tell. > > I have edited one page in one wiki, no wait, three pages in two. > > But editing is simply not the issue. > My comments are based on too much painful experience of trying > to *USE* wikis edited by others. > I don't see why only people who edit wikis count here. > I think that people who *USE* documentation matter, and that > having tried to *use* about a dozen wiki-"documented" open source > projects and finding intense frustration is relevant. > > I find the suggestion that only experienced editors of Wikis are > competent to judge the usability of wikis outrageous. > >>> How high *is* the bar? ?Have you ever submitted a documentation patch >>> and had it rejected? >> >> I've never submitted a documentation patch. > > Ah! ?You've never tried, but you just _know_ that the barrier is high. > >> You hate amateurs. > > No I do not. ?I *am* an amateur. > > Let's go back to Quintus. ?While we had technical writers, > the "engineers" often *originated* documentation. ?It just had to > go through technical writers before customers saw it. ?And it was > the same for code: ?with one notable exception, all changes that > affected what customers saw had to be reviewed before final > acceptance. ?And that one exception was the single dumbest thing > Quintus ever did, and sadly, one of the most copied. > > The chief problem with the existing Erlang documentation is that > it was released without an adequate review of the language. > >> I can understand that. But the experts fail sometimes too. > > It's not professional vs amateur. ?It is review-before-release > vs post-release-review-if-ever. > > By the way, if you want to see an appalling example of editing, > take a look at the ANSI Smalltalk standard. ?The 5th draft was > full of glitches, almost all of which survived in the published > standard. ?Even being-published-by-ANSI was no guarantee of > adequate proof-reading. >> >> If I submitted a patch to the seq_trace documentation so that it >> mentioned that seq_trace implements Lamport clocks, and it got >> rejected with "we're still thinking about how to work that fact in, >> maybe we'll have something in the next release," I think I'd end up >> writing Lamport to say, "Can you *believe* these people?" > > Ulf Wiger is not in charge of reviewing changes to the Erlang > documentation. ?The people who _are_ in charge might well be > perfectly happy with adding "Since its first release, seq_trace > has relied on Lamport clocks [ref], but it is not guaranteed > that it will always do so." > > Right now, I'm reading someone complaining about the extreme > difficulty of putting in a sentence and making up a story about > what he'll say to someone when rejected > > ? ? ? ?WHO HAS NOT TRIED IT. > > Try it. ?*Then* resume this conversation. > > No, wait. ?*I've* just posted a change to erlang-patches. > >> Good ideas, I'm sure. I'm using Erlang to implement a computational >> linguistics framework, so I'm definitely keen on those sorts of >> solutions myself. But automating a task to get "better than nothing" >> shouldn't be the goal here. This is supposed to be documentation by >> professionals, for professionals. > > And professionals use good tools. ?INEX isn't running "link-the-Wiki" > any more because you need to able to compare what you can do with a > "gold standard", and it turns out that while the Wikipedia is > *copiously* linked, it isn't always *well* linked, and when programs > did badly at matching what was there, it was sometimes because the > programs were doing better. > > As for unimplemented EEPs, that comes back to a documentation issue, > namely that the BEAM is not (yet) adequately documented. > From gordon@REDACTED Sat Jun 9 10:09:05 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sat, 9 Jun 2012 09:09:05 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: > Since you do use very large tables, have you thought about something > like Riak, which does handle this fine? We need transactions... Gordon On 8 June 2012 20:47, OvermindDL1 wrote: > On Fri, Jun 8, 2012 at 5:54 AM, Chandru > wrote: >> On 8 June 2012 12:33, Gordon Guthrie wrote: >>> >>> >>> Or mebbies your way is better, simply pass in a fun to the backup >>> traverse to write out the records in tables that I want to bring >>> over... >> >> >> FWIW, this is what we do on very large tables. > > Since you do use very large tables, have you thought about something > like Riak, which does handle this fine? From dmkolesnikov@REDACTED Sat Jun 9 10:21:59 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 9 Jun 2012 11:21:59 +0300 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: <6F18A906-B1D3-4AA8-BC2A-2943C2D7D3AE@gmail.com> References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> <-5124868496751168372@unknownmsgid> <6F18A906-B1D3-4AA8-BC2A-2943C2D7D3AE@gmail.com> Message-ID: Hi, If "trusted gateway" assign the session key both to client & server then this would be a good compromise. You original concern was a memory overhead for SSL. So you still have SSL in picture and it is handled by GW instead of Servers. 2min TTL is not long enough to dramatically reduce a load. I am not sure if you set-up becomes optimal in terms of costs. Have you estimated GW capacity vs Servers? - Dmitry On Jun 8, 2012, at 2:06 PM, Morgan Segalis wrote: > Hi ! > > As I explained answering Angela, Here's my idea for the connection protocol I'm thinking about. > > 1) The client should connect WITH SSL to a gateway (With Certificate and strong Cipher) > 2) Client and Server will negotiate a key together along with a sessionID (not sure we need Diffie-Hellman since it is a secure connection) > 3) The sessionID + key will be stored in a cassandra database (With a TimeToLive of 2min or less) > 4) The server will tell to which node the client should connect > 5) Client disconnects from gateway > 6) Client connects to the node specified by the gateway, and gives the SessionID to the node (in clear) > 7) The Server node retrieve the Key from the cassandra db > 8) If it exist, it sends an accept to the client encrypted, if not, it disconnects the client and we are back to square 1. > 9) Both client & server can communicate over AES using the key they now both have. > > in part 4, the gateway, a trusted SSL server will give the ip of the node the client should connect (encoded of course). This will make the client able to trust the server. > in part 8, the node will accept the client by sending him a specific message, that only a trusted server could have sent. This will make the client able to trust the server. > > I'm confident this protocol is secure, and will allow me to have solid encryption while having a good ratio memory/client. > But I might be wrong? > > Thanks ! > > Le 7 juin 2012 ? 14:29, dmitry kolesnikov a ?crit : > >> Hello, >> >> I guess you can use crypto:dh_compute_key for DH key exchange procedure. >> >> Your application cannot trust server w/o certificates. Therefore >> number of attacks are possible. If you are fine with all related >> threats then yes this is good option to save memory. >> >> On another hand, if you need to use multiple transport e.g. Tcp + udp >> then it is good as well. >> >> Btw, C ssl API allows to perform SSL:read/write directly to memory. It >> would be cool if such feature is available in Erlang. I suppose it >> might help to reduce memory footprint for some applications >> >> Best Regards, >> Dmitry >-|-|-*> >> >> >> On 7.6.2012, at 13.35, Morgan Segalis wrote: >> >>> Hi everyone, >>> >>> As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. >>> >>> I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. >>> Using SSL with this kinda feels like using a bazooka for smashing a fly. >>> >>> The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. >>> >>> Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) >>> >>> I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ >>> >>> Thanks for your help. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > From overminddl1@REDACTED Sat Jun 9 11:06:44 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sat, 9 Jun 2012 03:06:44 -0600 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: On Sat, Jun 9, 2012 at 1:30 AM, Michael Turner wrote: > /* snip all */ Is there not a wiki out there that by default shows a 'published' copy, but lets you see the current head with another click, and lets you make changes to the head, and an authenticated manager/editor/something/special can clean changes and vet it to publish as the new default view, preferably with a list of 'heads that do not match default' page so they know what to look at? From overminddl1@REDACTED Sat Jun 9 11:08:08 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sat, 9 Jun 2012 03:08:08 -0600 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On Sat, Jun 9, 2012 at 2:09 AM, Gordon Guthrie wrote: >> Since you do use very large tables, have you thought about something >> like Riak, which does handle this fine? > > We need transactions... A single process handling a specific set of rows would handle that in most cases? What are the transactions needed for? No way to work around? From janburse@REDACTED Sat Jun 9 11:27:15 2012 From: janburse@REDACTED (Jan Burse) Date: Sat, 09 Jun 2012 11:27:15 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: <4FD316F3.7070401@fastmail.fm> Michael Turner schrieb: > The usability of a*given* wiki? Or the fitness-for-purpose of wiki > software for helping maintain Erlang/OTP documentation, from an > administrative point of view? It's only the latter issues that concern > me here, and it's on those issues that you keep saying things I find > either incomprehensible or flatly wrong. There is an further issues with client-server based collaborative tools such as a wiki. You need to have all the time a working internet connection. This is ok if you are doing work from a location with a stable and fast connection. But sometimes the check-out and check-in model is superiour to the client-server based model. Especially if you have intervalls where you don't need a lot of interaction with peers and/or where connectivity is not possible. Since all your artefacts are local after check-out you will also experience a much faster responsiveness of your tools. Since one is allowed a smoother editing one usually has a higher productivity. Also before check-in again changes are often automatically compressed. There are a couple of integrated development environments either free or for a few bucks that integrate well with version management, and that also provide graphical diff tools. These diff tools can be used for many things like merge conflict resolution, change reviews, change promotion, etc.. Here is a typical diff view: http://codingbone.files.wordpress.com/2010/11/picture-2.png I am currently involved in changing a wiki solution with a Prolog driven automated kind of proof reading into a check-in and check-out solution. Most probably the outcome will be a hybrid, but in the future the real editing will be done kind of offline. And the proof reading load will be done locally and not anymore on the server, resulting in much more speed. But prerequisit for all this is also a kind of locality respectively predicatabilty of the involved data, i.e. pages. Low frequency higher grain check-out is only possible if one can predict what module one will need for a particular task at hand. In a memex like environment where one is about to follow unpredictably links, this is difficult. On the other hand in a hybrid solution, the wiki could still be used to browse accidential links. And I guess in a structured work environment it will be also possible to delineate modules. But I am not yet sure how well things will work. Problem are always highly shared modules which have a lot of updates. Check-out and check-in can also slow down an organization. But good automated merging might help here. Bye From desired.mta@REDACTED Sat Jun 9 11:42:41 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Sat, 9 Jun 2012 11:42:41 +0200 Subject: [erlang-questions] Suggestions for an Erlang Opensource Project to Work On In-Reply-To: References: Message-ID: <20120609094241.GB5175@localhost> On Sat, Jun 09, 2012 at 10:14:45AM +0530, Anoop Thomas Mathew wrote: > Hi All, > > I'm a beginner in erlang, and want to work on some opensource projects. > Can some one suggest a few projects, which can be handled by beginners (And > more specifically, a developer group where beginner's are welcome). magnet links support Etorrent! Motiejus From essen@REDACTED Sat Jun 9 11:43:24 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 09 Jun 2012 11:43:24 +0200 Subject: [erlang-questions] Suggestions for an Erlang Opensource Project to Work On In-Reply-To: References: Message-ID: <4FD31ABC.4070409@ninenines.eu> A common API for accessing many different RDBMS (MySQL, PostgreSQL, SQLite would be a good start). Can use existing drivers. Would be nice if it didn't have a required pool manager so that it could be done by the user's application. An image manipulation API. Can probably be done as a NIF for the most part. Various APIs to connect to webservices (Amazon, Facebook, Twitter...). On 06/09/2012 06:44 AM, Anoop Thomas Mathew wrote: > Hi All, > > I'm a beginner in erlang, and want to work on some opensource projects. > Can some one suggest a few projects, which can be handled by beginners > (And more specifically, a developer group where beginner's are welcome). > > Thanks, > Anoop Thomas Mathew > > atm > ___ > Life is short, Live it hard. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From msegalis@REDACTED Sat Jun 9 12:47:00 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Sat, 9 Jun 2012 12:47:00 +0200 Subject: [erlang-questions] Using my own Diffie-Hellman + AES instead of SSL In-Reply-To: References: <4FD65630-E8C0-4E00-A6F2-7DA269ADE9B3@gmail.com> <-5124868496751168372@unknownmsgid> <6F18A906-B1D3-4AA8-BC2A-2943C2D7D3AE@gmail.com> Message-ID: <00AA456D-A3CF-4B74-9ED1-EC19042A9535@gmail.com> Hi, Actually, the TTL is just here (on the cassandra column) waiting for the client to disconnect from the GW and connect to the server the gateway specified to it. If the client is too long connecting to the server, the pair SessionID/Key will be deleted from the Cassandra database and server will refuse the connection, the client will have to restart the connection protocol from the start and go ask for a SessionID/Key to the GW. Since the client will not be connected more than +/- 1 sec. (Just long enough to SSL handshake with GW, get SessionID/Key pair and the server's ip then disconnect) to the GW, the SSL overhead will not be an issue. Thanks, Morgan. Le 9 juin 2012 ? 10:21, Dmitry Kolesnikov a ?crit : > Hi, > > If "trusted gateway" assign the session key both to client & server then this would be a good compromise. > > You original concern was a memory overhead for SSL. So you still have SSL in picture and it is handled by GW instead of Servers. 2min TTL is not long enough to dramatically reduce a load. I am not sure if you set-up becomes optimal in terms of costs. Have you estimated GW capacity vs Servers? > > - Dmitry > > On Jun 8, 2012, at 2:06 PM, Morgan Segalis wrote: > >> Hi ! >> >> As I explained answering Angela, Here's my idea for the connection protocol I'm thinking about. >> >> 1) The client should connect WITH SSL to a gateway (With Certificate and strong Cipher) >> 2) Client and Server will negotiate a key together along with a sessionID (not sure we need Diffie-Hellman since it is a secure connection) >> 3) The sessionID + key will be stored in a cassandra database (With a TimeToLive of 2min or less) >> 4) The server will tell to which node the client should connect >> 5) Client disconnects from gateway >> 6) Client connects to the node specified by the gateway, and gives the SessionID to the node (in clear) >> 7) The Server node retrieve the Key from the cassandra db >> 8) If it exist, it sends an accept to the client encrypted, if not, it disconnects the client and we are back to square 1. >> 9) Both client & server can communicate over AES using the key they now both have. >> >> in part 4, the gateway, a trusted SSL server will give the ip of the node the client should connect (encoded of course). This will make the client able to trust the server. >> in part 8, the node will accept the client by sending him a specific message, that only a trusted server could have sent. This will make the client able to trust the server. >> >> I'm confident this protocol is secure, and will allow me to have solid encryption while having a good ratio memory/client. >> But I might be wrong? >> >> Thanks ! >> >> Le 7 juin 2012 ? 14:29, dmitry kolesnikov a ?crit : >> >>> Hello, >>> >>> I guess you can use crypto:dh_compute_key for DH key exchange procedure. >>> >>> Your application cannot trust server w/o certificates. Therefore >>> number of attacks are possible. If you are fine with all related >>> threats then yes this is good option to save memory. >>> >>> On another hand, if you need to use multiple transport e.g. Tcp + udp >>> then it is good as well. >>> >>> Btw, C ssl API allows to perform SSL:read/write directly to memory. It >>> would be cool if such feature is available in Erlang. I suppose it >>> might help to reduce memory footprint for some applications >>> >>> Best Regards, >>> Dmitry >-|-|-*> >>> >>> >>> On 7.6.2012, at 13.35, Morgan Segalis wrote: >>> >>>> Hi everyone, >>>> >>>> As always (for those who remember my old posts), I'm looking to reduce the memory footprint of a secure connection to my server. >>>> >>>> I'm right now looking into doing my own Diffie-Hellman key exchange + AES 128-bit encryption over a TCP connection instead of using Erlang SSL. >>>> Using SSL with this kinda feels like using a bazooka for smashing a fly. >>>> >>>> The thing is, I do not really need certificates, since the connection url is embedded in a application, and it is using a proprietary protocol. >>>> >>>> Before trying to implement this solution, I would like to hear pros and cons, from Erlang experts on this mailing lists ! (others are welcome too of course ;-) ) >>>> >>>> I'm actually looking to implement the same thing as here http://blog.diginux.net/2006/11/15/adding-aes-encryption-to-erlang-chat/ >>>> >>>> Thanks for your help. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> > From dafgar@REDACTED Sat Jun 9 14:58:37 2012 From: dafgar@REDACTED (Sergei Fomin) Date: Sat, 09 Jun 2012 16:58:37 +0400 Subject: [erlang-questions] enotconn from gen_tcp socket In-Reply-To: References: <4FD1FF85.7010500@yandex-team.ru> Message-ID: <4FD3487D.6050400@yandex-team.ru> Unfortunaly no, when I close connection correctly (i.e. fyn->, fyn_ack<-, ack->) and then do inet:peername, it still returns {ok, IP}. And yes, I have turned off exit_on_close socket option. Attila Rajmund Nohl wrote: > 2012/6/8 Sergei Fomin: >> Hello. >> >> There is a server which gen_tcp:accept new connections and immediately logs >> inet:peername. But still, "Accepted {error,enotconn}" can be seen in the log >> files. One possible reason of such error is described in >> http://erlang.org/pipermail/erlang-questions/2012-May/066802.html , i.e. a >> client does 3-way handshake and sends RST packet immediately. Though, this >> scenario seems uncommon. >> >> Wonder, what other causes can lead to enotconn on just opened socket? > > I think it doesn't have to be an RST, a normal connection closing > would also lead to this error. I've seen a very similar error a couple > of months ago when the OTP SFTP client did something similar as you, > i.e. it had a > > {ok, PeerAddr} = inet:peername(Socket), > > call soon after the Socket was created and sometimes this crashed with > a badmatch. The SFTP client has been fixed in R15B01, so there's a > case and error handling instead of the matching. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sergei Fomin Yandex development team http://staff.yandex-team.ru/dafgar From michael.eugene.turner@REDACTED Sat Jun 9 15:20:00 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 22:20:00 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: On Sat, Jun 9, 2012 at 6:06 PM, OvermindDL1 wrote: > On Sat, Jun 9, 2012 at 1:30 AM, Michael Turner > wrote: >> /* snip all */ > > Is there not a wiki out there that by default shows a 'published' > copy, but lets you see the current head with another click, and lets > you make changes to the head, and an authenticated > manager/editor/something/special can clean changes and vet it to > publish as the new default view, preferably with a list of 'heads that > do not match default' page so they know what to look at? I haven't looked. But here's how to approach such questions: wiki software is software. So ask yourself, "Could software do this?" If the answer is "yes, then not-too-surprisingly often, there is already a wiki (or an extension thereof) that does it. One reason why the idea might not be implemented: it isn't actually a good idea. In this case, it might not be. Wikis improve by evolution of a *current* version. They are more likely to improve if the current version is the most frequently seen version. If the default page for any article is instead the "published" copy, rather than the true *current* version, and if users need to notice and click a special link to get any contributed updates to that copy, you are asking volunteers to do more work and giving them more frustration. People like me who often want to fix errors will see them in the "published" and locked-down copy, and click through to the wikified version to navigate to the point of error, only to discover, all too often, that somebody has already corrected it. Better to put current version first, with a disclaimer, and make the "release version" on which it's based easily available through a prominently featured link. -michael turner From carlsson.richard@REDACTED Sat Jun 9 15:48:19 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 09 Jun 2012 15:48:19 +0200 Subject: [erlang-questions] Data structures are strange things In-Reply-To: <4fec23e3-fdd9-44e4-8453-836d734c4f66@knuth> References: <4fec23e3-fdd9-44e4-8453-836d734c4f66@knuth> Message-ID: <4FD35423.8000304@gmail.com> On 06/09/2012 01:51 AM, Robert Virding wrote: > No, it is not surprising when you think about it. I just found it a bit surprising that the conventions which seem to apply to imperative languages have been more less imported into the functional/declarative world where they don't apply to the same degree. It seems that in the functional world going "red-black" seems to be accepted even though there aren't many benefits of it. The code is a little simpler, but for 2-3 trees the difference is very small. And it is faster for insertion and deletion. > > ----- Original Message ----- >> If you take a look at a typical 2-3-4 tree implementation in Java >> http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java >>>> http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java >> you'll see nodes containing >> >> an integer saying how many keys there are >> three keys >> four links to children nodes >> a link to a parent node >> >> Allow one word for "what is the class of this object" and that's >> 10 words per node. >> >> Using Erlang tuples, we'd have >> {K1,C0,C1} 4 words >> {K1,K2,C0,C1,C2} 6 words >> {K1,K2,K3,C0,C1,C2,C3} 8 words One advantage the General Balanced trees have compared to RB-trees and AVL trees is that they don't need any additional information per node (like a colour or size). Each GB-tree node is {Key, Value, Smaller, Bigger} (5 words), and adding an extra field would mean 6 words, a 20% increase both in memory usage and in words written per node update. The code is also very straightforward, since there are no special cases anywhere, only a check for rebalancing. It makes sense that 2-3-(4) trees could perform even better, since they reduce the height of the tree and can effectively use the size of the Erlang tuple as the additional balancing information as Robert and ROK noted. 2-3 trees are just the simplest case of B-trees, so all leaves are at the same depth. (It's balancing on deletion that should be the tricky part.) Our experiences from implementing the array module indicate that the optimal node size for a tree in Erlang is around 8-10 elements, so the most efficient tree implementation would probably be something like a 2-3-...-8 tree with everything unrolled. :-) (You could of course generate the code from a spec.) /Richard From michael.eugene.turner@REDACTED Sat Jun 9 15:54:29 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 22:54:29 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <4FD316F3.7070401@fastmail.fm> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <4FD316F3.7070401@fastmail.fm> Message-ID: On Sat, Jun 9, 2012 at 6:27 PM, Jan Burse wrote: > Michael Turner schrieb: >> >> The usability of a*given* ?wiki? Or the fitness-for-purpose of wiki >> software for helping maintain Erlang/OTP documentation, from an >> administrative point of view? It's only the latter issues that concern >> me here, and it's on those issues that you [Richard O'Keefe] >> keep saying things I find either incomprehensible or flatly wrong. > > > There is an further issues with client-server based > collaborative tools such as a wiki. You need to have > all the time a working internet connection. [snip] I haven't done offline wiki editing. However, as with almost any idea that seems both possible in software and a good idea, there are solutions, at least in the case of MediaWiki. Here's what looks like a relatively simple one, a project started all the way back in 2004: http://search.cpan.org/~markj/WWW-Mediawiki-Client/bin/mvs Other packages listed here http://en.wikipedia.org/wiki/Wikipedia:Text_editor_support might provide offline support. I don't know. -michael turner From michael.eugene.turner@REDACTED Sat Jun 9 16:03:56 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 9 Jun 2012 23:03:56 +0900 Subject: [erlang-questions] Data structures are strange things In-Reply-To: <4FD35423.8000304@gmail.com> References: <4fec23e3-fdd9-44e4-8453-836d734c4f66@knuth> <4FD35423.8000304@gmail.com> Message-ID: I suggest *artificially* increasing the node sizes, and seeing how well the results hold up. I suspect the differences in performance are strongly dominated by the differences between distance-from-update-point-to-root, with differences in memory consumption being a second-order effect. -michael turner On Sat, Jun 9, 2012 at 10:48 PM, Richard Carlsson wrote: > On 06/09/2012 01:51 AM, Robert Virding wrote: >> >> No, it is not surprising when you think about it. I just found it a bit >> surprising that the conventions which seem to apply to imperative languages >> have been more less imported into the functional/declarative world where >> they don't apply to the same degree. It seems that in the functional world >> going "red-black" seems to be accepted even though there aren't many >> benefits of it. The code is a little simpler, but for 2-3 trees the >> difference is very small. And it is faster for insertion and deletion. >> >> ----- Original Message ----- >>> >>> If you take a look at a typical 2-3-4 tree implementation in Java >>> ? ? http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234.java >>>>> >>>>> ?http://www.cs.berkeley.edu/~jrs/61bs02/hw/hw7/dict/Tree234Node.java >>> >>> you'll see nodes containing >>> >>> ? ? ? ?an integer saying how many keys there are >>> ? ? ? ?three keys >>> ? ? ? ?four links to children nodes >>> ? ? ? ?a link to a parent node >>> >>> Allow one word for "what is the class of this object" and that's >>> 10 words per node. >>> >>> Using Erlang tuples, we'd have >>> ? ? ? ?{K1,C0,C1} ? ? ? ? ? ? ? ? ? ? ?4 words >>> ? ? ? ?{K1,K2,C0,C1,C2} ? ? ? ? ? ? ? ?6 words >>> ? ? ? ?{K1,K2,K3,C0,C1,C2,C3} ? ? ? ? ?8 words > > > One advantage the General Balanced trees have compared to RB-trees and AVL > trees is that they don't need any additional information per node (like a > colour or size). Each GB-tree node is {Key, Value, Smaller, Bigger} (5 > words), and adding an extra field would mean 6 words, a 20% increase both in > memory usage and in words written per node update. The code is also very > straightforward, since there are no special cases anywhere, only a check for > rebalancing. > > It makes sense that 2-3-(4) trees could perform even better, since they > reduce the height of the tree and can effectively use the size of the Erlang > tuple as the additional balancing information as Robert and ROK noted. 2-3 > trees are just the simplest case of B-trees, so all leaves are at the same > depth. (It's balancing on deletion that should be the tricky part.) > > Our experiences from implementing the array module indicate that the optimal > node size for a tree in Erlang is around 8-10 elements, so the most > efficient tree implementation would probably be something like a 2-3-...-8 > tree with everything unrolled. :-) (You could of course generate the code > from a spec.) > > ? /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Sat Jun 9 16:13:28 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 9 Jun 2012 16:13:28 +0200 Subject: [erlang-questions] socket unrecv Message-ID: gen_tcp has the undocumented function `unrecv/2` which is using `inet(6)_tcp:unrecv/2` but I don't see the equivalent in the SSL module. Is there any way in the standard library to fill a buffer in the socket ? My use case is the following: I have an api that read some data from the socket then pase it and keeps the rest in a buffer. Now I want to use this buffer with another API that only read from the socket. I could of course make a change in this API to read the data from a buffer if it`s available, but since I don't control this API it will make some time to have this change included. So rather to say to others to use my own patched API in between, I could use this trick. So if there is a way to put back the data in the socket pid ? - beno?t From a.shneyderman@REDACTED Sat Jun 9 23:42:11 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sat, 9 Jun 2012 23:42:11 +0200 Subject: [erlang-questions] xmerl troubles Message-ID: Hi, there! I am using xmerl (via xmlrpc) and it seems to be having problems under heavy load. I have a system that tries to simulate load on our system. I have to shoot a lot of little XML RPC-like messages towards the system and parse responses from it. Each simulation runs in its own process. Parsing of the payload is done with xmlrpc which in turn uses xmerl. When load on the system is small around 250 agents, things work well. When I up the number pseudo-agents to around 290 xmerl starts throwing a hissy fit. A message that is similar in all respects but the data inside elements to the previously parsed messages become problematic. Code is nothing extra-ordinary and looks like this: case xmlrpc_decode:payload(XmlStr) of {ok, {response, [{struct, ResponseParams}]}}-> lager:debug("XML-RPC payload parsed:~n {response, [{struct, ~p}]} ~n", [ResponseParams]), {ok,{login_response,ResponseParams},[RestOfBinary]}; {error, Reason} -> lager:debug("XML-RPC payload could not be parsed:~n~p~n", [Reason]), {error,Reason} end; the exception I get: 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload could not be parsed: {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147 Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82' talking on local port 64001 Anyone had similar experience with xmerl? Is there any setting I can tweak to make xmerl behave nicely? Thanks, Alex. From cgsmcmlxxv@REDACTED Sat Jun 9 23:49:10 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Sat, 9 Jun 2012 23:49:10 +0200 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: Have you tried http://www.erlang.org/doc/man/erl.html for erl options? CGS On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane wrote: > Where may I look further for the basic understanding of spatio-temporal > relations among the "things" named as ERTS, Emulator and Shell? The > question occurred while I was toying with "erl" as Unix-filter > shell-command and couldn't get over the options (to erl) like "-noshell", > "-s", "-detached", init:stop(), and so on in the documentation and in the > FAQ. > > Thanks > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Sun Jun 10 00:04:28 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Sun, 10 Jun 2012 00:04:28 +0200 Subject: [erlang-questions] xmerl troubles In-Reply-To: References: Message-ID: Hi Alex, The problem may be also coming from the network transfer, not only from xmlerl/xmlrpc. From the error you provided, {{badmatch,{error,emfile}}, it looks like the XML file wasn't correctly transmitted/received and that may come from traffic congestion as well. Of course, that if the XML file exists and it contains no error. CGS On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman wrote: > Hi, there! > > I am using xmerl (via xmlrpc) and it seems to be having problems under > heavy load. > > I have a system that tries to simulate load on our system. I have to > shoot a lot of little > XML RPC-like messages towards the system and parse responses from it. > Each simulation > runs in its own process. Parsing of the payload is done with xmlrpc > which in turn uses xmerl. > > When load on the system is small around 250 agents, things work well. > When I up the number > pseudo-agents to around 290 xmerl starts throwing a hissy fit. A > message that is similar in > all respects but the data inside elements to the previously parsed > messages become problematic. > > Code is nothing extra-ordinary and looks like this: > > case xmlrpc_decode:payload(XmlStr) of > {ok, {response, [{struct, ResponseParams}]}}-> > lager:debug("XML-RPC payload parsed:~n {response, [{struct, > ~p}]} > ~n", [ResponseParams]), > {ok,{login_response,ResponseParams},[RestOfBinary]}; > {error, Reason} -> > lager:debug("XML-RPC payload could not be parsed:~n~p~n", > [Reason]), > {error,Reason} > end; > > the exception I get: > > 2012-06-09 23:02:25.037 [debug] > <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload > could not be parsed: > > {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147 > Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82' > talking on local port 64001 > > Anyone had similar experience with xmerl? Is there any setting I can > tweak to make xmerl behave nicely? > > Thanks, > Alex. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Sun Jun 10 00:24:23 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sun, 10 Jun 2012 00:24:23 +0200 Subject: [erlang-questions] xmerl troubles In-Reply-To: References: Message-ID: I am logging problematic XML (there are no files involved). And diffing it with the XML received in one of the first parses (that actually parse fine) does not reveal significant difference in structure, all match except for but internal data within the elements. If I leave the number of agents at around 290 system works for hours. Parsing messages back and forth. 290 seems to be my threshold which is too low for my experiments :-( Thanks. PS: CGS, sorry for the duplicate my first reply was not to all :-( On Sun, Jun 10, 2012 at 12:04 AM, CGS wrote: > Hi Alex, > > The problem may be also coming from the network transfer, not only from > xmlerl/xmlrpc. From the error you provided,?{{badmatch,{error,emfile}}, it > looks like the XML file wasn't correctly transmitted/received and that may > come from traffic congestion as well. Of course, that if the XML file exists > and it contains no error. > > CGS > > > > > On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman > wrote: >> >> Hi, there! >> >> I am using xmerl (via xmlrpc) and it seems to be having problems under >> heavy load. >> >> I have a system that tries to simulate load on our system. I have to >> shoot a lot of little >> XML RPC-like messages towards the system and parse responses from it. >> Each simulation >> runs in its own process. Parsing of the payload is done with xmlrpc >> which in turn uses xmerl. >> >> When load on the system is small around 250 agents, things work well. >> When I up the number >> pseudo-agents to around 290 xmerl starts throwing a hissy fit. A >> message that is similar in >> all respects but the data inside elements to the previously parsed >> messages become problematic. >> >> Code is nothing extra-ordinary and looks like this: >> >> case xmlrpc_decode:payload(XmlStr) of >> ? ? ? ?{ok, {response, [{struct, ResponseParams}]}}-> >> ? ? ? ? ? ? ? ?lager:debug("XML-RPC payload parsed:~n {response, [{struct, >> ~p}]} >> ~n", [ResponseParams]), >> ? ? ? ? ? ? ? ?{ok,{login_response,ResponseParams},[RestOfBinary]}; >> ? ? ? ?{error, Reason} -> >> ? ? ? ? ? ? ? ?lager:debug("XML-RPC payload could not be parsed:~n~p~n", >> [Reason]), >> ? ? ? ? ? ? ? ?{error,Reason} >> end; >> >> the exception I get: >> >> 2012-06-09 23:02:25.037 [debug] >> <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload >> could not be parsed: >> >> {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} >> 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147 >> Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82' >> talking on local port 64001 >> >> Anyone had similar experience with xmerl? Is there any setting I can >> tweak to make xmerl behave nicely? >> >> Thanks, >> Alex. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From cgsmcmlxxv@REDACTED Sun Jun 10 00:27:38 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Sun, 10 Jun 2012 00:27:38 +0200 Subject: [erlang-questions] socket unrecv In-Reply-To: References: Message-ID: Hi Benoit, Forgive my lack of imagination, but how do you manage to listen to the same socket with another process? Can you pass the socket from the other API? Moreover, how do you make the other API to listen after your one without putting it in front of the other API? If you can access and pass the socket from the other API, I think you can bounce back the data toward the same socket (gen_tcp:send/2) because then the other API should receive the same data as the original transmitter (hopefully, the other side won't crash because it doesn't expect reply). But, still, this is only in the case you can receive the packets in the same time with the other API and the other API trashes the original packets (otherwise you just double the data in the other API). CGS On Sat, Jun 9, 2012 at 4:13 PM, Benoit Chesneau wrote: > gen_tcp has the undocumented function `unrecv/2` which is using > `inet(6)_tcp:unrecv/2` but I don't see the equivalent in the SSL > module. > > Is there any way in the standard library to fill a buffer in the socket ? > > My use case is the following: I have an api that read some data from > the socket then pase it and keeps the rest in a buffer. Now I want to > use this buffer with another API that only read from the socket. I > could of course make a change in this API to read the data from a > buffer if it`s available, but since I don't control this API it will > make some time to have this change included. So rather to say to > others to use my own patched API in between, I could use this trick. > > So if there is a way to put back the data in the socket pid ? > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Sun Jun 10 00:35:01 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sat, 9 Jun 2012 23:35:01 +0100 Subject: [erlang-questions] Mnesia Won't Start Message-ID: Folks I have a problem restarting mnesia - it is hanging on loading a particular table. I start it with -mnesia debug debug and it works through the tables until it gets to one: Mnesia('hnlive@REDACTED'): Getting table 'pk46.tiny.hn&80&item' (disc_copies) from disc: local_only Mnesia('hnlive@REDACTED'): Getting table 'pk46.tiny.hn&80&kvstore' (disc_only_copies) from disc: local_only Mnesia('hnlive@REDACTED'): Getting table 'pk46.tiny.hn&80&local_obj' (disc_copies) from disc: local_only At this point CPU goes to 100% and it just hangs. The table is quite large with 3 secondary indices on it. The table is running on a single node. I don't really know how to progress here - any thoughts appreciated. Gordon From bob@REDACTED Sun Jun 10 00:45:18 2012 From: bob@REDACTED (Bob Ippolito) Date: Sat, 9 Jun 2012 15:45:18 -0700 Subject: [erlang-questions] xmerl troubles In-Reply-To: References: Message-ID: Are you using Mac OS X? The ulimit probably needs to be adjusted to accommodate the number of sockets you want to open. The default is notoriously low. In any case, the error that's happening is of course not in the code you've shared. On Saturday, June 9, 2012, Alex Shneyderman wrote: > I am logging problematic XML (there are no files involved). And diffing it > with the XML received in one of the first parses (that actually parse fine) > does not reveal significant difference in structure, all match except for > but > internal data within the elements. > > If I leave the number of agents at around 290 system works for hours. > Parsing > messages back and forth. 290 seems to be my threshold which is too low for > my > experiments :-( > > Thanks. > > PS: CGS, sorry for the duplicate my first reply was not to all :-( > > On Sun, Jun 10, 2012 at 12:04 AM, CGS > > wrote: > > Hi Alex, > > > > The problem may be also coming from the network transfer, not only from > > xmlerl/xmlrpc. From the error you provided, {{badmatch,{error,emfile}}, > it > > looks like the XML file wasn't correctly transmitted/received and that > may > > come from traffic congestion as well. Of course, that if the XML file > exists > > and it contains no error. > > > > CGS > > > > > > > > > > On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman < > a.shneyderman@REDACTED > > > wrote: > >> > >> Hi, there! > >> > >> I am using xmerl (via xmlrpc) and it seems to be having problems under > >> heavy load. > >> > >> I have a system that tries to simulate load on our system. I have to > >> shoot a lot of little > >> XML RPC-like messages towards the system and parse responses from it. > >> Each simulation > >> runs in its own process. Parsing of the payload is done with xmlrpc > >> which in turn uses xmerl. > >> > >> When load on the system is small around 250 agents, things work well. > >> When I up the number > >> pseudo-agents to around 290 xmerl starts throwing a hissy fit. A > >> message that is similar in > >> all respects but the data inside elements to the previously parsed > >> messages become problematic. > >> > >> Code is nothing extra-ordinary and looks like this: > >> > >> case xmlrpc_decode:payload(XmlStr) of > >> {ok, {response, [{struct, ResponseParams}]}}-> > >> lager:debug("XML-RPC payload parsed:~n {response, > [{struct, > >> ~p}]} > >> ~n", [ResponseParams]), > >> {ok,{login_response,ResponseParams},[RestOfBinary]}; > >> {error, Reason} -> > >> lager:debug("XML-RPC payload could not be parsed:~n~p~n", > >> [Reason]), > >> {error,Reason} > >> end; > >> > >> the exception I get: > >> > >> 2012-06-09 23:02:25.037 [debug] > >> <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload > >> could not be parsed: > >> > >> > {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > >> 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147 > >> Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82' > >> talking on local port 64001 > >> > >> Anyone had similar experience with xmerl? Is there any setting I can > >> tweak to make xmerl behave nicely? > >> > >> Thanks, > >> Alex. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Sun Jun 10 01:06:22 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sun, 10 Jun 2012 00:06:22 +0100 Subject: [erlang-questions] Mnesia Won't Start In-Reply-To: References: Message-ID: If I leave it long enough it runs out of memory eheap_alloc: Cannot allocate 97953500 bytes of memory (of type "old_heap"). Gordon On 9 June 2012 23:35, Gordon Guthrie wrote: > Folks > > I have a problem restarting mnesia - it is hanging on loading a > particular table. > > I start it with -mnesia debug debug and it works through the tables > until it gets to one: > > Mnesia('hnlive@REDACTED'): Getting table > 'pk46.tiny.hn&80&item' (disc_copies) from disc: local_only > Mnesia('hnlive@REDACTED'): Getting table > 'pk46.tiny.hn&80&kvstore' (disc_only_copies) from disc: local_only > Mnesia('hnlive@REDACTED'): Getting table > 'pk46.tiny.hn&80&local_obj' (disc_copies) from disc: local_only > > At this point CPU goes to 100% and it just hangs. > > The table is quite large with 3 secondary indices on it. > > The table is running on a single node. > > I don't really know how to progress here - any thoughts appreciated. > > Gordon From a.shneyderman@REDACTED Sun Jun 10 06:57:21 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sun, 10 Jun 2012 06:57:21 +0200 Subject: [erlang-questions] xmerl troubles In-Reply-To: References: Message-ID: Thanks, Bob. That indeed was the problem. I upped the limit to 8K and all is working now. Odd that the error I am getting has nothing to do with the problem at hand :-( Cheers, Alex. On Sun, Jun 10, 2012 at 12:45 AM, Bob Ippolito wrote: > Are you using Mac OS X? The ulimit probably needs to be adjusted to > accommodate the number of sockets you want to open. The default is > notoriously low. > From avinash@REDACTED Sun Jun 10 07:17:45 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Sun, 10 Jun 2012 10:47:45 +0530 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: Of course, that's where I came from. Not that I couldn't employ those options to work fruitfully for the immediate tasks at hand (towards this end, the online Erlang documentation is adequately equipped) but I am trying to get through them into the machinery that stands behind them - what that machinery is, how it works, and the most importantly, why it is like what it is. This understanding is indispensable for my self-adaption to Erlang for solving customers' problems. I guess every programmer-to-be (at various levels) of Erlang will have this need. Right now, we have Joe's thesis at one end of the continuum and the (open) source code at the end. The 3 books are very good (and I have them all) but the desire to know, understand and apply is insatiable! :-) On Sun, 10 Jun 2012 03:19:10 +0530, CGS wrote: > Have you tried http://www.erlang.org/doc/man/erl.html for erl options? > > CGS > > > > > On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane > wrote: > >> Where may I look further for the basic understanding of spatio-temporal >> relations among the "things" named as ERTS, Emulator and Shell? The >> question occurred while I was toying with "erl" as Unix-filter >> shell-command and couldn't get over the options (to erl) like >> "-noshell", >> "-s", "-detached", init:stop(), and so on in the documentation and in >> the >> FAQ. >> >> Thanks >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions From cgsmcmlxxv@REDACTED Sun Jun 10 11:53:17 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Sun, 10 Jun 2012 11:53:17 +0200 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: Then, maybe the next step for you should be to read some books about how Linux manages processes, pipelines and so on so forth. CGS On Sun, Jun 10, 2012 at 7:17 AM, Avinash Dhumane wrote: > Of course, that's where I came from. > > Not that I couldn't employ those options to work fruitfully for the > immediate tasks at hand (towards this end, the online Erlang documentation > is adequately equipped) but I am trying to get through them into the > machinery that stands behind them - what that machinery is, how it works, > and the most importantly, why it is like what it is. This understanding is > indispensable for my self-adaption to Erlang for solving customers' > problems. I guess every programmer-to-be (at various levels) of Erlang will > have this need. > > Right now, we have Joe's thesis at one end of the continuum and the (open) > source code at the end. The 3 books are very good (and I have them all) but > the desire to know, understand and apply is insatiable! :-) > > > On Sun, 10 Jun 2012 03:19:10 +0530, CGS wrote: > > Have you tried http://www.erlang.org/doc/man/**erl.htmlfor erl options? >> >> CGS >> >> >> >> >> On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane > >wrote: >> >> Where may I look further for the basic understanding of spatio-temporal >>> relations among the "things" named as ERTS, Emulator and Shell? The >>> question occurred while I was toying with "erl" as Unix-filter >>> shell-command and couldn't get over the options (to erl) like "-noshell", >>> "-s", "-detached", init:stop(), and so on in the documentation and in the >>> FAQ. >>> >>> Thanks >>> ______________________________****_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/****listinfo/erlang-questions >>> >>> > >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Sun Jun 10 12:47:20 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sun, 10 Jun 2012 11:47:20 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: > A single process handling a specific set of rows would handle that in > most cases? ?What are the transactions needed for? ?No way to work > around? You need to co-ordinate the update to several tables and roll back and restore on failure - either use a TM or write one - better not to write your own.. Gordon On 9 June 2012 10:08, OvermindDL1 wrote: > On Sat, Jun 9, 2012 at 2:09 AM, Gordon Guthrie wrote: >>> Since you do use very large tables, have you thought about something >>> like Riak, which does handle this fine? >> >> We need transactions... > > A single process handling a specific set of rows would handle that in > most cases? ?What are the transactions needed for? ?No way to work > around? From gordon@REDACTED Sun Jun 10 12:54:25 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sun, 10 Jun 2012 11:54:25 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: > Or mebbies your way is better, simply pass in a fun to the backup > traverse to write out the records in tables that I want to bring > over... That's turned out to be a scoosh, once you have worked out that the default backup module is called mnesia_backup - which the doco's don't say anywhere :( Cheers folks, just need to get mnesia starting again (but that's a matter for the other thread...) Gordon On 8 June 2012 12:33, Gordon Guthrie wrote: > Dan > > I think what I will do is create a second node on the target system, > restore to that, then replicate/copy the tables across... > > Then I can take down the restored db node. > > Or mebbies your way is better, simply pass in a fun to the backup > traverse to write out the records in tables that I want to bring > over... > > Gordon > > On 8 June 2012 12:30, Dan Gudmundsson wrote: >> On Fri, Jun 8, 2012 at 1:11 PM, Gordon Guthrie wrote: >>> So essentially there is no way to selectively restore large tables? >> >> Nope, you will have to restart from a complete system backup. >> >> Or you can write if you do something dirty, like take the system >> offline, down to single node, >> traverse the backup and and lot of dirty_writes. From cgsmcmlxxv@REDACTED Sun Jun 10 13:35:06 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Sun, 10 Jun 2012 13:35:06 +0200 Subject: [erlang-questions] xmerl troubles In-Reply-To: References: Message-ID: Actually, it does. ERL_MAX_PORTS environment variable and +P option act on the number of file descriptors as well. {{badmatch,{error,emfile}} says that the file couldn't be opened or it was opened with errors. It was my bad I didn't think of the file descriptors as well, but I thought 290 FD's is incredibly low (I knew 1024 value is the default for Erlang on every OS). But, thanks to Bob, the problem got solved. Well, my lack of experience with Mac OS X had its fingerprint in my judgement. Sorry for that. Good luck! CGS On Sun, Jun 10, 2012 at 6:57 AM, Alex Shneyderman wrote: > Thanks, Bob. That indeed was the problem. I upped the limit to 8K and > all is working now. Odd that the error I am getting has nothing to do > with the problem at hand :-( > > Cheers, > Alex. > > On Sun, Jun 10, 2012 at 12:45 AM, Bob Ippolito wrote: > > Are you using Mac OS X? The ulimit probably needs to be adjusted to > > accommodate the number of sockets you want to open. The default is > > notoriously low. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sun Jun 10 19:52:23 2012 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sun, 10 Jun 2012 10:52:23 -0700 Subject: [erlang-questions] OTP-9649 and further changes In-Reply-To: References: Message-ID: Bj?rn, I wanted to follow up on this. I am investigating this whiole issue (as I really would love to keep tuple calls) and I would like to know whether you have any numbers on how significant the speed impediment of this feature is (the run-time cost incurred in apply/3) that you mentioned earlier? Do you have any comparisons? Yurii On Mon, Jan 23, 2012 at 11:22 PM, Bj?rn Gustavsson wrote: > 2012/1/24 Yurii Rashkovskii : >> Bj?rn, >> >> Is there any way tuple module calls can be separated from parametrized >> modules and subsequently documented and become part of standard >> Erlang? >> > > Perhaps. > > The problem with the current implementation of "tuple modules" > is that the M in apply(M, F, A) and M:F(...) has a strange type, i.e. M can > be either an atom or a tuple. Unless we implement parameterized modules, > we want M to be only an atom. (There is a run-time cost in the > implementation of apply/3 to allow parameterized modules/"tuple modules", > and parameterized modules prevented an optimization I wanted to do in the > compiler for R15B.) > > We might consider an alternative implementation of "tuple modules" > that does not use apply(M, F, A) or M:F(...), using either new BIFs or new > syntax. (Example: tuple_module_apply(M, F, A) could behave as apply/3 > currently behaves, or alternatively only allow M to be a tuple. This is not > a serious suggestion, just an example to make it clear what I mean.) > > I suggest that you write an EEP. > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From yrashk@REDACTED Sun Jun 10 20:48:36 2012 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sun, 10 Jun 2012 11:48:36 -0700 Subject: [erlang-questions] OTP-9649 and further changes In-Reply-To: References: Message-ID: Bj?rn, I actually just did a totally unscientific test. I took the maint branch, and ran 10,000,000 Mod:fn() calls where Mod was passed from another function call and was just an atom. I ran it on maint, and on my Mac Pro timer:tc clocked 446983 (that's 0.446 seconds). After that, I brutally removed tuple call support from apply and fixed_apply BIFs in beam_emu.c, making `if (is_not_atom(module))` go straight to error label. I also removed `this` variable which was used in those checks because there was no need for it anymore. I re-ran the test, and timer:tc clocked 440680 and around. So, the runtime cost difference was about 6000 microseconds, or 0.006 seconds, for 10 million calls. If I am not mistaking, this is a 1.35% speedup on something as low cost as about 0.0446 microseconds per call (or 0.0440 microseconds per call without support for tuple calls). Is it this big of a difference to support the removal? What if we made tuple calls a documented feature? I'd be glad to write an EEP to support it. You can find my test patch attached. P.S. I also ran the same test for a tuple call (in the maint branch), timer:tc clocked 545656 and around. So there is a penalty for using tuple calls, but that's expectable and seems to be a reasonable tradeoff for me. P.P.S. Please let me know if something in this test is incorrect, I'd be glad to improve it. On Sun, Jun 10, 2012 at 10:52 AM, Yurii Rashkovskii wrote: > Bj?rn, > > I wanted to follow up on this. I am investigating this whiole issue > (as I really would love to keep tuple calls) and I would like to know > whether you have any numbers on how significant the speed impediment > of this feature is (the run-time cost incurred in apply/3) that you > mentioned earlier? Do you have any comparisons? > > Yurii > > On Mon, Jan 23, 2012 at 11:22 PM, Bj?rn Gustavsson > wrote: >> 2012/1/24 Yurii Rashkovskii : >>> Bj?rn, >>> >>> Is there any way tuple module calls can be separated from parametrized >>> modules and subsequently documented and become part of standard >>> Erlang? >>> >> >> Perhaps. >> >> The problem with the current implementation of "tuple modules" >> is that the M in apply(M, F, A) and M:F(...) has a strange type, i.e. M can >> be either an atom or a tuple. Unless we implement parameterized modules, >> we want M to be only an atom. (There is a run-time cost in the >> implementation of apply/3 to allow parameterized modules/"tuple modules", >> and parameterized modules prevented an optimization I wanted to do in the >> compiler for R15B.) >> >> We might consider an alternative implementation of "tuple modules" >> that does not use apply(M, F, A) or M:F(...), using either new BIFs or new >> syntax. (Example: tuple_module_apply(M, F, A) could behave as apply/3 >> currently behaves, or alternatively only allow M to be a tuple. This is not >> a serious suggestion, just an example to make it clear what I mean.) >> >> I suggest that you write an EEP. >> >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB -------------- next part -------------- A non-text attachment was scrubbed... Name: patch.diff Type: application/octet-stream Size: 2236 bytes Desc: not available URL: From yrashk@REDACTED Sun Jun 10 21:00:24 2012 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sun, 10 Jun 2012 12:00:24 -0700 Subject: [erlang-questions] OTP-9649 and further changes In-Reply-To: References: Message-ID: I also got a piece of advise to calculate the cost of running an empty cycle on both versions of the runtime. It was about 70,000 microseconds on both. This means that while the speedup remains the same, the cost of actual operations has to be dropped even lower, to about 0.0370 microseconds per call (on that hardware) On Sun, Jun 10, 2012 at 11:48 AM, Yurii Rashkovskii wrote: > Bj?rn, > > I actually just did a totally unscientific test. I took the maint > branch, and ran 10,000,000 Mod:fn() calls where Mod was passed from > another function call and was just an atom. I ran it on maint, and on > my Mac Pro timer:tc clocked 446983 (that's 0.446 seconds). > > After that, I brutally removed tuple call support from apply and > fixed_apply BIFs in beam_emu.c, making `if (is_not_atom(module))` go > straight to error label. I also removed `this` variable which was used > in those checks because there was no need for it anymore. > > I re-ran the test, and timer:tc clocked 440680 and around. > > So, the runtime cost difference was about 6000 microseconds, or 0.006 > seconds, for 10 million calls. If I am not mistaking, this is a 1.35% > speedup on something as low cost as about 0.0446 microseconds per call > (or 0.0440 microseconds per call without support for tuple calls). > > Is it this big of a difference to support the removal? What if we made > tuple calls a documented feature? I'd be glad to write an EEP to > support it. > > You can find my test patch attached. > > P.S. I also ran the same test for a tuple call (in the maint branch), > timer:tc clocked 545656 and around. So there is a penalty for using > tuple calls, but that's expectable and seems to be a reasonable > tradeoff for me. > P.P.S. Please let me know if something in this test is incorrect, I'd > be glad to improve it. > > > On Sun, Jun 10, 2012 at 10:52 AM, Yurii Rashkovskii wrote: >> Bj?rn, >> >> I wanted to follow up on this. I am investigating this whiole issue >> (as I really would love to keep tuple calls) and I would like to know >> whether you have any numbers on how significant the speed impediment >> of this feature is (the run-time cost incurred in apply/3) that you >> mentioned earlier? Do you have any comparisons? >> >> Yurii >> >> On Mon, Jan 23, 2012 at 11:22 PM, Bj?rn Gustavsson >> wrote: >>> 2012/1/24 Yurii Rashkovskii : >>>> Bj?rn, >>>> >>>> Is there any way tuple module calls can be separated from parametrized >>>> modules and subsequently documented and become part of standard >>>> Erlang? >>>> >>> >>> Perhaps. >>> >>> The problem with the current implementation of "tuple modules" >>> is that the M in apply(M, F, A) and M:F(...) has a strange type, i.e. M can >>> be either an atom or a tuple. Unless we implement parameterized modules, >>> we want M to be only an atom. (There is a run-time cost in the >>> implementation of apply/3 to allow parameterized modules/"tuple modules", >>> and parameterized modules prevented an optimization I wanted to do in the >>> compiler for R15B.) >>> >>> We might consider an alternative implementation of "tuple modules" >>> that does not use apply(M, F, A) or M:F(...), using either new BIFs or new >>> syntax. (Example: tuple_module_apply(M, F, A) could behave as apply/3 >>> currently behaves, or alternatively only allow M to be a tuple. This is not >>> a serious suggestion, just an example to make it clear what I mean.) >>> >>> I suggest that you write an EEP. >>> >>> -- >>> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From overminddl1@REDACTED Sun Jun 10 22:18:28 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 10 Jun 2012 14:18:28 -0600 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: On Sun, Jun 10, 2012 at 4:47 AM, Gordon Guthrie wrote: >> A single process handling a specific set of rows would handle that in >> most cases? ?What are the transactions needed for? ?No way to work >> around? > > You need to co-ordinate the update to several tables and roll back and > restore on failure - either use a TM or write one - better not to > write your own.. Generally in KV stores you want to store as much information in a row as possible, there is no way to coalesce the row information together? And Riak has support for secondary indexes (as of 1.0 I think?) to enable fast lookup on other cells too. From gordon@REDACTED Sun Jun 10 22:43:03 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Sun, 10 Jun 2012 21:43:03 +0100 Subject: [erlang-questions] Mnesia Backup/Fallback Problems With Schema In-Reply-To: References: Message-ID: > Generally in KV stores you want to store as much information in a row > as possible, there is no way to coalesce the row information together? > ?And Riak has support for secondary indexes (as of 1.0 I think?) to > enable fast lookup on other cells too. There are relations that need to be managed - moving them to relations between rows in the same table doesn't help. Gordon On 10 June 2012 21:18, OvermindDL1 wrote: > On Sun, Jun 10, 2012 at 4:47 AM, Gordon Guthrie wrote: >>> A single process handling a specific set of rows would handle that in >>> most cases? ?What are the transactions needed for? ?No way to work >>> around? >> >> You need to co-ordinate the update to several tables and roll back and >> restore on failure - either use a TM or write one - better not to >> write your own.. > > Generally in KV stores you want to store as much information in a row > as possible, there is no way to coalesce the row information together? > ?And Riak has support for secondary indexes (as of 1.0 I think?) to > enable fast lookup on other cells too. From jose.valim@REDACTED Sun Jun 10 23:16:31 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Sun, 10 Jun 2012 23:16:31 +0200 Subject: [erlang-questions] OTP-9649 and further changes In-Reply-To: References: Message-ID: I believe it is also worth mentioning that according to Joe Armstrong "the abuse of param modules are here to stay" and should be documented: https://groups.google.com/group/erlang-programming/browse_thread/thread/c59e72387167a834/82441aed16f8a7ba?show_docid=82441aed16f8a7ba&pli=1 Cheers, * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Sun, Jun 10, 2012 at 9:00 PM, Yurii Rashkovskii wrote: > I also got a piece of advise to calculate the cost of running an empty > cycle on both versions of the runtime. It was about 70,000 > microseconds on both. This means that while the speedup remains the > same, the cost of actual operations has to be dropped even lower, to > about 0.0370 microseconds per call (on that hardware) > > On Sun, Jun 10, 2012 at 11:48 AM, Yurii Rashkovskii > wrote: > > Bj?rn, > > > > I actually just did a totally unscientific test. I took the maint > > branch, and ran 10,000,000 Mod:fn() calls where Mod was passed from > > another function call and was just an atom. I ran it on maint, and on > > my Mac Pro timer:tc clocked 446983 (that's 0.446 seconds). > > > > After that, I brutally removed tuple call support from apply and > > fixed_apply BIFs in beam_emu.c, making `if (is_not_atom(module))` go > > straight to error label. I also removed `this` variable which was used > > in those checks because there was no need for it anymore. > > > > I re-ran the test, and timer:tc clocked 440680 and around. > > > > So, the runtime cost difference was about 6000 microseconds, or 0.006 > > seconds, for 10 million calls. If I am not mistaking, this is a 1.35% > > speedup on something as low cost as about 0.0446 microseconds per call > > (or 0.0440 microseconds per call without support for tuple calls). > > > > Is it this big of a difference to support the removal? What if we made > > tuple calls a documented feature? I'd be glad to write an EEP to > > support it. > > > > You can find my test patch attached. > > > > P.S. I also ran the same test for a tuple call (in the maint branch), > > timer:tc clocked 545656 and around. So there is a penalty for using > > tuple calls, but that's expectable and seems to be a reasonable > > tradeoff for me. > > P.P.S. Please let me know if something in this test is incorrect, I'd > > be glad to improve it. > > > > > > On Sun, Jun 10, 2012 at 10:52 AM, Yurii Rashkovskii > wrote: > >> Bj?rn, > >> > >> I wanted to follow up on this. I am investigating this whiole issue > >> (as I really would love to keep tuple calls) and I would like to know > >> whether you have any numbers on how significant the speed impediment > >> of this feature is (the run-time cost incurred in apply/3) that you > >> mentioned earlier? Do you have any comparisons? > >> > >> Yurii > >> > >> On Mon, Jan 23, 2012 at 11:22 PM, Bj?rn Gustavsson > >> wrote: > >>> 2012/1/24 Yurii Rashkovskii : > >>>> Bj?rn, > >>>> > >>>> Is there any way tuple module calls can be separated from parametrized > >>>> modules and subsequently documented and become part of standard > >>>> Erlang? > >>>> > >>> > >>> Perhaps. > >>> > >>> The problem with the current implementation of "tuple modules" > >>> is that the M in apply(M, F, A) and M:F(...) has a strange type, i.e. > M can > >>> be either an atom or a tuple. Unless we implement parameterized > modules, > >>> we want M to be only an atom. (There is a run-time cost in the > >>> implementation of apply/3 to allow parameterized modules/"tuple > modules", > >>> and parameterized modules prevented an optimization I wanted to do in > the > >>> compiler for R15B.) > >>> > >>> We might consider an alternative implementation of "tuple modules" > >>> that does not use apply(M, F, A) or M:F(...), using either new BIFs or > new > >>> syntax. (Example: tuple_module_apply(M, F, A) could behave as apply/3 > >>> currently behaves, or alternatively only allow M to be a tuple. This > is not > >>> a serious suggestion, just an example to make it clear what I mean.) > >>> > >>> I suggest that you write an EEP. > >>> > >>> -- > >>> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Mon Jun 11 00:38:11 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 10 Jun 2012 23:38:11 +0100 (BST) Subject: [erlang-questions] Data structures are strange things In-Reply-To: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> Message-ID: I have some more data to add. I did a quick version of AVL trees so the final data is: ----- Original Message ----- > Here are the figures. I have also included dict, which is the slowest > on insertion/deletion but significantly faster lookup times. All > times are in usecs for the whole set: > Operation ? ? ? ? ? ?Load ? ? ? ?Fetch ? ? ? Load > ? ? ? ?Fetch > # of elements? ? ? ? ? ? ? ? ?100000 ? ? ? ? ? ? ? ? ?10000 > Overhead time? ? ? ? ? ? ? ? ? 43000 ? ? ? ? ? ? ? ? ? 3700 avldict???? AVL trees 430000 150000 31000 14500 > ttdict 2-3 trees 310000 161000 24000 15500 > ttfdict 2-3-4 trees 312000 173000 24200 16500 > llrbdict Left Leaning RB trees 390000 164000 27100 14900 > rbdict Red Black trees 380000 167000 27100 15000 > aadict AA trees 423000 167000 30000 15000 > gb_trees using faster insert/3 370000 190000 26500 16200 > dict 522000 107000 28000 11000 I hope the table is still readable. The loading for AVL trees can be made faster as the algorithm I use is quite naive and probably does too much copying. As Richard C pointed out the RB trees, AA trees and now AVL trees all need an extra field, which the 2-3 and 2-3-4 trees do not. So in terms of space 2-3-4 is better than 2-3 is better than gb_trees is better than other trees. The code size for 2-3-4 trees is bigger than for 2-3 trees, more alternatives to consider. Considering the added code size, though not really complexity, 2-3 trees seems to be a reasonable choice for a tree module with the dict API in terms of speed, code size and data size. Or maybe the Left Leaning RB trees and AVL trees. The fastest trees yet. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jun 11 02:00:10 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jun 2012 12:00:10 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: On 9/06/2012, at 7:30 PM, Michael Turner wrote: > > http://www.linkedin.com/pub/kennita-watson/1/321/a19 > > Kennita's most recent resume entry there: > > too much experience > Long Term Disability > July 2005 ? Present (7 years) Sunnyvale, CA > > I haven't worked since 2005; the purpose of this entry is to get > LinkedIn to stop asking me for a current position. > --- > > Maybe she's only joking about the disability part. I'm not a member of LinkedIn, so I am unable to read her profile. She may well not be joking; one of the highly innovative people in the Prolog community, Lee Naish, pretty much dropped out of sight for similar reasons. He's a really nice person too. (And Melbourne is an Erlang-friendly University: the Mercury language/system has a back end that generates Erlang. If Lee hadn't become ill, he might have contributed a lot to Erlang.) > >> And now we come to the crux of the matter. It's the *review* process. >> That takes time and effort and management, and it's not at all clear >> to me what, if anything, makes a wiki easier to manage than the current >> setup. > > I don't know either, because I don't know Ericsson Erlang/OTP > documentation patch review processes. > > For how it would work in a wiki, try starting at this link > > http://en.wikipedia.org/w/index.php?title=Richard_O%27Keefe&diff=489842048&oldid=489773645 > > and keep clicking "Previous edit". I think this interface makes it > pretty easy to tell that a small, simple edit (which is most of them, > and which would be most of them, in what I propose) is in fact > correct. If it's not, note the "undo" links -- very often, "undo" just > works without fuss. That actually tells me a lot about review processes, and it is damning. ? 12:19, 29 April 2012 (diff | hist) . . (-4)? . . Richard O'Keefe ? ? 12:19, 29 April 2012 (diff | hist) . . (+13)? . . Richard O'Keefe ? ? 12:18, 29 April 2012 (diff | hist) . . (+67)? . . Richard O'Keefe ? ? 23:22, 30 March 2012 (diff | hist) . . (-39)? . . Stiliyan Petrov ? (??External links) ? 20:53, 13 February 2012 (diff | hist) . . (+50)? . . Syd Barrett ? There are several edits there, purportedly by me, of a version that claims I'm the cousin of some footballer, which I am not. - The information is untrue. - The purported author is not the real author. - The subject is alive and readily reachable and the changes were not verified. I'd like to thank 'Gadfium' who apparently corrected the article, but I have no idea who Gadfium is. When I tried to send feedback saying that I'd like to thank Gadfium for the correction, I got a "Form submission error" with no indication of what the error actually was. In my wildest dreams I couldn't have asked for a better example of a Wiki going wrong. And what the history features of Wikipedia actually guarantee is that the wrong information will be discoverable forever. > So if this is the "crux of the matter", maybe a wiki would actually > make documentation patch review and acceptance *easier* than it is now > at Ericsson: I do not see any advantage to be gained by making rapid acceptance of untrue information with forged credentials easy. From bchesneau@REDACTED Mon Jun 11 04:38:48 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 11 Jun 2012 04:38:48 +0200 Subject: [erlang-questions] socket unrecv In-Reply-To: References: Message-ID: On Sun, Jun 10, 2012 at 12:27 AM, CGS wrote: > Hi Benoit, > > Forgive my lack of imagination, but how do you manage to listen to the same > socket with another process? Can you pass the socket from the other API? > Moreover, how do you make the other API to listen after your one without > putting it in front of the other API? > > If you can access and pass the socket from the other API, I think you can > bounce back the data toward the same socket (gen_tcp:send/2) because then > the other API should receive the same data as the original transmitter > (hopefully, the other side won't crash because it doesn't expect reply). > But, still, this is only in the case you can receive the packets in the same > time with the other API and the other API trashes the original packets > (otherwise you just double the data in the other API). > > CGS > My case was a little more simpler here :) By API I meant another internal code that wasn't supposed to be used by the initial code doing the parsing and getting data from the socket. Anyway I fixed it by rewriting the first part. It works well now :) - beno?t From michael.eugene.turner@REDACTED Mon Jun 11 06:53:08 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 11 Jun 2012 13:53:08 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: > There are several edits there, purportedly by me, of a version that claims > I'm the cousin of some footballer, which I am not. SInce I've written over and over on this thread about how becoming an editor should be a much more restrictive process than Wikipedia's (which is something you can do in MediaWiki - it offers fairly fine-grained access control.) I can only take the complaint you offer as a sign of either serious amnesia ()get that looked at) of serious disrespect for anything I have to say. > I'd like to thank 'Gadfium' who apparently corrected the article, but I > have no idea who Gadfium is. You could post here, on his talk page http://en.wikipedia.org/wiki/User_talk:Gadfium if you were thankful. If you need to know who he is, you could ask there, and he might send you private e-mail. In any case, this is all irrelevant for our discussion, since policy of any Erlang/OTP wiki could easily include (and probably should include) a rule that everybody editing must be identified by their real-world name. This rule could be enforced by putting a human being in the loop of signup, someone who makes sure that the e-mail address sent as part of signup is that of a real person, and that this real person sent that application for editorship. > In my wildest dreams I couldn't have asked for a better example of a > Wiki going wrong. Then you don't have much experience at all with editing wikis, because much worse stuff happens all the time. It needn't happen with an Erlang/OTP wiki, for reasons I've explained over and over, in earlier message, and which you apparently either can't remember (get that looked at, the amnesia may be effecting your work too) or choose to ignore (can't help you with that problem.) > I do not see any advantage to be gained by making > rapid acceptance of untrue information with forged credentials easy. Nor do I. MediaWiki, and all wikis worth of the name, provide administrator dashboards from which one can enforce requirements on membership. As you can see, the need for it is obvious. And in fact, employed often enough. I've been part of the process of banning people from editing, including IP blocks. In one case, it involved someone who made legal threats against me, and only for revising an article on a controversial subject to be more accurate (in case you're wondering why most editors - not me, but most - choose anonymity.) Again, you're falling back on "wikis suck because I found something bad." You're not asking, "Why does this bad thing happen, and is there anything to be done to prevent it?" In fact, these problems are quite manageable, and Wikipedia, in the interests of gathering the largest possible membership, leaves editing quite open. It also means some information is unreliable at one time or another. Anyone who reads Wikipedia without keeping that in mind is simply a fool. -michael turner On Mon, Jun 11, 2012 at 9:00 AM, Richard O'Keefe wrote: > > On 9/06/2012, at 7:30 PM, Michael Turner wrote: >> >> ?http://www.linkedin.com/pub/kennita-watson/1/321/a19 >> >> Kennita's most recent resume entry there: >> >> too much experience >> Long Term Disability >> July 2005 ? Present (7 years) Sunnyvale, CA >> >> I haven't worked since 2005; the purpose of this entry is to get >> LinkedIn to stop asking me for a current position. >> --- >> >> Maybe she's only joking about the disability part. > > I'm not a member of LinkedIn, so I am unable to read her profile. > She may well not be joking; one of the highly innovative people > in the Prolog community, Lee Naish, pretty much dropped out of > sight for similar reasons. ?He's a really nice person too. > (And Melbourne is an Erlang-friendly University: ?the Mercury > language/system has a back end that generates Erlang. ?If Lee > hadn't become ill, he might have contributed a lot to Erlang.) > >> >>> And now we come to the crux of the matter. ?It's the *review* process. >>> That takes time and effort and management, and it's not at all clear >>> to me what, if anything, makes a wiki easier to manage than the current >>> setup. >> >> I don't know either, because I don't know Ericsson Erlang/OTP >> documentation patch review processes. >> >> For how it would work in a wiki, try starting at this link >> >> ?http://en.wikipedia.org/w/index.php?title=Richard_O%27Keefe&diff=489842048&oldid=489773645 >> >> and keep clicking "Previous edit". I think this interface makes it >> pretty easy to tell that a small, simple edit (which is most of them, >> and which would be most of them, in what I propose) is in fact >> correct. If it's not, note the "undo" links -- very often, "undo" just >> works without fuss. > > That actually tells me a lot about review processes, and it is damning. > > ? ? ? ?? 12:19, 29 April 2012 (diff | hist) . . (-4)? . . Richard O'Keefe ? > ? ? ? ?? 12:19, 29 April 2012 (diff | hist) . . (+13)? . . Richard O'Keefe ? > ? ? ? ?? 12:18, 29 April 2012 (diff | hist) . . (+67)? . . Richard O'Keefe ? > ? ? ? ?? 23:22, 30 March 2012 (diff | hist) . . (-39)? . . Stiliyan Petrov ? (??External links) > ? ? ? ?? 20:53, 13 February 2012 (diff | hist) . . (+50)? . . Syd Barrett ? > > There are several edits there, purportedly by me, of a version that claims > I'm the cousin of some footballer, which I am not. > > ?- The information is untrue. > ?- The purported author is not the real author. > ?- The subject is alive and readily reachable and > ? the changes were not verified. > > I'd like to thank 'Gadfium' who apparently corrected the article, but I > have no idea who Gadfium is. ?When I tried to send feedback saying that > I'd like to thank Gadfium for the correction, I got a "Form submission > error" with no indication of what the error actually was. > > In my wildest dreams I couldn't have asked for a better example of a > Wiki going wrong. ?And what the history features of Wikipedia actually > guarantee is that the wrong information will be discoverable forever. > >> So if this is the "crux of the matter", maybe a wiki would actually >> make documentation patch review and acceptance *easier* than it is now >> at Ericsson: > > I do not see any advantage to be gained by making > rapid acceptance of untrue information with forged credentials easy. > > From a.shneyderman@REDACTED Mon Jun 11 09:30:35 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Mon, 11 Jun 2012 09:30:35 +0200 Subject: [erlang-questions] Out of memory troubles. Message-ID: Hi, all! I have an out of memory exception I do not understand. I posted yesterday about not being able to open more than certain number of persistent connections towards my server. That turned out to be due to ulimit not being high enough on my mac's shell. I increased that to 8192 and things started to work better. I was able to open 1K connections towards the server with no problems and memory footprint of the OS process that keeps those 1K connections while exchanging data between the server and the processes on erlang VM is about 150MB. The next step was to increase the number of persistent connections from 1K to 2K. The number of connections where not allowed to go further than 1024 that as I understood was due to the way Erlang was compiled on Mac. I searched on this list and found a suggestion by Joel Raymont on how to up that number by increasing __DARWIN_FD_SETSIZE (http://erlang.org/pipermail/erlang-questions/2011-December/063119.html). Then I recompiled erlang and ran the same application trying to request 2K connections and here is where I get out of memory error. Here is what I see in the shell: 09:08:42.229 [error] erts_poll_wait() failed: einval (22) beam.smp(71174,0xb032e000) malloc: *** mmap(size=730857472) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug crash reports are basically identical to what I see on the console except for the malloc business. >From the footprint that this simulation leaves on my server, I can see that there were 1008 successful connections. I was trying to monitor memory consumption of the erlang process and it jumps very high very fast after around 1K connections. It goes from 150M to about 1.5GB really fast and then VM crashes. As I mentioned 1K connections only has about 150M of memory footprint on my system and this laptop has about 1.5G of spare memory at the time I start the process - so, I'd expect 8-10K connections possible. I can certainly run several VMs with 1K connections per VM - as the matter of fact that is what I do now to avoid out of memory exceptions and crashes - but I would like to be able to have more agents (connections) on the same VM which simplifies deployment for me. Any ideas on what might I be doing wrong? Thanks, Alex. From maruthavanan_s@REDACTED Mon Jun 11 12:23:29 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Mon, 11 Jun 2012 06:23:29 -0400 Subject: [erlang-questions] Large Flat file operations - optimization. Message-ID: Hi, I am about to process flatfiles, may be CSV or may be line based one. I came to know about CSV parsing in the below link. http://blog.vmoroz.com/2011/01/csv-in-erlang.html But I do not want to keep a big file in Erlang VM memory and also wanted to handle that for lines based ones. So I changed the parse code as below that it would seperate the lines and give me back where I would read in 1024 or X chunks parse1([],Lines,CurrentLine)-> {lists:reverse([lists:reverse(L) ||L <- Lines]),lists:reverse(CurrentLine)};parse1([$\r|[]],Lines,CurrentLine) -> {lists:reverse([lists:reverse(L) ||L <- [CurrentLine|Lines]]),[]};parse1([$\n|[]],Lines,CurrentLine) -> {lists:reverse([lists:reverse(L) ||L <- [CurrentLine|Lines]]),[]};parse1([C|T],Lines,CurrentLine) when C == $\r; C ==$\n -> parse1(T,[CurrentLine|Lines],[]);parse1([C|T],Lines,CurrentLine) -> parse1(T,Lines,[C|CurrentLine]). but now to convert into CSV record of each line, I am thinking of the below.2> {List,Remaining}=parse1(file:read(IO,1024),[],[]). %%Sample data {["1,2,3","3,4,5"],"7,8"}3> [string:tokens(N,",")|| N <- List]. %% Sample Data [["1","2","3"],["3","4","5]] But is using string:tokens performs well for handling huge data? Can I use any other list comprehension or some thing to perform better? Kindly suggest. Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From meetprashant007@REDACTED Mon Jun 11 13:49:15 2012 From: meetprashant007@REDACTED (Prashant Sharma) Date: Mon, 11 Jun 2012 17:19:15 +0530 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: Avinash, He said so because generally these things are obvious. I like your enthusiasm, I will try if I can help! -detached Starts the Erlang runtime system detached from the system console. Useful for running daemons and backgrounds processes. Means it will deamonize the process. http://www.itp.uzh.ch/~dpotter/howto/daemonize?utm_source=twitterfeed&utm_medium=twitter (Full explanation, If you love C) Which means the process has no terminal(parent) process attached. Which in effect would mean that the process does not print anything on the console nor can you stop or interact with it. And when we say no input or no shell. That means the process can not receive anything through pipes as in ls | erl Eshell V5.10 (abort with ^G) 1> * 2: syntax error before: core 1> *** Terminating erlang (nonode@REDACTED) See its reporting errors as it is trying to interpret input as commands. ls | erl -noinput will not show such behavior. Not sure if you are already aware about unix pipes and input of a shell command as argument or via pipe. The above command simply closes its pipe. http://www.gnu.org/software/libc/manual/html_node/Pipes-and-FIFOs.html Try this you get the clarity. To the list: Please correct me If I am wrong. So before you start reading the books on erlang. You can get some idea about these things, I had say try writing an application using MPI and stuff just so you understand the pain of doing those things in C. Those books start and teach from the perspective of erlang. >From this thread a question sparked in my mind, which is "What would one need to know to start learning or working on erlang.?" That would mean everything, (Things like understand operating system, processes etc). Wish I could list down everything ! On Sun, Jun 10, 2012 at 3:23 PM, CGS wrote: > Then, maybe the next step for you should be to read some books about how > Linux manages processes, pipelines and so on so forth. > > CGS > > > > > > > > > On Sun, Jun 10, 2012 at 7:17 AM, Avinash Dhumane > wrote: >> >> Of course, that's where I came from. >> >> Not that I couldn't employ those options to work fruitfully for the >> immediate tasks at hand (towards this end, the online Erlang documentation >> is adequately equipped) but I am trying to get through them into the >> machinery that stands behind them - what that machinery is, how it works, >> and the most importantly, why it is like what it is. This understanding is >> indispensable for my self-adaption to Erlang for solving customers' >> problems. I guess every programmer-to-be (at various levels) of Erlang will >> have this need. >> >> Right now, we have Joe's thesis at one end of the continuum and the (open) >> source code at the end. The 3 books are very good (and I have them all) but >> the desire to know, understand and apply is insatiable! :-) >> >> >> On Sun, 10 Jun 2012 03:19:10 +0530, CGS wrote: >> >>> Have you tried http://www.erlang.org/doc/man/erl.html for erl options? >>> >>> CGS >>> >>> >>> >>> >>> On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane >>> wrote: >>> >>>> Where may I look further for the basic understanding of spatio-temporal >>>> relations among the "things" named as ERTS, Emulator and Shell? The >>>> question occurred while I was toying with "erl" as Unix-filter >>>> shell-command and couldn't get over the options (to erl) like >>>> "-noshell", >>>> "-s", "-detached", init:stop(), and so on in the documentation and in >>>> the >>>> FAQ. >>>> >>>> Thanks >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> >>>> http://erlang.org/mailman/**listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Prashant Sharma Development Engineer Pramati Technologies Begumpet "Hare Krishna" From dmkolesnikov@REDACTED Mon Jun 11 13:51:18 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Mon, 11 Jun 2012 14:51:18 +0300 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: Message-ID: <6477442149500295393@unknownmsgid> Hi, The choose technique would not give you an optimal performance and memory footprint. The best option would be binary based parser. Take a look into my csv parser. It does both sequential and parallel parsing of csv files. The target was 300k lines in less then second https://github.com/fogfish/csv Best Regards, Dmitry >-|-|-*> On 11.6.2012, at 13.23, Maruthavanan Subbarayan wrote: Hi, I am about to process flatfiles, may be CSV or may be line based one. I came to know about CSV parsing in the below link. http://blog.vmoroz.com/2011/01/csv-in-erlang.html But I do not want to keep a big file in Erlang VM memory and also wanted to handle that for lines based ones. So I changed the parse code as below that it would seperate the lines and give me back where I would read in 1024 or X chunks parse1([],Lines,CurrentLine)-> {lists:reverse([lists:reverse(L) ||L <- Lines]),lists:reverse(CurrentLine)}; parse1([$\r|[]],Lines,CurrentLine) -> {lists:reverse([lists:reverse(L) ||L <- [CurrentLine|Lines]]),[]}; parse1([$\n|[]],Lines,CurrentLine) -> {lists:reverse([lists:reverse(L) ||L <- [CurrentLine|Lines]]),[]}; parse1([C|T],Lines,CurrentLine) when C == $\r; C ==$\n -> parse1(T,[CurrentLine|Lines],[]); parse1([C|T],Lines,CurrentLine) -> parse1(T,Lines,[C|CurrentLine]). but now to convert into CSV record of each line, I am thinking of the below. 2> {List,Remaining}=parse1(file:read(IO,1024),[],[]). %%Sample data {["1,2,3","3,4,5"],"7,8"} 3> [string:tokens(N,",")|| N <- List]. %% Sample Data [["1","2","3"],["3","4","5]] But is using string:tokens performs well for handling huge data? Can I use any other list comprehension or some thing to perform better? Kindly suggest. Thanks, Marutha _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Mon Jun 11 15:06:04 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 11 Jun 2012 15:06:04 +0200 Subject: [erlang-questions] Regarding the life cycle of ERL_NIF_TERM In-Reply-To: <4FD06F14.3000406@gmail.com> References: <4FD06F14.3000406@gmail.com> Message-ID: <4FD5ED3C.1010407@erix.ericsson.se> I realize that the documentation might be a bit ambiguous here. We are actually talking about lifetimes of two different entities: 1. An Erlang term. 2. An ERL_NIF_TERM value as a reference to a term. For process independent environments both term and reference have the same lifetime, that is the same as the lifetime of the environment. For process bound environments the lifetime of a term is determined by the GC. Passing a term as argument to a NIF will not affect the lifetime of the term. The lifetime of an ERL_NIF_TERM reference is however limited to the execution scope of the NIF. The reference is only valid until the NIF returns. A subsequent call to the same NIF from the same process with the same term as argument may result in a different ERL_NIF_TERM value. /Sverker, Erlang/OTP Ericsson Xiaopong Tran wrote: > Hi, > > I have a question regarding the life cycle of ERL_NIF_TERM. According to > the Erlang document http://www.erlang.org/doc/man/erl_nif.html: > > "All ERL_NIF_TERM's belong to an environment (ErlNifEnv). A term can not > be destructed individually, it is valid until its environment is > destructed." > > And the description regarding the ErlNifEnv: > > "A process bound environment is passed as the first argument to all > NIFs. All function arguments passed to a NIF will belong to that > environment." > > Could someone clarify what that means exactly, in terms of the > ERL_NIF_TERM life cycle? > > So, in this case, the environment is process bound. Does this > mean that, all ERL_NIF_TERM passed in a parameters to NIF function, > and the ERL_NIF_TERM return value, will never get GC, as long > as the process is running? > > Cheers, > > > Xiaopong > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From gordon@REDACTED Mon Jun 11 16:07:08 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 11 Jun 2012 15:07:08 +0100 Subject: [erlang-questions] Out of memory troubles. In-Reply-To: References: Message-ID: Alex I had a similar problem on Linux - not sure how relevant is to a Mac but I thought I would mention it. On Linux you can increase the limits but only subject to the system-level restrictions placed on your username by a file called /etc/security/limits.conf To get my open-file limit up on Linux I had to bump that up as well.. Mebbies there is something like that on OSX as well? Gordon On 11 June 2012 08:30, Alex Shneyderman wrote: > Hi, all! > > I have an out of memory exception I do not understand. > > I posted yesterday about not being able to open more than certain > number of persistent connections towards my server. That turned out to > be due to ulimit not being high enough on my mac's shell. I increased > that to 8192 and things started to work better. I was able to open 1K > connections towards the server with no problems and memory footprint > of the OS process that keeps those 1K connections while exchanging > data between the server and ?the processes on erlang VM is about > 150MB. The next step was to increase the number of persistent > connections from 1K to 2K. > > The number of connections where not allowed to go further than 1024 > that as I understood was due to the way Erlang was compiled on Mac. I > searched on this list and found a suggestion by Joel Raymont on how to > up that number by increasing __DARWIN_FD_SETSIZE > (http://erlang.org/pipermail/erlang-questions/2011-December/063119.html). > Then I recompiled erlang and ran the same application trying to > request 2K connections and here is where I get out of memory error. > Here is what I see in the shell: > > 09:08:42.229 [error] erts_poll_wait() failed: einval (22) > > > beam.smp(71174,0xb032e000) malloc: *** mmap(size=730857472) failed > (error code=12) > > ? ? ? ? ? *** error: can't allocate region > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? *** set a breakpoint in > malloc_error_break to debug > > > crash reports are basically identical to what I see on the console > except for the malloc business. > > From the footprint that this simulation leaves on my server, I can see > that there were 1008 successful connections. I was trying to monitor > memory consumption of the erlang process and it jumps very high very > fast after around 1K connections. It goes from 150M to about 1.5GB > really fast and then VM crashes. > > As I mentioned 1K connections only has about 150M of memory footprint > on my system and this laptop has about 1.5G of spare memory at the > time I start the process - so, I'd expect 8-10K connections possible. > I can certainly run several VMs with 1K connections per VM - as the > matter of fact that is what I do now to avoid out of memory exceptions > and crashes - but I would like to be able to have more agents > (connections) on the same VM which simplifies deployment for me. > > Any ideas on what might I be doing wrong? > > Thanks, > Alex. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hobson42@REDACTED Mon Jun 11 16:30:04 2012 From: hobson42@REDACTED (Ian) Date: Mon, 11 Jun 2012 15:30:04 +0100 Subject: [erlang-questions] Learning Erlang In-Reply-To: <4FD24F73.1030903@gmail.com> References: <4FD24F73.1030903@gmail.com> Message-ID: <4FD600EC.6000801@gmail.com> On 08/06/2012 20:16, Ian wrote: > Hi All, > > Is there a book, on-line resource, or even a well written open-source > project, that would help me understand how to write good clean Erlang > code? Thanks to everyone for their suggestions. In particular Erik and Volker for their suggestions - just what I was looking for. Ian From avinash@REDACTED Mon Jun 11 16:34:41 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Mon, 11 Jun 2012 20:04:41 +0530 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: My question was not so much about the semantics of options to "erl" (and how to employ them) as the "characteristic relations" between the 3 "entities" implied by the words mentioned in the subject-line. "erl" (w/ its options) is merely an occasion into the inquiry. Thanks Avinash PS: I have my Erlang-solutions running in the field (and that's purely Erlang's greatness!), and prior to that I have spent 15+ years on Unix. >>>> On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane >>>> wrote: >>>> >>>>> Where may I look further for the basic understanding of >>>>> spatio-temporal >>>>> relations among the "things" named as ERTS, Emulator and Shell? The >>>>> question occurred while I was toying with "erl" as Unix-filter >>>>> shell-command and couldn't get over the options (to erl) like >>>>> "-noshell", >>>>> "-s", "-detached", init:stop(), and so on in the documentation and in >>>>> the >>>>> FAQ. >>>>> >>>>> Thanks From dan@REDACTED Mon Jun 11 16:38:56 2012 From: dan@REDACTED (Daniel Dormont) Date: Mon, 11 Jun 2012 10:38:56 -0400 Subject: [erlang-questions] The Trinity: ERTS, Emulator, Shell In-Reply-To: References: Message-ID: If I may, I think Avinash is asking a slightly more abstract question (or maybe I'm projecting a similar question I have onto him, could be). Which is to say, from the standpoint of design, architecture, the relationships among the different parts of the system, what are the differences among ERTS, the emulator, and the shell? In other words, when one starts erl -noshell, what does it mean to not have a shell? What aspects of what I get when I run plain old "erl" are part of the shell and are therefore missing if I run erl -noshell? dan On Mon, Jun 11, 2012 at 7:49 AM, Prashant Sharma wrote: > Avinash, > > He said so because generally these things are obvious. I like your > enthusiasm, I will try if I can help! > > -detached > Starts the Erlang runtime system detached from the system console. > Useful for running daemons and backgrounds processes. > > Means it will deamonize the process. > > http://www.itp.uzh.ch/~dpotter/howto/daemonize?utm_source=twitterfeed&utm_medium=twitter > (Full explanation, If you love C) > > Which means the process has no terminal(parent) process attached. > Which in effect would mean that the process does not print anything on > the console nor can you stop or interact with it. > > And when we say no input or no shell. That means the process can not > receive anything through pipes as in > > > ls | erl > Eshell V5.10 (abort with ^G) > 1> * 2: syntax error before: core > 1> *** Terminating erlang (nonode@REDACTED) > See its reporting errors as it is trying to interpret input as commands. > > ls | erl -noinput > > will not show such behavior. > > Not sure if you are already aware about unix pipes and input of a > shell command as argument or via pipe. The above command simply closes > its pipe. > http://www.gnu.org/software/libc/manual/html_node/Pipes-and-FIFOs.html > > Try this you get the clarity. > > To the list: Please correct me If I am wrong. > > So before you start reading the books on erlang. You can get some idea > about these things, I had say try writing an application using MPI and > stuff just so you understand the pain of doing those things in C. > Those books start and teach from the perspective of erlang. > > From this thread a question sparked in my mind, which is "What would > one need to know to start learning or working on erlang.?" That would > mean everything, (Things like understand operating system, processes > etc). Wish I could list down everything ! > > > On Sun, Jun 10, 2012 at 3:23 PM, CGS wrote: > > Then, maybe the next step for you should be to read some books about how > > Linux manages processes, pipelines and so on so forth. > > > > CGS > > > > > > > > > > > > > > > > > > On Sun, Jun 10, 2012 at 7:17 AM, Avinash Dhumane > > wrote: > >> > >> Of course, that's where I came from. > >> > >> Not that I couldn't employ those options to work fruitfully for the > >> immediate tasks at hand (towards this end, the online Erlang > documentation > >> is adequately equipped) but I am trying to get through them into the > >> machinery that stands behind them - what that machinery is, how it > works, > >> and the most importantly, why it is like what it is. This understanding > is > >> indispensable for my self-adaption to Erlang for solving customers' > >> problems. I guess every programmer-to-be (at various levels) of Erlang > will > >> have this need. > >> > >> Right now, we have Joe's thesis at one end of the continuum and the > (open) > >> source code at the end. The 3 books are very good (and I have them all) > but > >> the desire to know, understand and apply is insatiable! :-) > >> > >> > >> On Sun, 10 Jun 2012 03:19:10 +0530, CGS wrote: > >> > >>> Have you tried http://www.erlang.org/doc/man/erl.html for erl options? > >>> > >>> CGS > >>> > >>> > >>> > >>> > >>> On Fri, Jun 8, 2012 at 10:24 AM, Avinash Dhumane > >>> wrote: > >>> > >>>> Where may I look further for the basic understanding of > spatio-temporal > >>>> relations among the "things" named as ERTS, Emulator and Shell? The > >>>> question occurred while I was toying with "erl" as Unix-filter > >>>> shell-command and couldn't get over the options (to erl) like > >>>> "-noshell", > >>>> "-s", "-detached", init:stop(), and so on in the documentation and in > >>>> the > >>>> FAQ. > >>>> > >>>> Thanks > >>>> ______________________________**_________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> > >>>> http://erlang.org/mailman/**listinfo/erlang-questions< > http://erlang.org/mailman/listinfo/erlang-questions> > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Prashant Sharma > Development Engineer > Pramati Technologies > Begumpet > > "Hare Krishna" > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Jun 11 17:18:09 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 11 Jun 2012 17:18:09 +0200 Subject: [erlang-questions] [ANN] Bullet 0.2.0 Message-ID: <4FD60C31.4030404@ninenines.eu> Hello! I'm glad to finally announce the first beta version of the Bullet project. Bullet is a Cowboy handler and associated Javascript library for maintaining a persistent connection between a client and a server. https://github.com/extend/bullet In short: it does websocket when they're available, it simulates them with long-polling otherwise, both transports have the same exact API client and server-side, and it reconnects automatically on disconnection. It's already used in production by myself at least. The documentation is probably a bit lacking so feel free to send some feedback. It currently requires Cowboy 0.6.0 (the newer version). Hope it'll be of interest to some of you! Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines From max.lapshin@REDACTED Mon Jun 11 19:05:25 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 11 Jun 2012 21:05:25 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: <6477442149500295393@unknownmsgid> References: <6477442149500295393@unknownmsgid> Message-ID: I have refused from parallel parsing because many of my CSV are gzipped and thus seek doesn't work there. From dmkolesnikov@REDACTED Mon Jun 11 19:08:56 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 11 Jun 2012 20:08:56 +0300 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <6477442149500295393@unknownmsgid> Message-ID: Agreed! I've gave up on parallel parsing for online jobs but still using it for offline batch jobs... - Dmitry On Jun 11, 2012, at 8:05 PM, Max Lapshin wrote: > I have refused from parallel parsing because many of my CSV are > gzipped and thus seek doesn't work there. From desired.mta@REDACTED Mon Jun 11 19:11:35 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 11 Jun 2012 19:11:35 +0200 Subject: [erlang-questions] [ANN] Bullet 0.2.0 In-Reply-To: <4FD60C31.4030404@ninenines.eu> References: <4FD60C31.4030404@ninenines.eu> Message-ID: On Mon, Jun 11, 2012 at 5:18 PM, Lo?c Hoguin wrote: > Hello! > > I'm glad to finally announce the first beta version of the Bullet project. > > Bullet is a Cowboy handler and associated Javascript library for > maintaining a persistent connection between a client and a server. > > ?https://github.com/extend/bullet > > In short: it does websocket when they're available, it simulates them with > long-polling otherwise, both transports have the same exact API client and > server-side, and it reconnects automatically on disconnection. > > It's already used in production by myself at least. The documentation is > probably a bit lacking so feel free to send some feedback. Hi, looks interesting indeed. It could be very useful if it had -callback specs, so we could implement a "bullet handler". Something like this: -module(bullet_handler). -callback init(eh(), eh(), list(), true | false | once) -> {ok, something(), term()}.= (I was too lazy to go to cowboy internals to look better at it). -callback stream(...) -> ... In that case: * we will know what callback modules are supposed to return * dialyzer will verify we are returning sane things Since -callback is supported only from R15B. I will ask a follow-up question for erlangers to find out erlang version using macros. Then -callback could be "constanted out". The client library looks awesome, thanks for that. :) -- Motiejus Jak?tys From max.lapshin@REDACTED Mon Jun 11 19:14:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 11 Jun 2012 21:14:19 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <6477442149500295393@unknownmsgid> Message-ID: On Mon, Jun 11, 2012 at 9:08 PM, Dmitry Kolesnikov wrote: > Agreed! > > I've gave up on parallel parsing for online jobs but still using it for offline batch jobs... > Are you using it for gzipped files? In fact I've refused from reading CSV and I translate CSV files to erlang:term_to_binary(Rows) format, which is many times faster, than any parsing in erlang. From essen@REDACTED Mon Jun 11 19:18:05 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 11 Jun 2012 19:18:05 +0200 Subject: [erlang-questions] [ANN] Bullet 0.2.0 In-Reply-To: References: <4FD60C31.4030404@ninenines.eu> Message-ID: <4FD6284D.8090509@ninenines.eu> On 06/11/2012 07:11 PM, Motiejus Jak?tys wrote: > On Mon, Jun 11, 2012 at 5:18 PM, Lo?c Hoguin wrote: >> Hello! >> >> I'm glad to finally announce the first beta version of the Bullet project. >> >> Bullet is a Cowboy handler and associated Javascript library for >> maintaining a persistent connection between a client and a server. >> >> https://github.com/extend/bullet >> >> In short: it does websocket when they're available, it simulates them with >> long-polling otherwise, both transports have the same exact API client and >> server-side, and it reconnects automatically on disconnection. >> >> It's already used in production by myself at least. The documentation is >> probably a bit lacking so feel free to send some feedback. > > Hi, > looks interesting indeed. > > It could be very useful if it had -callback specs, so we could > implement a "bullet handler". Something like this: > > -module(bullet_handler). > -callback init(eh(), eh(), list(), true | false | once) -> {ok, > something(), term()}.= > (I was too lazy to go to cowboy internals to look better at it). > -callback stream(...) -> ... > > In that case: > * we will know what callback modules are supposed to return > * dialyzer will verify we are returning sane things > > Since -callback is supported only from R15B. I will ask a follow-up > question for erlangers to find out erlang version using macros. Then > -callback could be "constanted out". > > The client library looks awesome, thanks for that. :) Yeah that's coming for the next version. We decided to stop supporting R14 after Cowboy 0.6 so all the callback stuff will be added after that release, without the macros. Thanks for the feedback! -- Lo?c Hoguin Erlang Cowboy Nine Nines From desired.mta@REDACTED Mon Jun 11 19:30:52 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 11 Jun 2012 19:30:52 +0200 Subject: [erlang-questions] OTP version in a built in constant Message-ID: Hi, I am writing some code which is supposed to run on R13-R15. For behaviors I would really like to use this: -callback my_fun() -> {oh, cool}. However, this is invalid syntax in {oh, cool}. -else behaviour_info(callbacks) -> [{my_fun, 0}]. -endif. What should I replace -ifdef ... -endif with? In other words, is there a built-in constant which has OTP version defined? -- Motiejus Jak?tys From watson.timothy@REDACTED Mon Jun 11 20:20:56 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 11 Jun 2012 19:20:56 +0100 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <6477442149500295393@unknownmsgid> Message-ID: Can you point out that code for the rest of us to look at max? On 11 Jun 2012, at 18:14, Max Lapshin wrote: > On Mon, Jun 11, 2012 at 9:08 PM, Dmitry Kolesnikov > wrote: >> Agreed! >> >> I've gave up on parallel parsing for online jobs but still using it for offline batch jobs... >> > > Are you using it for gzipped files? > In fact I've refused from reading CSV and I translate CSV files to > erlang:term_to_binary(Rows) format, which is many times faster, than > any parsing in erlang. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ronny.meeus@REDACTED Mon Jun 11 20:23:35 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Mon, 11 Jun 2012 20:23:35 +0200 Subject: [erlang-questions] socket unrecv In-Reply-To: References: Message-ID: On Mon, Jun 11, 2012 at 4:38 AM, Benoit Chesneau wrote: > On Sun, Jun 10, 2012 at 12:27 AM, CGS wrote: >> Hi Benoit, >> >> Forgive my lack of imagination, but how do you manage to listen to the same >> socket with another process? Can you pass the socket from the other API? >> Moreover, how do you make the other API to listen after your one without >> putting it in front of the other API? >> >> If you can access and pass the socket from the other API, I think you can >> bounce back the data toward the same socket (gen_tcp:send/2) because then >> the other API should receive the same data as the original transmitter >> (hopefully, the other side won't crash because it doesn't expect reply). >> But, still, this is only in the case you can receive the packets in the same >> time with the other API and the other API trashes the original packets >> (otherwise you just double the data in the other API). >> >> CGS >> > My case was a little more simpler here :) By API I meant another > internal code that wasn't supposed to be used by the initial code > doing the parsing and getting data from the socket. > > Anyway I fixed ?it by rewriting the first part. It works well now :) > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Hello on a standard socket interface (for example C) the MSG_PULL option is available. When this option is specified during the recv call, the data is passed to the calling application but it also stays present in the socket's buffers. Why is it not possible to use this option field in Erlang? Best regards, Ronny From max.lapshin@REDACTED Mon Jun 11 21:17:59 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 11 Jun 2012 23:17:59 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <6477442149500295393@unknownmsgid> Message-ID: https://github.com/maxlapshin/csv_reader This is the fastest csv reader, I've benchmarked. Nobody have shared faster code yet. But it is slower than file:read_file and erlang:binary_to_term From gordon@REDACTED Mon Jun 11 21:28:56 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 11 Jun 2012 20:28:56 +0100 Subject: [erlang-questions] Mnesia Deadlock Message-ID: Folks I am getting a transactional deadlock: mnesia:info(). ---> Processes holding locks <--- Lock: {{'hypernumbers.dev&9000&item',1339363066444676}, write, {tid,630,<0.263.0>}} Lock: {{'hypernumbers.dev&9000&local_obj',1339363066444676}, read, {tid,630,<0.263.0>}} Lock: {{'hypernumbers.dev&9000&site',1339363066444676}, write, {tid,630,<0.263.0>}} Lock: {{'hypernumbers.dev&9000&del_local',1339363066444676}, read, {tid,630,<0.263.0>}} Lock: {{'hypernumbers.dev&9000&group',"doobie"},write,{tid,632,<0.281.0>}} ---> Processes waiting for locks <--- Tid {tid,630,<0.263.0>} waits for write lock on oid {'hypernumbers.dev&9000&group', '______WHOLETABLE_____'} owned by {tid, 632, <0.281.0>} ---> Participant transactions <--- ---> Coordinator transactions <--- Tid: 630 (owned by <0.263.0>) Tid: 632 (owned by <0.281.0>) ---> Uncertain transactions <--- ---> Active tables <--- Process <0.263.0> is doing a foldl over a small table and process <0.281.0> is trying to write to it. How do I get out of this? It is RB1403. Gordon From ronny.meeus@REDACTED Mon Jun 11 21:48:20 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Mon, 11 Jun 2012 21:48:20 +0200 Subject: [erlang-questions] Strange observation running in an SMP environment. Message-ID: Hello What I find a really interesting feature about Erlang is that the user does not need to take care of protecting its data against concurrent access and all issues that popup when going from the single core to a multi-core environment. To play with this I have made a test application which implements a process ring (like implemented a lot as a starters example). My code is based on the process ring running on the Raspberry Pi that makes a LED flash each time the token passes the head of the ring. The code is available at bitbucket: https://bitbucket.org/meeusr/erl-relay The machine on which I'm running the test is an Intel i7 (4 cores): vendor_id : GenuineIntel model name : Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz cpu MHz : 2933.000 cache size : 8192 KB The basic idea is that I create a LED process that manages the state of the faked led. Next to this a ring is created, which contains a head node and a lot of non-head nodes (1000 in total in the example below). Once this ring is created, I can send a message to the head process (hidden behind the relay:start) and this message will be passed around the ring as indicated in the start. Each time the messages passes the head node, the state of the LED is changed by sending a message to the LED process. Once all cycles are completed, the time needed to process the messages is printed. $ erl Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> Led = led:start(1). <0.33.0> 2> Head = relay:create_ring(1000,Led). <0.1034.0> 3> relay:start(Head,10000). ok Finished. Time:3474789 If I force the Erlang VM to run on 1 core only and repeat the same test as above, I see that the execution time is indeed longer (changes from 3474789ms to 4186125ms). $ taskset 1 erl Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> 1> Led = led:start(1). <0.33.0> 2> Head = relay:create_ring(1000,Led). <0.1034.0> 3> relay:start(Head,10000). ok 4> Finished. Time:4186125 4> q(). ok I want to stress the system a bit more and create 2 rings, which will run in parallel: $ erl Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> Led = led:start(1). <0.33.0> 2> Head = relay:create_ring(1000,Led). <0.1034.0> 3> Led1 = led:start(1). <0.1036.0> 4> Head1 = relay:create_ring(1000,Led1). <0.2037.0> 5> relay:start(Head,10000),relay:start(Head1,10000). ok Finished. Time:7746973 Finished. Time:7824631 An observation is that the execution time is more than doubled (7746973ms and 7824631ms versus 3474789ms). But the strange thing I see that if I start to force the VM to run on 1 core, the performance actually becomes better (see below). Forcing the erlang VM on 1 core ("taskset 1 erl") leads to following result: 7> relay:start(Head,10000),relay:start(Head1,10000). ok Finished. Time:6564535 Finished. Time:6755538 Running on 2 cores ("taskset 3 erl") results in: 7> relay:start(Head,10000),relay:start(Head1,10000). ok Finished. Time:7186033 Finished. Time:7196646 Running without a taskset ("erl") results in: 7> relay:start(Head,10000),relay:start(Head1,10000). ok Finished. Time:7490020 Finished. Time:7814036 So this observation is exactly the opposite of what I would expect. I would expect that the more cores are being used, the faster the execution completes but the observation is that the more core are being used, the slower it becomes. Now one can say that this is not real parallel work since the token passes the nodes of the ring one by one, but for the 2 rings example I would expect that if the number of cores is doubled, the execution should be finished in more or less the same time as the execution of the 1 ring instance. Can somebody explain this? Many thanks, Ronny From n.oxyde@REDACTED Mon Jun 11 22:12:25 2012 From: n.oxyde@REDACTED (nox) Date: Mon, 11 Jun 2012 22:12:25 +0200 Subject: [erlang-questions] Using Rebar to guess Erlang version (was: Re: [ANN] Bullet 0.2.0) Message-ID: You can do it with Rebar like this: https://github.com/extend/cowboy/pull/217 -- Anthony Ramine Le 11 juin 2012 ? 19:11, Motiejus Jak?tys a ?crit : Since -callback is supported only from R15B. I will ask a follow-up question for erlangers to find out erlang version using macros. Then -callback could be "constanted out". From ok@REDACTED Tue Jun 12 00:02:02 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 12 Jun 2012 10:02:02 +1200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> Message-ID: <6D4EEFCB-9D7B-41A1-A463-A227C173077E@cs.otago.ac.nz> On 11/06/2012, at 4:53 PM, Michael Turner wrote: >> In my wildest dreams I couldn't have asked for a better example of a >> Wiki going wrong. > > Then you don't have much experience at all with editing wikis, I have repeatedly said that I have little experience of *editing* wikis and that my opposition is based on the serious frustration I've repeatedly experienced *using* them. Apparently you also suffer from amnesia. > > Again, you're falling back on "wikis suck because I found something > bad." I have never made any claim that general. I say that *documentation* wikis are in my experience bad; far far worse than the existing Erlang documentation. I've actually pointed to Wikipedia as unusually *good* in my experience. I may be wrong about that: http://catless.ncl.ac.uk/Risks/26.79.html#subj6.1 60% of Wikipedia entries about companies contain errors When respondents attempted to engage editors through Wikipedia's "Talk" pages to request factual corrections to entries, 40 percent said it took "days" to receive a response, 12 percent indicated "weeks," while 24 percent never received any type of response. ... Of those who were familiar with the process of editing Wikipedia entries, 23 percent said making changes was "near impossible." Twenty-nine percent said their interactions with Wikipedia editors were "never productive." That's citing http://www.sciencedaily.com/releases/2012/04/120417113527.htm?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+sciencedaily+(ScienceDaily:+Latest+Science+News) which has a number of links that are very positive about Wikipedia; this particular article was based on Marcia W. DiStaso. "Measuring Public Relations Wikipedia Engagement: How Bright is the Rule?" Public Relations Journal, Vol. 6, No. 2 and while I was initially apprehensive because our library's e-subscription lapsed, it turns out the PDF of that article is at http://www.prsa.org/Intelligence/PRJournal/Documents/2012DiStaso.pdf The title of the RISKS article is a little misleading: out of the people surveyed, 60% of those whose companies had a Wikipedia entry, who knew it had, and who had checked it, had found factual errors worthy of correction. We have no information about the error rate in the entries for companies whose PR people _don't_ know it has a Wikipedia page (25% of the PR people surveyed said they didn't know if their company had a Wikipedia entry or not, which should give their employers pause for thought). 21% of the people who reported errors had found spelling errors, which gives _me_ pause for thought about the utility of a Wiki as a device for correcting spelling. The paper has some sobering figures about corrections being undone. "Half of respondents who have either directly edited or used the talk pages to make edits indicated that they believe the process of making changes to a company or client?s Wikipedia article is typically time consuming (n=275), but 27% stated that it was easy (n=152), while 23% said it was near impossible (n=127)." There's a followup in RISKS http://catless.ncl.ac.uk/Risks/26.89.html#subj18 by someone who is more concerned about one UK PR firm who were caught adding positive information about their clients than about other companies' difficulty getting wrong information correctly. > You're not asking, "Why does this bad thing happen, and is there > anything to be done to prevent it?" No, and that's because I'm the wrong person to do so. That's *your* role in this debate. > In fact, these problems are quite > manageable, and Wikipedia, in the interests of gathering the largest > possible membership, leaves editing quite open. It also means some > information is unreliable at one time or another. Anyone who reads > Wikipedia without keeping that in mind is simply a fool. The thing is, the more you talk about "management", the less a Wiki sounds substantively different from the system we have right now. A restricted pool of editors? We have that now: the main difference is that you expect to become one of them. Low barrier to entry? We have that now. The one major difference is that (like many of the PR professionals surveyed in that paper) I would be happiest with review-before-publication, where you seem to be happy with review-after-publication-if-ever. I repeat something I've said before, in case you have forgotten. Nothing stops you setting up an Erlang documentation Wiki right now. From maruthavanan_s@REDACTED Tue Jun 12 06:55:43 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Tue, 12 Jun 2012 00:55:43 -0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: , <6477442149500295393@unknownmsgid>, , , , , Message-ID: Hi, Thanks Max! Hope you understood my situation, I want to open the file and do not want to load the entire csv into erlang VM as file size may be in Gigabytes. I may be process hardly 50 lines at any point of time in a process. As a newbie, Can I make use of this? If so please guide me as I could not follow the csv_benchmark.erl Thanks,Marutha > Date: Mon, 11 Jun 2012 23:17:59 +0400 > From: max.lapshin@REDACTED > To: watson.timothy@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Large Flat file operations - optimization. > > https://github.com/maxlapshin/csv_reader > > This is the fastest csv reader, I've benchmarked. Nobody have shared > faster code yet. > > But it is slower than file:read_file and erlang:binary_to_term > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Jun 12 07:49:58 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 12 Jun 2012 14:49:58 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: <6D4EEFCB-9D7B-41A1-A463-A227C173077E@cs.otago.ac.nz> References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <6D4EEFCB-9D7B-41A1-A463-A227C173077E@cs.otago.ac.nz> Message-ID: On Tue, Jun 12, 2012 at 7:02 AM, Richard O'Keefe wrote: >> Again, you're falling back on "wikis suck because I found something >> bad." > > I have never made any claim that general. > I say that *documentation* wikis are in my experience bad; > far far worse than the existing Erlang documentation. Again: if all the existing Erlang/OTP documentation were suddenly in a wiki, word for word, would it be "far, far worse" than the existing documentation? That, and the quality of the ensuing administration, are the only two things that matter for this discussion. NOT that somebody added some incorrect information about you to your Wikipedia biography. > I've actually pointed to Wikipedia as unusually *good* in my > experience. ?I may be wrong about that: It's incredibly bad in some places, and very good in others. That, again, is a function of an administrative choice: Wikipedia can be edited by anybody, even anonymously. High-interest articles tend to have a lot of watchers to help maintain quality. Low-interest articles can have crap in them that nobody notices for months. > http://catless.ncl.ac.uk/Risks/26.79.html#subj6.1 > > ? ? ? ?60% of Wikipedia entries about companies contain errors > ? ? ? ?When respondents attempted to engage editors through > ? ? ? ?Wikipedia's "Talk" pages to request factual corrections > ? ? ? ?to entries, 40 percent said it took "days" to receive a > ? ? ? ?response, 12 percent indicated "weeks," while 24 percent > ? ? ? ?never received any type of response. ... This is ridiculous. If you think a factual correction is required to a Wikipedia article, DO IT YOURSELF. If you have the editing skills required to alter a Talk page, you probably know enough to make the change you think is necessary in the article itself. Afraid to touch an article? Then don't get involved in the first place. One of the Five Pillars of Wikipedia is "Be bold." http://en.wikipedia.org/wiki/Wikipedia:Be_bold If you approach Wikipedia in something other than the spirit Wikipedia itself strongly encourages, you shouldn't complain about the results. Editors with a strong interest in articles set up watchlists mainly *for* the articles, not so much for the article Talk pages. You want attention to a problem in an article? Apply your own attention first: make what you think is a fix. Who knows? You might actually be right! > ..... Of those who were > ? ? ? ?familiar with the process of editing Wikipedia entries, > ? ? ? ?23 percent said making changes was "near impossible." > ? ? ? ?Twenty-nine percent said their interactions with Wikipedia > ? ? ? ?editors were "never productive." This is also a ridiculous criticism of Wikipedia when, if anything, it's to Wikipedia's credit. Go look at Articles for Deletion http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion And that's just just the tip of the iceberg of the generalized egomania out there, an obstacle that Wikipedia is always trying to get past while staying as open as possible. A dismayingly high proportion of those who get "familiar with the process of editing Wikipedia entries" are people who familiarized themselves in order insert self-serving information of one kind or another, including entire articles about themselves, their non-notable music careers, their non-notable companies, their non-notable self-published novels. Even those who actually DO clear Wikipedia's bar of notability (and their fans) will insist on trying to add information to the resulting articles that doesn't qualify as properly sourced or written from a neutral point of view. http://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view And how likely is it that a survey of *PR* "professionals" (corporate flacks) who tried to edit Wikipedia would have tried to do it without violating http://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view and guidelines about Conflict of Interest? http://en.wikipedia.org/wiki/Wikipedia:Conflict_of_interest This is a function of Wikipedia's access control, which is pegged at the most liberal end of the spectrum. It's also possible to set up a MediaWiki-based wiki that permits nobody except a tiny coterie of administrators to contribute edits. And there is a range of possibilities in between, including customized access control algorithms for those who know PHP/MySQL. (Yeah, yeah, I know: that choice of technologies is a bug in itself. Whatever.) > The title of the RISKS article is a little misleading: > out of the people surveyed, 60% of those whose companies had a Wikipedia > entry, who knew it had, and who had checked it, had found factual errors > worthy of correction. Wikipedia has a number of areas where it's weak. In my experience, except for the more prominent companies, business is one of them. Bad information about companies is one syndrome in that topic area. Another much more serious one, however, owes to the PR people surveyed in the paper you cite: companies using Wikipedia for self-promotion. None of which is relevant to this discussion -- unless perhaps Ericsson wants to claim that it invented Lamport clocks, without any supporting independent references for the statement. No, we're talking about what's *theoretically* possible in putting the Erlang/OTP documentation into a wiki, under access control policies that would presumably be acceptable under the terms of licensing by Ericsson, and that would favor contributions of reasonable quality. I've already suggested one strategy for access control, and I've repeated it several times since. Find it and tell me what's wrong with it, if you want to criticize it. > ...?21% of the people who reported errors [in business articles] had found > spelling errors, which gives _me_ pause for thought about the utility > of a Wiki as a device for correcting spelling. That's probably because when you can see from the first paragraph that the Wikipedia article is yet another puff-piece that got in there somehow, you might as well just go to the company website. (And let your eyes glaze over reading that.) When people search on a general topic ("bees", "galaxies"), the Wikipedia page often head the rankings. And the #1 result gets most of the attention. When people search on a company, they usually they get the company website first. Since so many Wikipedia articles about companies are transparently nothing but plagiarized PR (often self-plagiarized) from company websites, I'm not surprised that they get so little copyediting attention, for any added information beyond the PR. > The paper has some sobering figures about corrections being undone. > > "Half of respondents who have either directly edited or used the talk > ?pages to make edits indicated that they believe the process of making > ?changes to a company or client?s Wikipedia article is typically time > ?consuming (n=275), but 27% stated that it was easy (n=152), while > ?23% said it was near impossible (n=127)." "Sobering"? I'm laughing like a hyena. I'm pretty sure that the overwhelming majority of those frustrated 23%-29% were trying to do something Wikipedia actually rules against: stating "facts" without reliable-source backup. http://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable_sources And that's not even getting into "neutral point of view" and "conflict of interest" issues. Let's say that the Wikipedia article on Erlang/OTP were to say, on nothing more than the strength of belief on the part of some Ericsson veterans or current employees, that Lamport clocks were invented as part of tracing, in the late 70s, and for the AXE series of switches. I.e., that there was no citation to anything qualifying under Wikipedia's Reliable Source rules. In that case, the statement should be deleted. Anybody who insisted putting it back into the article based only on their own personal opinion could, if they were sufficiently insistent, end up being banned from editing Wikipedia. Would they report that they were "frustrated"? Probably. SHOULD they have been "frustrated"? Probably. Even when something is true, it's not enough. In general, it has to be reported elsewhere, independently. I did some edits on http://en.wikipedia.org/wiki/Brad_Cox and contacted him about getting some reliable sources for certain points in his resume (since he always seemed rather unsung to me.) He asked, "What do you do when you yourself are the only source?" I replied saying that I didn't doubt him personally, but that the rules existed for good reason: the incredible level of dishonesty and egregious self-promotion that happens on Wikipedia if you don't have such rules. >> You're not asking, "Why does this bad thing happen, and is there >> anything to be done to prevent it?" > > No, and that's because I'm the wrong person to do so. > That's *your* role in this debate. I don't need to ask "Why does this bad thing happen?" Bad? I've already seen it all. Fought it all. >> In fact, these problems are quite >> manageable, and Wikipedia, in the interests of gathering the largest >> possible membership, leaves editing quite open. It also means some >> information is unreliable at one time or another. Anyone who reads >> Wikipedia without keeping that in mind is simply a fool. > > The thing is, the more you talk about "management", the less a Wiki > sounds substantively different from the system we have right now. > A restricted pool of editors? ?We have that now FINALLY you acknowledge a point I've made repeatedly: most of your concerns about bad information getting into a wiki can be addressed by MediaWiki access control, which Wikipedia only uses on a case-by-case basis. > the main difference > is that you expect to become one of them. Actually, quite the contrary, at least on the specific point of getting the seq_trace documentation to mention that it implements Lamport clocks if only I submit a patch. The internal corporate self-mythologizing on this point appears to be quite strong, if the views of two former employees are any indication. >?Low barrier to entry? We have that now. Eye of the beholder again. In the Erlang/OTP documentation I find errors in grammar, punctuation and spelling that never seem to get fixed. There are points in the text where a link would be enormously helpful, but those links never seem to get added. If the barrier to entry for fixes is so "low", as you claim, why weren't these things fixed long ago? > Nothing stops you setting up an Erlang documentation Wiki right now. Unless if violates the Ericsson license terms. And then there's this: the discouraging impression that highly intelligent people can nevertheless reject ideas for reasons stemming almost entirely from their own ignorance and prejudices, and who will, having taken such a stand, go cherry-picking all subsequent evidence they present, from of irrelevant contexts if necessary, and who will quickly change the subject to one those picked cherries when shown evidence that they are wrong on a given point. Case in point: you. -michael turner From dave@REDACTED Tue Jun 12 08:10:03 2012 From: dave@REDACTED (Dave Cottlehuber) Date: Tue, 12 Jun 2012 08:10:03 +0200 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <6D4EEFCB-9D7B-41A1-A463-A227C173077E@cs.otago.ac.nz> Message-ID: On 12 June 2012 07:49, Michael Turner wrote: [snip] I value this list above almost all other ones that I subscribe to, for the high quality of contributions and the care & time put into the responses. It's a model technical community. Unfortunately this thread has nothing to do with Erlang any more, and at times could be misinterpreted as ad-hominem arguments, which I am sure was never the intention of those contributors. A patch to enable smoother contributions to the Erlang docs would be welcomed by all; given the complexity some direction & guidelines from the OTP team might facilitate this. What functionality is required? A+ Dave From dangud@REDACTED Tue Jun 12 08:29:48 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 12 Jun 2012 08:29:48 +0200 Subject: [erlang-questions] Mnesia Deadlock In-Reply-To: References: Message-ID: That is not a deadlock, You have one process holding the locks <0.263.0> and one waiting to get a table lock <0.281.0> Whenever 263 is finished 281 will get the lock. /Dan On Mon, Jun 11, 2012 at 9:28 PM, Gordon Guthrie wrote: > Folks > > I am getting a transactional deadlock: > > mnesia:info(). > ---> Processes holding locks <--- > Lock: {{'hypernumbers.dev&9000&item',1339363066444676}, > ? ? ? write, > ? ? ? {tid,630,<0.263.0>}} > Lock: {{'hypernumbers.dev&9000&local_obj',1339363066444676}, > ? ? ? read, > ? ? ? {tid,630,<0.263.0>}} > Lock: {{'hypernumbers.dev&9000&site',1339363066444676}, > ? ? ? write, > ? ? ? {tid,630,<0.263.0>}} > Lock: {{'hypernumbers.dev&9000&del_local',1339363066444676}, > ? ? ? read, > ? ? ? {tid,630,<0.263.0>}} > Lock: {{'hypernumbers.dev&9000&group',"doobie"},write,{tid,632,<0.281.0>}} > ---> Processes waiting for locks <--- > Tid {tid,630,<0.263.0>} waits for write lock on oid > {'hypernumbers.dev&9000&group', > > '______WHOLETABLE_____'} owned by {tid, > > ? ? ? ? ? ? ? ? 632, > > ? ? ? ? ? ? ? ? <0.281.0>} > ---> Participant transactions <--- > ---> Coordinator transactions <--- > Tid: 630 (owned by <0.263.0>) > Tid: 632 (owned by <0.281.0>) > ---> Uncertain transactions <--- > ---> Active tables <--- > > Process <0.263.0> is doing a foldl over a small table and process > <0.281.0> is trying to write to it. > > How do I get out of this? > > It is RB1403. > > Gordon > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Tue Jun 12 09:13:20 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 12 Jun 2012 16:13:20 +0900 Subject: [erlang-questions] FOP (was: Re: Trace-Driven Development) In-Reply-To: References: <3B0669E4-C4A2-433F-89C9-5CD94F416133@cs.otago.ac.nz> <6D4EEFCB-9D7B-41A1-A463-A227C173077E@cs.otago.ac.nz> Message-ID: > Unfortunately this thread has nothing to do with Erlang any more, and at > times could be misinterpreted as ad-hominem arguments, which I am sure > was never the intention of those contributors. It still has to do with a proposal for a process for improving the Erlang documentation, so it's not "nothing to do with Erlang anymore." However, it has gotten to the point where I'm expending a great deal of effort on addressing a certain kind of objections to the proposal: that an Erlang/OTP wiki, even if it started as a literal copy of the existing documentation, and restricted access to editing for those most likely to edit in good faith, will necessarily manifest all of the pathologies of wikis that "just grew" without very little (except after-the-fact) access control for contributors who misbehave. And arguing against position IS a waste of time. > A patch to enable smoother contributions to the Erlang docs would be > welcomed by all; given the complexity some direction & guidelines from the > OTP team might facilitate this. > > What functionality is required? Can this be done with a mere "patch". I don't see how, unless you mean "patch" figuratively. (I *have* proposed that the documentation have a conspicuous solicitation for help, with a link to a "getting started with documentation patches" that supplies *all* of the instruction, inline, to get from "I want to change this word to a plural" to a submitted patch, even if the submitted has no github account to start. Is that what you mean?) My answer? Given that Erlang/OTP has never, and probably will never, hire a technical writer? (1) A wiki that starts as a literal copy of the existing documentation. (2) The (I believe, small) amount of scripting required to periodically and selectively reflect changes made to the wiki back into the existing documentation sources, either day-at-a-time or to reflect new releases of the software. (3) Access to adminship/editorship limited to Ericsson Erlang/OTP employees, plus anyone on this list who applies for membership in the ranks of Erlang/OTP wiki editorship, subject to the approval of an appointed Erlang/OTP employee. (Alternatively: the signup scripting might compare the e-mail address you submit with archives of this mailing list, and only automatically grant editing rights to those who have been a contributor for more than a few weeks. Subject to later revocation by Ericsson.) That last puts ultimate responsibility in the appropriate place, I believe. Erlang/OTP is not just an open source project, but also a corporate product. As I've written before on this thread, I don't think an Erlang/OTP documentation wiki effort can survive without some degree of substantive official support. The question is: how much? The design of (3) above should aim at keeping the amount of employee *monitoring* of wiki edits to an absolute minimum - perhaps a five-minute daily review of "Recent changes" would do it, with much less time if there were in fact no changes that day. I'd expect there would be appreciable peaks of employee activity on the wiki only around the times when they'd see peak effort on documentation anyway: shortly before a new release. Ideally, at that point, employees simply change a header in each article from "Planned for the X.Y.Z release" to "Made available [or removed, in the case of finalized deprecations] in release X.Y.Z" (with appropriate changes elsewhere in the article as needed), while bumping anything that didn't actually make it into Rel X.Y.Z into a section, "Planned for the X.Y.Z+1 release." These are the kinds of edits they'd have to make in *some* eventually-public document anyway (e.g., release notes, manual pages, etc.) -michael turner From max.lapshin@REDACTED Tue Jun 12 09:19:43 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 12 Jun 2012 11:19:43 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <6477442149500295393@unknownmsgid> Message-ID: Read, 10 KBytes, split them with binary:split and parse 10 lines. Remember their byte size and calculate offset of next 10 lines. From maruthavanan_s@REDACTED Tue Jun 12 12:44:55 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Tue, 12 Jun 2012 06:44:55 -0400 Subject: [erlang-questions] Check for file used by any process Message-ID: Hi, Is there any BIF where I can check whether a file is used by any other process? I have the below scenario. A directory being watch at a regular interval, I have to process the file only if that is being written fully. If some other process is writing the file, I should check and avoid those. How can I achieve this. I can do as os:cmd("lsof | grep "). But wanted to check is there any other way I can check that in Erlang BIF. Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Tue Jun 12 15:42:42 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Tue, 12 Jun 2012 14:42:42 +0100 Subject: [erlang-questions] Mnesia Deadlock In-Reply-To: References: Message-ID: > That is not a deadlock, It's much worse than that :( > Whenever 263 is finished 281 will get the lock. 263 is trying to execute a fn that has been passed in by 281 - one process has a lock on a table and synchronously asks another fun to acquire another lock on the same table, do something, and then return :( This isn't a deadlock is 'a demonic apparition from the spirit world-lock' And its my fault :( Gordon On 12 June 2012 07:29, Dan Gudmundsson wrote: > That is not a deadlock, > > You have one process holding the locks <0.263.0> > and one waiting to get a table lock <0.281.0> > > Whenever 263 is finished 281 will get the lock. > > /Dan > > On Mon, Jun 11, 2012 at 9:28 PM, Gordon Guthrie wrote: >> Folks >> >> I am getting a transactional deadlock: >> >> mnesia:info(). >> ---> Processes holding locks <--- >> Lock: {{'hypernumbers.dev&9000&item',1339363066444676}, >> ? ? ? write, >> ? ? ? {tid,630,<0.263.0>}} >> Lock: {{'hypernumbers.dev&9000&local_obj',1339363066444676}, >> ? ? ? read, >> ? ? ? {tid,630,<0.263.0>}} >> Lock: {{'hypernumbers.dev&9000&site',1339363066444676}, >> ? ? ? write, >> ? ? ? {tid,630,<0.263.0>}} >> Lock: {{'hypernumbers.dev&9000&del_local',1339363066444676}, >> ? ? ? read, >> ? ? ? {tid,630,<0.263.0>}} >> Lock: {{'hypernumbers.dev&9000&group',"doobie"},write,{tid,632,<0.281.0>}} >> ---> Processes waiting for locks <--- >> Tid {tid,630,<0.263.0>} waits for write lock on oid >> {'hypernumbers.dev&9000&group', >> >> '______WHOLETABLE_____'} owned by {tid, >> >> ? ? ? ? ? ? ? ? 632, >> >> ? ? ? ? ? ? ? ? <0.281.0>} >> ---> Participant transactions <--- >> ---> Coordinator transactions <--- >> Tid: 630 (owned by <0.263.0>) >> Tid: 632 (owned by <0.281.0>) >> ---> Uncertain transactions <--- >> ---> Active tables <--- >> >> Process <0.263.0> is doing a foldl over a small table and process >> <0.281.0> is trying to write to it. >> >> How do I get out of this? >> >> It is RB1403. >> >> Gordon >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Tue Jun 12 16:21:41 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 12 Jun 2012 16:21:41 +0200 Subject: [erlang-questions] [ANN] mochicow 0.4 Message-ID: Hi all, I finally released mochicow 0.4 [1]. There is now a nice README [2] and you can upgrade a cowboy handler to use a mochiweb request. Which means that you can use both cowboy and mochiweb handlers at the same time which allows a smooth transition from mochiweb to cowboy if you need it (see the demo [3] or the README for a quick howto) . It is using a custom mochicow_request module in place of the mochiweb one to do that. This module keep a receive buffer around and pass it to the cowby protocol when it ends. Previous attempt was removing the use of the buffer. But this change was problematic for some cowboy handlers. The change is transparent for the users. I tested it on rcouch [4]?my own fork of Apache CouchDB and all tests pass. Enjoy, - beno?t [1] https://github.com/benoitc/mochicow [2] https://github.com/benoitc/mochicow/blob/master/README.md [3] https://github.com/benoitc/mochicow/tree/master/examples/upgrade_demo [4] https://github.com/refuge/rcouch/compare/master...rcouch-0.6 From spawn.think@REDACTED Tue Jun 12 17:05:42 2012 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 12 Jun 2012 17:05:42 +0200 Subject: [erlang-questions] TLB shootdowns Message-ID: Hi everyone, We are running a customized version of ejabberd on our servers (Debian Squeeze). We are seeing a high number of interrupts (TLB shootdowns), around 14K at peak. Did anyone experienced something similar before? Any hint where to look? Also, do you know if using hugetlb in linux would help? Does erlang get any benefits from that? Best Regards, Ahmed Omar -------------- next part -------------- An HTML attachment was scrubbed... URL: From corticalcomputer@REDACTED Tue Jun 12 19:31:21 2012 From: corticalcomputer@REDACTED (G.S.) Date: Tue, 12 Jun 2012 13:31:21 -0400 Subject: [erlang-questions] DXNN: Topology and Parameter Evolving Universal Learning Network system/platform, released to GitHub. Message-ID: Hello all, DXNN [1,4] is Topology and Parameter Evolving Universal Learning Network (TPEULN) system, similar to topology and weight evolving artificial neural network, but more general, and not constrained to the use of only sigmoid based activation function neurons. Erlang was chosen because of its perfect and complete mapping to the neural network architecture. DXNN is a TPEULN platform that uses direct and indirect encoding (neural and substrate respectively [5]), has a cross-validation system for experimentation, decoupled sensor/actuator systems, decoupled learning/selection/... algorithms (in MK2), a built in 2d world simulator called flatland for ALife experiments (all in gs()). The second generation (mk2) DXNN is available as a branch of the original project, and is a clean implementation of this computational intelligence evolving system. It is also the system explained and created in my Springer book: Handbook of Neuroevolution Through Erlang [2,3], with a foreword written by Joe Armstrong. The book will go into print this September. There are not a lot of comments within the source code on github, but I will continue to add more comments as time permits. Upcoming features: 1. Visualisation system. 2. New selection algorithm modules. 3. New speciation and diversification functions. 4. An improved cross-validation system for the experiment database. 5. Full population backup, so that all agents are saved, and only manually deleted at the researcher's request (they don't take much space, and it would make for an interesting visualisation, and ability to traverse from the seed agent to the current agent). -Gene [1] https://github.com/CorticalComputer/DXNN First generation DXNN has a convoluted implementation. DXNN mk2 is a very clean implementation and is currently on the non master branch, it will eventually overwrite the master branch but both have the same features (almost) at this time. [2] http://www.springer.com/computer/swe/book/978-1-4614-4462-6 [3] http://www.amazon.com/Handbook-Neuroevolution-through-Erlang-Gene/dp/1461444624/ref=sr_1_1?ie=UTF8&qid=1338163875&sr=8-1 [4] http://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher [5] http://arxiv.org/abs/1111.5892 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jun 12 21:36:49 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 12 Jun 2012 21:36:49 +0200 Subject: [erlang-questions] Strange observation running in an SMP environment. In-Reply-To: References: Message-ID: <4FD79A51.8080805@erlang-solutions.com> On Mon Jun 11 21:48:20 2012, Ronny Meeus wrote: [Ring benchmark - showing that 2 rings is considerably slower than one] > Can somebody explain this? There are a couple of factors which plays a role here. First of all, you can use the nice percept tool to look at your code: https://gist.github.com/2919608 run t:start(), t:analyze(), t:server(). Then point your browser to localhost:8989 and have fun :) The first problem is that the concurrency of your rings are 1 both of them. This means you can at most keep one core busy on each ring and your maximum speedup is thus 2. To see this, note that one one process can run in the ring at a time. All others are waiting for the token to come to them. This is also shown in the percept graph: At the start, you have a very high concurrency level because you just spawned a lot of processes and they are now runable. Then the ring connect phase happens and this blocks many of them in a receive waiting for a {connect, ...} message. When we run around the ring, the concurrency level is hovering around 2 to 4. Which makes sense since we added two Led-processes which also needs to run from time to time. The second problem is that your benchmark is not doing work. So what you are measuring is locking overhead from the Erlang VM and your kernel. Passing a message is VERY fast in Erlang and that is all you are doing. If you have 4 cores in your machine, they are stealing work from each other all the time and this accounts for the overhead. You could try playing around with `erl +sbt tnnps` as an erlang flag, but it may/may not give you anything here. it is also worth trying to disable SMP entirely `erl -smp disable` since this avoids any kind of locking. In some cases that is faster for some workloads. And yours might be one. -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK From magnus.henoch@REDACTED Tue Jun 12 22:01:17 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Tue, 12 Jun 2012 21:01:17 +0100 (BST) Subject: [erlang-questions] Dialyzer to cache Hipe compilation In-Reply-To: <1ec13596-4de5-4c15-81da-41df185887db@knuth> Message-ID: <9477fbac-fe71-4fae-970a-d60ee4a5c839@knuth> Hi all, I finally decided to try to do something about Dialyzer's startup time, as it's compiling modules to native code for about a minute on my machine. The results are here: https://github.com/legoscia/otp/compare/dialyzer-hipe-cache git clone -b dialyzer-hipe-cache https://github.com/legoscia/otp.git The only changed files are dialyzer_cl.erl and hipe.erl; you can compile them manually and copy them into the appropriate ebin directories. This makes Dialyzer cache the compiled modules into ~/.dialyzer_hipe_cache the first time it's run. There's probably a nicer solution to this, like building the modules in question at install time, but this hack seems to work for me... Feedback is very welcome. Regards, -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From ronny.meeus@REDACTED Tue Jun 12 22:39:02 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Tue, 12 Jun 2012 22:39:02 +0200 Subject: [erlang-questions] Strange observation running in an SMP environment. In-Reply-To: <4FD79A51.8080805@erlang-solutions.com> References: <4FD79A51.8080805@erlang-solutions.com> Message-ID: On Tue, Jun 12, 2012 at 9:36 PM, Jesper Louis Andersen wrote: > On Mon Jun 11 21:48:20 2012, Ronny Meeus wrote: > > [Ring benchmark - showing that 2 rings is considerably slower than one] > >> Can somebody explain this? > > > There are a couple of factors which plays a role here. First of all, you can > use the nice percept tool to look at your code: > > https://gist.github.com/2919608 > > run t:start(), t:analyze(), t:server(). Then point your browser to > localhost:8989 and have fun :) > Thanks for this hint. It is a wonderful tool that I was not aware of. This is also one of the things I like about Erlang: it comes with a lot of tools and libraries out of the box. > The first problem is that the concurrency of your rings are 1 both of them. > This means you can at most keep one core busy on each ring and your maximum > speedup is thus 2. To see this, note that one one process can run in the > ring at a time. All others are waiting for the token to come to them. This > is also shown in the percept graph: At the start, you have a very high > concurrency level because you just spawned a lot of processes and they are > now runable. Then the ring connect phase happens and this blocks many of > them in a receive waiting for a {connect, ...} message. When we run around > the ring, the concurrency level is hovering around 2 to 4. Which makes sense > since we added two Led-processes which also needs to run from time to time. > > The second problem is that your benchmark is not doing work. So what you are > measuring is locking overhead from the Erlang VM and your kernel. Passing a > message is VERY fast in Erlang and that is all you are doing. If you have 4 > cores in your machine, they are stealing work from each other all the time > and this accounts for the overhead. You could try playing around with `erl > +sbt tnnps` as an erlang flag, but it may/may not give you anything here. > > it is also worth trying to disable SMP entirely `erl -smp disable` since > this avoids any kind of locking. In some cases that is faster for some > workloads. And yours might be one. > > -- > Jesper Louis Andersen > ?Erlang Solutions Ltd., Copenhagen, DK I added a load consumer to the application: each time a worker receives a message, it forwards the message and generates some load (code uploaded to BitBucket). After this change the perftool clearly shows that on average something like 15 processes are consuming load. These are the new measured times (for a ring of 10 threads and 10 message roundtrips): erl -smp disable Finished. Time:12769406 Finished. Time:12843219 taskset 1 erl (1 core) Finished. Time:13701993 Finished. Time:14170071 taskset 3 erl (2 cores) Finished. Time:5954985 Finished. Time:7362113 erl (4 cores): Finished. Time:3542247 Finished. Time:3555036 So this looks better: now the execution time reduces linearly with the number of cores. Also running on 1 core with SMP disabled runs faster than running in SMP mode on 1 core using taskset. Does the issue that I had (pure message passing without extra load) have anything to do with the fact that I'm running on Linux? I sometimes have the impression (not in the context of my Erlang playing) that the scheduler does not always makes the right decision during the migration of processes between cores? Many thanks for the quick response. -- Regards, Ronny From tuncer.ayaz@REDACTED Tue Jun 12 23:50:02 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 12 Jun 2012 23:50:02 +0200 Subject: [erlang-questions] Dialyzer to cache Hipe compilation In-Reply-To: <9477fbac-fe71-4fae-970a-d60ee4a5c839@knuth> References: <1ec13596-4de5-4c15-81da-41df185887db@knuth> <9477fbac-fe71-4fae-970a-d60ee4a5c839@knuth> Message-ID: On Tue, Jun 12, 2012 at 10:01 PM, Magnus Henoch wrote: > Hi all, > > I finally decided to try to do something about Dialyzer's startup time, > as it's compiling modules to native code for about a minute on my machine. > > The results are here: > https://github.com/legoscia/otp/compare/dialyzer-hipe-cache > git clone -b dialyzer-hipe-cache https://github.com/legoscia/otp.git > > The only changed files are dialyzer_cl.erl and hipe.erl; you can compile > them manually and copy them into the appropriate ebin directories. ?This > makes Dialyzer cache the compiled modules into ~/.dialyzer_hipe_cache > the first time it's run. > > There's probably a nicer solution to this, like building the modules in > question at install time, but this hack seems to work for me... Feedback > is very welcome. Have you tried building with --enable-native-libs? From jesper.louis.andersen@REDACTED Wed Jun 13 11:12:26 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Jun 2012 11:12:26 +0200 Subject: [erlang-questions] Strange observation running in an SMP environment. In-Reply-To: References: <4FD79A51.8080805@erlang-solutions.com> Message-ID: <4FD8597A.3030202@erlang-solutions.com> On Tue Jun 12 22:39:02 2012, Ronny Meeus wrote: > Does the issue that I had (pure message passing without extra load) > have anything to do with the fact that I'm running on Linux? > I sometimes have the impression (not in the context of my Erlang > playing) that the scheduler does not always makes the right decision > during the migration of processes between cores? No, it has not. I ran your examples on an Apple OSX 10.7.4, tiefling:~ jlouis$ uname -a Darwin tiefling.issuu.com 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 and I got roughly the same results as yours. The problem is that really good parallelism speedup requires you to do very specific things upon your data in order to achieve that parallel speedup. Erlang is built for concurrency more than parallelism - if you want that I'd suggest looking into Haskell :) -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK From magnus.henoch@REDACTED Wed Jun 13 11:21:56 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Wed, 13 Jun 2012 10:21:56 +0100 (BST) Subject: [erlang-questions] Dialyzer to cache Hipe compilation In-Reply-To: Message-ID: <3c2f06af-e1a0-40b2-a672-a7c70bf27a03@knuth> > Have you tried building with --enable-native-libs? The last time I tried it, it didn't actually speed up Dialyzer startup - but that was a long time ago, and I hear things have been fixed since. But in my case, I'd prefer to run a non-hipe erlang most of the time to match what's eventually going into production; I just need that little boost for dialyzer. Regards, -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From hukl@REDACTED Wed Jun 13 11:29:45 2012 From: hukl@REDACTED (John-Paul Bader) Date: Wed, 13 Jun 2012 11:29:45 +0200 Subject: [erlang-questions] [ANN] etest and etest_http 1.0.0.beta Message-ID: <4FD85D89.8040809@berlin.ccc.de> Hey, at Wooga we are writing erlang backends with http api's and in the past we've taken several approaches to test those applications properly. There was some combination of using eunit for unit and ruby for integration testing for example or a mixture of eunit, ct and ruby. In one of our teams we have now written a super lightweight test framework that makes it easier to do test-driven development with erlang for those kind of applications. The base framework is called etest. It includes basic assertion macros and a test-runner. In addition to that we wrote a supplementary library that makes testing http apis much easier. Its called etest_http and as the name suggests, has etest as a dependency. https://github.com/wooga/etest https://github.com/wooga/etest_http In my team we are using it instead of eunit/ct/ruby now and by announcing it here I hope some of you will also take a look to provide some feedback. It is also fairly simple to convert an eunit test case to etest and vice versa - after all its not that different to eunit. Somehow the tests just look cleaner, the runner is about 100 lines of code and its more fun (at least for my team) to write tests with it. I also made a small screencast that shows how to set up etest. https://vimeo.com/43672318 A video demonstrating etest_http will follow shortly. So please take a look and let me know what you think! ~ John From ronny.meeus@REDACTED Wed Jun 13 11:32:36 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Wed, 13 Jun 2012 11:32:36 +0200 Subject: [erlang-questions] Strange observation running in an SMP environment. In-Reply-To: <4FD8597A.3030202@erlang-solutions.com> References: <4FD79A51.8080805@erlang-solutions.com> <4FD8597A.3030202@erlang-solutions.com> Message-ID: On Wed, Jun 13, 2012 at 11:12 AM, Jesper Louis Andersen wrote: > On Tue Jun 12 22:39:02 2012, Ronny Meeus wrote: > >> Does the issue that I had (pure message passing without extra load) >> have anything to do with the fact that I'm running on Linux? >> I sometimes have the impression (not in the context of my Erlang >> playing) that the scheduler does not always makes the right decision >> during the migration of processes between cores? > > > No, it has not. I ran your examples on an Apple OSX 10.7.4, > > tiefling:~ jlouis$ uname -a > Darwin tiefling.issuu.com 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr ?9 > 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 > > and I got roughly the same results as yours. The problem is that really good > parallelism speedup requires you to do very specific things upon your data > in order to achieve that parallel speedup. Erlang is built for concurrency > more than parallelism - if you want that I'd suggest looking into Haskell :) > > > -- > Jesper Louis Andersen > ?Erlang Solutions Ltd., Copenhagen, DK Thanks for your help. Best regards, Ronny From dangodiac@REDACTED Wed Jun 13 11:51:18 2012 From: dangodiac@REDACTED (benefit) Date: Wed, 13 Jun 2012 02:51:18 -0700 (PDT) Subject: [erlang-questions] Erlang gen server and c++ client Message-ID: <1339581078013-4655136.post@n4.nabble.com> Hello. Please help me. I need to make a erlang gen_server which works with an erlang port. An c++ application must connect to erlang port and send a string. Server should return the same string to the application. I have troubles working with erlang. I found an example code : connect(Message) -> Cmd = "./myqtwindowapp \n", Port = open_port({spawn,Cmd}, [stream,use_stdio,exit_status]), Payload = string:concat(Message, "\n"), erlang:port_command(Port, Payload), receive {Port, {data, Data}} -> ?DBG("Received data: ~p~n", [Data]), Other -> io:format("Unexpected data: ~p~n", [Other]) after 15000 -> ?DBG("Received nothing~n", []) end. But i dont know how to run this code. I tried erl -run serv start but recieved error in do_boot. And how to include this code into a small module? Even i recieved error in dbg macros. Please help me. Suggest me how to finalize. I am newbie here, please suggest me how to run this code or other. Any help is appreciated. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-gen-server-and-c-client-tp4655136.html Sent from the Erlang Questions mailing list archive at Nabble.com. From rapsey@REDACTED Wed Jun 13 14:08:12 2012 From: rapsey@REDACTED (Rapsey) Date: Wed, 13 Jun 2012 14:08:12 +0200 Subject: [erlang-questions] Erlang gen server and c++ client In-Reply-To: <1339581078013-4655136.post@n4.nabble.com> References: <1339581078013-4655136.post@n4.nabble.com> Message-ID: Your description is very confusing. Erlang runs a c++ app and communicates with it over a port? Your problem is how to actually launch Erlang that executes your code? erl -eval "module:function()" Sergej On Wed, Jun 13, 2012 at 11:51 AM, benefit wrote: > Hello. > Please help me. > I need to make a erlang gen_server which works with an erlang port. > An c++ application must connect to erlang port and send a string. Server > should return the same string to the application. > > I have troubles working with erlang. > I found an example code : > > connect(Message) -> > Cmd = "./myqtwindowapp \n", > Port = open_port({spawn,Cmd}, [stream,use_stdio,exit_status]), > Payload = string:concat(Message, "\n"), > erlang:port_command(Port, Payload), > receive > {Port, {data, Data}} -> > ?DBG("Received data: ~p~n", [Data]), > Other -> > io:format("Unexpected data: ~p~n", [Other]) > after 15000 -> > ?DBG("Received nothing~n", []) > end. > > > But i dont know how to run this code. I tried erl -run serv start but > recieved error in do_boot. And how to include this code into a small > module? > Even i recieved error in dbg macros. > > Please help me. Suggest me how to finalize. I am newbie here, please > suggest > me how to run this code or other. > Any help is appreciated. > > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/Erlang-gen-server-and-c-client-tp4655136.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Jun 13 14:15:42 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 13 Jun 2012 14:15:42 +0200 Subject: [erlang-questions] [ANN] etest and etest_http 1.0.0.beta In-Reply-To: <4FD85D89.8040809@berlin.ccc.de> References: <4FD85D89.8040809@berlin.ccc.de> Message-ID: Hi John-Paul & Wooga Team, Thanks for sharing. I played a bit with etest_http and found it very useful. How one ?an use these macros under CommonTest? Are you planning to add any stateful feature to it (eg. to be able to play or replat full HTTP sessions) ? Anyway, well done guys. Regards, Zabrane On Jun 13, 2012, at 11:29 AM, John-Paul Bader wrote: > Hey, > > > at Wooga we are writing erlang backends with http api's and in the past we've taken several approaches to test those applications properly. There was some combination of using eunit for unit and ruby for integration testing for example or a mixture of eunit, ct and ruby. > > In one of our teams we have now written a super lightweight test framework that makes it easier to do test-driven development with erlang for those kind of applications. > > The base framework is called etest. It includes basic assertion macros and a test-runner. In addition to that we wrote a supplementary library that makes testing http apis much easier. Its called etest_http and as the name suggests, has etest as a dependency. > > https://github.com/wooga/etest > https://github.com/wooga/etest_http > > In my team we are using it instead of eunit/ct/ruby now and by announcing it here I hope some of you will also take a look to provide some feedback. It is also fairly simple to convert an eunit test case to etest and vice versa - after all its not that different to eunit. > Somehow the tests just look cleaner, the runner is about 100 lines of code and its more fun (at least for my team) to write tests with it. > > I also made a small screencast that shows how to set up etest. > > https://vimeo.com/43672318 > > A video demonstrating etest_http will follow shortly. > > So please take a look and let me know what you think! > > ~ John > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Wed Jun 13 14:16:38 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 13 Jun 2012 14:16:38 +0200 Subject: [erlang-questions] Erlang gen server and c++ client In-Reply-To: <1339581078013-4655136.post@n4.nabble.com> References: <1339581078013-4655136.post@n4.nabble.com> Message-ID: <1339589798.4883.1.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, To run Erlang code you need to put it into a module (file) and compile it. See http://learnyousomeerlang.com or another similar website. bengt On Wed, 2012-06-13 at 11:51 +0200, benefit wrote: > Hello. > Please help me. > I need to make a erlang gen_server which works with an erlang port. > An c++ application must connect to erlang port and send a string. Server > should return the same string to the application. > > I have troubles working with erlang. > I found an example code : > > connect(Message) -> > Cmd = "./myqtwindowapp \n", > Port = open_port({spawn,Cmd}, [stream,use_stdio,exit_status]), > Payload = string:concat(Message, "\n"), > erlang:port_command(Port, Payload), > receive > {Port, {data, Data}} -> > ?DBG("Received data: ~p~n", [Data]), > Other -> > io:format("Unexpected data: ~p~n", [Other]) > after 15000 -> > ?DBG("Received nothing~n", []) > end. > > > But i dont know how to run this code. I tried erl -run serv start but > recieved error in do_boot. And how to include this code into a small module? > Even i recieved error in dbg macros. > > Please help me. Suggest me how to finalize. I am newbie here, please suggest > me how to run this code or other. > Any help is appreciated. > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-gen-server-and-c-client-tp4655136.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From serge@REDACTED Wed Jun 13 14:31:58 2012 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 13 Jun 2012 08:31:58 -0400 Subject: [erlang-questions] Erlang gen server and c++ client In-Reply-To: <1339581078013-4655136.post@n4.nabble.com> References: <1339581078013-4655136.post@n4.nabble.com> Message-ID: <4FD8883E.3030906@aleynikov.org> See this tutorial for help: http://trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs On 6/13/2012 5:51 AM, benefit wrote: > Hello. > Please help me. > I need to make a erlang gen_server which works with an erlang port. > An c++ application must connect to erlang port and send a string. Server > should return the same string to the application. > > I have troubles working with erlang. > I found an example code : > > connect(Message) -> > Cmd = "./myqtwindowapp \n", > Port = open_port({spawn,Cmd}, [stream,use_stdio,exit_status]), > Payload = string:concat(Message, "\n"), > erlang:port_command(Port, Payload), > receive > {Port, {data, Data}} -> > ?DBG("Received data: ~p~n", [Data]), > Other -> > io:format("Unexpected data: ~p~n", [Other]) > after 15000 -> > ?DBG("Received nothing~n", []) > end. > > > But i dont know how to run this code. I tried erl -run serv start but > recieved error in do_boot. And how to include this code into a small module? > Even i recieved error in dbg macros. > > Please help me. Suggest me how to finalize. I am newbie here, please suggest > me how to run this code or other. > Any help is appreciated. > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-gen-server-and-c-client-tp4655136.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From techabc@REDACTED Wed Jun 13 17:09:48 2012 From: techabc@REDACTED (techabc) Date: Wed, 13 Jun 2012 23:09:48 +0800 Subject: [erlang-questions] Erlang gen server and c++ client In-Reply-To: <4FD8883E.3030906@aleynikov.org> References: <1339581078013-4655136.post@n4.nabble.com> <4FD8883E.3030906@aleynikov.org> Message-ID: hope tinch++ would help you: tinch++: Interfacing Erlang from C++ http://www.adampetersen.se/code/tinchpp.htm Introduction tinch++ is a platform independent, open-source library for building distributed Erlang nodes in C++. Distributed Erlang nodes provide a high-level model for integrating other languages with Erlang programs. With tinch++, your C++ code will be able to communicate with Erlang processes by means of message passing. To the Erlang processes, your C++ node will look and behave like any Erlang node. Besides integrating C++ and Erlang programs, tinch++ may be used as a thread-safe bidirectional queue between multiple threads in an application. For an example, check out the example program thread_safe_queue.cpp included in the tinch++ release. 2012/6/13 Serge Aleynikov : > See this tutorial for help: > > http://trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs > > On 6/13/2012 5:51 AM, benefit wrote: >> Hello. >> Please help me. >> I need to make a erlang gen_server which works with an erlang port. >> An c++ application must connect to erlang port and send a string. Server >> should return the same string to the application. >> >> I have troubles working with erlang. >> I found an example code : >> >> connect(Message) -> >> ? ? Cmd = "./myqtwindowapp \n", >> ? ? Port = open_port({spawn,Cmd}, [stream,use_stdio,exit_status]), >> ? ? Payload = string:concat(Message, "\n"), >> ? ? erlang:port_command(Port, Payload), >> ? ? receive >> ? ? ? ? {Port, {data, Data}} -> >> ? ? ? ? ? ? ?DBG("Received data: ~p~n", [Data]), >> ? ? ? ? Other -> >> ? ? ? ? ? ? io:format("Unexpected data: ~p~n", [Other]) >> ? ? after 15000 -> >> ? ? ? ? ? ? ?DBG("Received nothing~n", []) >> ? ? end. >> >> >> But i dont know how to run this code. I tried erl -run serv start but >> recieved error in do_boot. And how to include this code into a small module? >> Even i recieved error in dbg macros. >> >> Please help me. Suggest me how to finalize. I am newbie here, please suggest >> me how to run this code or other. >> Any help is appreciated. >> >> -- >> View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-gen-server-and-c-client-tp4655136.html >> Sent from the Erlang Questions mailing list archive at Nabble.com. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sunwood360@REDACTED Wed Jun 13 18:49:07 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Wed, 13 Jun 2012 09:49:07 -0700 Subject: [erlang-questions] performance of websocket Message-ID: from: https://github.com/ericmoritz/wsdemo/blob/master/results.md It seems Erlang's connection time is a little bit high. SummaryImplementationConnection Time (mean)Latency (mean)MessagesTimeouts Erlang00.865ms00.017ms28492940Java (Webbit)00.567ms00.835ms1028390157Go 00.284ms18.503ms2398180225Node.js00.768ms42.580ms11708474299Python (ws4py) 01.561ms34.889ms10529965208 Both the Python gevent/ws4py implementation and the Node.js websocket implementation failed hard with around half of the connections hitting the 2 second TCP connection timeout. I expected Go to kick Erlang's ass in the performance department but the message latency was much higher than Erlang's latency and we had 225 unhappy customers. Go only reached C9.775k; close but no cigar. I did not know what to expect with the Java Webbit implementation. I expected it to perform somewhere close to the Go implementation. The Webbit implementation did do much better than the Go implementation but it still had 157 connection timeouts which is unacceptable. I was very surprised that node.js fell down after 5001 connections and that gevent fell down at 4792 connections. Both platforms were specifically built to for the C10k problem and both platforms could barely handle C5k. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Wed Jun 13 19:53:47 2012 From: eric@REDACTED (Eric Moritz) Date: Wed, 13 Jun 2012 13:53:47 -0400 Subject: [erlang-questions] performance of websocket In-Reply-To: References: Message-ID: I made a dumb mistake at 3am on this table. Those ms times are actually seconds. I've updated the table at this link. Disregard the table in this email. https://github.com/ericmoritz/wsdemo/blob/master/results.md If you look at the raw data you will also notice that the handshake times for 2,500 of the 10,000 connections are greater than 742ms which is pretty lousy as well. While none of the TCP connections timed out, 25% of the connections had to wait at least 743ms to start sending messages. Once the handshake eventually finished, the per-message latency was really good compared to the others with a mean of 17ms. Eric. On Wed, Jun 13, 2012 at 12:49 PM, envelopes envelopes wrote: > from: https://github.com/ericmoritz/wsdemo/blob/master/results.md > > It seems Erlang's connection time is a little bit high. > Summary ImplementationConnection Time (mean) Latency (mean)Messages > Timeouts Erlang00.865ms00.017ms 28492940 Java (Webbit)00.567ms 00.835ms > 1028390157 Go00.284ms 18.503ms2398180225 Node.js00.768ms 42.580ms1170847 > 4299 Python (ws4py)01.561ms 34.889ms10529965208 > > Both the Python gevent/ws4py implementation and the Node.js websocket > implementation failed hard with around half of the connections hitting the > 2 second TCP connection timeout. > > I expected Go to kick Erlang's ass in the performance department but the > message latency was much higher than Erlang's latency and we had 225 > unhappy customers. Go only reached C9.775k; close but no cigar. > > I did not know what to expect with the Java Webbit implementation. I > expected it to perform somewhere close to the Go implementation. The Webbit > implementation did do much better than the Go implementation but it still > had 157 connection timeouts which is unacceptable. > > I was very surprised that node.js fell down after 5001 connections and > that gevent fell down at 4792 connections. Both platforms were specifically > built to for the C10k problem and both platforms could barely handle C5k. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Jun 13 20:54:05 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 13 Jun 2012 22:54:05 +0400 Subject: [erlang-questions] performance of websocket In-Reply-To: References: Message-ID: My main question is: why Java, Node and python are dropping connections? From eric@REDACTED Wed Jun 13 21:45:08 2012 From: eric@REDACTED (Eric Moritz) Date: Wed, 13 Jun 2012 15:45:08 -0400 Subject: [erlang-questions] performance of websocket In-Reply-To: References: Message-ID: I was wondering the same thing. I haven't had time to investigate the cause. I stood up a naked m1.medium AWS instance with ubuntu-12.04 and ran this script: https://raw.github.com/ericmoritz/wsdemo/master/configure_ubuntu.sh The contents of the sysctl.conf file that I stole from Richard Jones Is here: https://github.com/ericmoritz/wsdemo/blob/master/etc/sysctl.conf I ran the servers and the ./runtest script as root on seperate servers instances with a ulimit of 999999. I tried my best to document everything that was done to the servers in order to trace any faults I had in the process. On Wed, Jun 13, 2012 at 2:54 PM, Max Lapshin wrote: > My main question is: ?why Java, Node and python are dropping connections? From vladdu55@REDACTED Wed Jun 13 23:10:04 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 13 Jun 2012 23:10:04 +0200 Subject: [erlang-questions] starting with long name fails Message-ID: Hi! I am a little confused, hopefully someone can get me on the right track. I noticed that on some machines, when starting "erl -name foo", I get an error "Can't set long name!". I suppose this is because the machine doesn't have a properly configured FQDN? Also, I don't know where I had gotten this idea from, but I was using "erl -name foo@REDACTED". This works fine with any weird value for the host name, but of course there will be problem if the real host name isn't "mypc.com". Given that sometimes (as above) the runtime checks for a valid host name, why does it allow anything if specified on the comman line? (Needless to saay, I got in some delicate situation because of this :-) best regards, Vlad From dan@REDACTED Thu Jun 14 00:09:36 2012 From: dan@REDACTED (Daniel Dormont) Date: Wed, 13 Jun 2012 18:09:36 -0400 Subject: [erlang-questions] starting with long name fails In-Reply-To: References: Message-ID: forgot to cc the list. On Wed, Jun 13, 2012 at 6:09 PM, Daniel Dormont wrote: > I had a similar question a few months ago. Here was the response. > > http://erlang.org/pipermail/erlang-questions/2011-December/063066.html > > Short answer, 'erl -name foo' without an @ sign (or anything after it) is > indeed the preferred way to start a long-named node, but it only works if > the machine has a FQDN (note that this can be faked using a host file if > your DNS isn't set up to give you one). > > -Dan > > > On Wed, Jun 13, 2012 at 5:10 PM, Vlad Dumitrescu wrote: > >> Hi! >> >> I am a little confused, hopefully someone can get me on the right track. >> >> I noticed that on some machines, when starting "erl -name foo", I get >> an error "Can't set long name!". I suppose this is because the machine >> doesn't have a properly configured FQDN? >> >> Also, I don't know where I had gotten this idea from, but I was using >> "erl -name foo@REDACTED". This works fine with any weird value for the >> host name, but of course there will be problem if the real host name >> isn't "mypc.com". Given that sometimes (as above) the runtime checks >> for a valid host name, why does it allow anything if specified on the >> comman line? (Needless to saay, I got in some delicate situation >> because of this :-) >> >> best regards, >> Vlad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Jun 14 08:26:10 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 14 Jun 2012 08:26:10 +0200 Subject: [erlang-questions] starting with long name fails In-Reply-To: References: Message-ID: Thanks Daniel, I had missed that thread. As it happens, I have some machines that aren't configured properly and I didn't see that as long as I was only connecting several nodes on the same machine. When I got really distributed, then the problem became apparent. I hope I can get the admin to fix the configuration... regards, Vlad > On Wed, Jun 13, 2012 at 6:09 PM, Daniel Dormont > wrote: >> >> I had a similar question a few months ago. Here was the response. >> >> http://erlang.org/pipermail/erlang-questions/2011-December/063066.html >> >> Short answer, 'erl -name foo' without an @ sign (or anything after it) is >> indeed the preferred way to start a long-named node, but it only works if >> the machine has a FQDN (note that this can be faked using a host file if >> your DNS isn't set up to give you one). >> >> -Dan >> >> >> On Wed, Jun 13, 2012 at 5:10 PM, Vlad Dumitrescu >> wrote: >>> >>> Hi! >>> >>> I am a little confused, hopefully someone can get me on the right track. >>> >>> I noticed that on some machines, when starting "erl -name foo", I get >>> an error "Can't set long name!". I suppose this is because the machine >>> doesn't have a properly configured FQDN? >>> >>> Also, I don't know where I had gotten this idea from, but I was using >>> "erl -name foo@REDACTED". This works fine with any weird value for the >>> host name, but of course there will be problem if the real host name >>> isn't "mypc.com". Given that sometimes (as above) the runtime checks >>> for a valid host name, why does it allow anything if specified on the >>> comman line? (Needless to saay, I got in some delicate situation >>> because of this :-) >>> >>> best regards, >>> Vlad >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andre@REDACTED Thu Jun 14 09:01:43 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Thu, 14 Jun 2012 09:01:43 +0200 Subject: [erlang-questions] performance of websocket In-Reply-To: References: Message-ID: Thank you very much for sharing this! Cheers Andr? On 13 June 2012 18:49, envelopes envelopes wrote: > from: https://github.com/ericmoritz/wsdemo/blob/master/results.md > > It seems Erlang's connection time is a little bit high. > Summary ImplementationConnection Time (mean) Latency (mean)Messages > Timeouts Erlang00.865ms00.017ms 28492940 Java (Webbit)00.567ms 00.835ms > 1028390157 Go00.284ms 18.503ms2398180225 Node.js00.768ms 42.580ms1170847 > 4299 Python (ws4py)01.561ms 34.889ms10529965208 > > Both the Python gevent/ws4py implementation and the Node.js websocket > implementation failed hard with around half of the connections hitting the > 2 second TCP connection timeout. > > I expected Go to kick Erlang's ass in the performance department but the > message latency was much higher than Erlang's latency and we had 225 > unhappy customers. Go only reached C9.775k; close but no cigar. > > I did not know what to expect with the Java Webbit implementation. I > expected it to perform somewhere close to the Go implementation. The Webbit > implementation did do much better than the Go implementation but it still > had 157 connection timeouts which is unacceptable. > > I was very surprised that node.js fell down after 5001 connections and > that gevent fell down at 4792 connections. Both platforms were specifically > built to for the C10k problem and both platforms could barely handle C5k. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Jun 14 10:35:04 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 14 Jun 2012 10:35:04 +0200 Subject: [erlang-questions] Java error making docs in R15B01 In-Reply-To: References: <41CC9FD9-91DC-4DAF-9F3D-C360DCDDDFED@cs.otago.ac.nz> Message-ID: On Fri, Jun 8, 2012 at 1:54 AM, Richard O'Keefe wrote: > > On 7/06/2012, at 11:59 PM, Michael Turner wrote: > >> "By the way, are the DTDs used in building Erlang/OTP documentation >> *documented* anywhere?" >> >> Didn't Ulf just point them out on another thread? > > E-mail is a *distributed* system. ?Ulf's message saying where to > find the documentation arrived in my mailbox the day after I sent > my question. > > The right thing, of course, is for each of the DTDs to contain > a comment saying where to find the documentation, or better still, > for the documentation to be automatically extracted from comments > in the DTDs. > > There are some issues that are not documented. > > When I've set up DTDs and had a use for something that was also > found in HTML, I have generally use the HTML names for things. > Thus when I have a table, I use
, , and so on. When I'm told to use instead of
, , , I want to > know why. ?Perhaps some other template is being copied? > DocBook maybe? ?It would be useful to know, because I might find > useful background in the thing that was copied. As far as I am aware the design is ad-hock and not based on anything else. I am currently revising the DTDs. My goal is one DTD, as near as possible to a subset of XHTML with additional tags for Erlang specific things. I will also remove rare tags that are hardly every used. > > Similarly, the HTML 3.2 character entities are copied, well and > good, but why not the HTML 4 ones? ?I'm thinking of section 24.3 > "Character entity references for symbols, mathematical symbols, > and Greek letters" with pubid "-//W3C//ENTITIES Symbols//EN//HTML". > If we're using xsltproc and FOP, which use Unicode, what's wrong > with these Unicode characters? ?Can I use numeric references to > get them? ?If there's some basic limit here, if, for example, it > is still thought important to generate pure HTML 3.2, I'd like > to know. ?Or is it a limitation of UNIX man pages? ?Need that > limit apply to documents that won't be processed by troff? > > I'll investigate this... /Joe From a.shneyderman@REDACTED Thu Jun 14 16:17:17 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Thu, 14 Jun 2012 16:17:17 +0200 Subject: [erlang-questions] Out of memory troubles. In-Reply-To: References: Message-ID: On Mon, Jun 11, 2012 at 4:07 PM, Gordon Guthrie wrote: > To get my open-file limit up on Linux I had to bump that up as well.. > > Mebbies there is something like that on OSX as well? Thanks. Sure and as Bob Ippolito mentioned the limits are way too low on Macs. Anyhow, the problem at the end was that I recompiled erlang with new limits and kernel poll enabled but when starting VM did not have +K true. Adding that to the startup made all problems disappear and my RAM is well under control, where I'd expect it to be. Cheers, Alex. From eduard.sergeev@REDACTED Fri Jun 15 04:55:55 2012 From: eduard.sergeev@REDACTED (Eduard Sergeev) Date: Fri, 15 Jun 2012 12:55:55 +1000 Subject: [erlang-questions] Reltool error: "Module potentially included by two different applications" In-Reply-To: References: Message-ID: Hi, I was wondering what was the reason behind the following behaviour of reltool: If my reltool.config uses default?'mod_cond' and 'incl_cond' options and if one of my included applications has a module which also happens to be a part of some application installed on my machine but NOT included in my release `relltool:get_target_spec/1` returns: {error,?"Module potentially included by two different applications: and ."} Which is annoying since is NOT a part of my release (neither directly nor indirectly). Can't reltool actually figure out that will not be included in my release? Id that why it is "potentially included"? Anyway, in order to generate my release I have to either explicitly exclude via `{app, [{incl_cond, exclude}]}` which is ugly since this is just happens to be installed in 'lib' directory of the machine where I do the build (it may not be installed on other build machines) and has nothing to do with my release. The actual example: 'tsung-1.4.3' includes 'mochijson2' module so I have a problem building my release which should include 'mochiweb' app on the machine that has 'tsung' installed (but not on other machines). Another option would be to change top-level 'incl_cond' from {incl_cond, derived} to?{incl_cond, exclude} and then manually include all the application which I want to be a part my release which is better (will work on any build machine) but still not great since it has to be done manually (I want to rely on relltool to figure out the dependencies). So the question is why do we have such a situation? Why just a mere presence of some application on the build machine lead to the above reltool error? Regards, Eduard From shahrdad1@REDACTED Fri Jun 15 15:52:53 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Fri, 15 Jun 2012 09:52:53 -0400 Subject: [erlang-questions] DXNN: Topology and Parameter Evolving Universal Learning Network system/platform, released to GitHub. In-Reply-To: References: Message-ID: Grate work! I cannot wait to read your book. I am also working on using Erlang in statistical machine learning, this requires of mathematical/statistical library functions (like linear algebra / statistical libraries and so on) to be implemented in Erlang which is taking a lot of time from me. I wonder if you ever looked at statistical approach to AI and why you didn't follow that path as opposed to neuron-genetic approach. Thanks a lot Best regards Shahrdad On Tue, Jun 12, 2012 at 1:31 PM, G.S. wrote: > Hello all, > > DXNN [1,4] is Topology and Parameter Evolving Universal Learning Network > (TPEULN) system, similar to topology and weight evolving artificial neural > network, but more general, and not constrained to the use of only sigmoid > based activation function neurons. Erlang was chosen because of its perfect > and complete mapping to the neural network architecture. > > DXNN is a TPEULN platform that uses direct and indirect encoding (neural > and substrate respectively [5]), has a cross-validation system for > experimentation, decoupled sensor/actuator systems, decoupled > learning/selection/... algorithms (in MK2), a built in 2d world simulator > called flatland for ALife experiments (all in gs()). > > The second generation (mk2) DXNN is available as a branch of the original > project, and is a clean implementation of this computational intelligence > evolving system. It is also the system explained and created in my Springer > book: Handbook of Neuroevolution Through Erlang [2,3], with a foreword > written by Joe Armstrong. The book will go into print this September. > > There are not a lot of comments within the source code on github, but I > will continue to add more comments as time permits. > > Upcoming features: > 1. Visualisation system. > 2. New selection algorithm modules. > 3. New speciation and diversification functions. > 4. An improved cross-validation system for the experiment database. > 5. Full population backup, so that all agents are saved, and only manually > deleted at the researcher's request (they don't take much space, and it > would make for an interesting visualisation, and ability to traverse from > the seed agent to the current agent). > > -Gene > [1] https://github.com/CorticalComputer/DXNN First generation DXNN has a > convoluted implementation. DXNN mk2 is a very clean implementation and is > currently on the non master branch, it will eventually overwrite the master > branch but both have the same features (almost) at this time. > [2] http://www.springer.com/computer/swe/book/978-1-4614-4462-6 > [3] > http://www.amazon.com/Handbook-Neuroevolution-through-Erlang-Gene/dp/1461444624/ref=sr_1_1?ie=UTF8&qid=1338163875&sr=8-1 > [4] http://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher > [5] http://arxiv.org/abs/1111.5892 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From lambdadmitry@REDACTED Fri Jun 15 16:00:11 2012 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Fri, 15 Jun 2012 18:00:11 +0400 Subject: [erlang-questions] Reference binary comparison problem Message-ID: There is a little problem with reference binaries that are used with NIFs: they all are treated as equal. It's quite uncomfortable when one works with NIFs. I can try to make a patch to fix this (I'm not sure about it's quality though) or I can just wait for OTP team (it looks like quite small fix) What's better? From sverker.eriksson@REDACTED Fri Jun 15 16:36:21 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 15 Jun 2012 16:36:21 +0200 Subject: [erlang-questions] Reference binary comparison problem In-Reply-To: References: Message-ID: <4FDB4865.9060708@erix.ericsson.se> Here's an earlier discussion on this topic: http://erlang.2086793.n4.nabble.com/Allow-limited-pattern-matching-on-magic-binaries-td3382967.html It turns out that it's not that trivial as one might first think. /Sverker, Erlang/OTP Ericsson Dmitry Groshev wrote: > There is a little problem with reference binaries that are used with > NIFs: they all are treated as equal. It's quite uncomfortable when one > works with NIFs. I can try to make a patch to fix this (I'm not sure > about it's quality though) or I can just wait for OTP team (it looks > like quite small fix) What's better? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From michael.santos@REDACTED Fri Jun 15 17:30:05 2012 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 15 Jun 2012 11:30:05 -0400 Subject: [erlang-questions] Reference binary comparison problem In-Reply-To: <4FDB4865.9060708@erix.ericsson.se> References: <4FDB4865.9060708@erix.ericsson.se> Message-ID: <20120615153005.GB23533@ioctl> On Fri, Jun 15, 2012 at 04:36:21PM +0200, Sverker Eriksson wrote: > Here's an earlier discussion on this topic: > > http://erlang.2086793.n4.nabble.com/Allow-limited-pattern-matching-on-magic-binaries-td3382967.html > > It turns out that it's not that trivial as one might first think. > > > /Sverker, Erlang/OTP Ericsson > > > Dmitry Groshev wrote: > >There is a little problem with reference binaries that are used with > >NIFs: they all are treated as equal. It's quite uncomfortable when one > >works with NIFs. I can try to make a patch to fix this (I'm not sure > >about it's quality though) or I can just wait for OTP team (it looks > >like quite small fix) What's better? In practice, it's simple to work around this in your NIF. Return a tagged tuple with a ref, something like: return enif_make_tuple2(env, atom_ok, enif_make_tuple4(env, atom_resource, type, enif_make_ref(env), res)); Which ends up being {ok, {resource, foo, #Ref<0.0.0.32>, <<>>}} and can be defined as a record: #resource{type = foo, ref = Ref, res = Res} From ulf@REDACTED Fri Jun 15 20:33:25 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2012 11:33:25 -0700 Subject: [erlang-questions] conditional pub-sub in gproc Message-ID: I threw in a function for doing pub/sub with simple filter conditions in gproc. https://github.com/esl/gproc/blob/master/doc/gproc_ps.md#subscribe_cond-3 A simple example: Eshell V5.9 (abort with ^G) 1> application:start(gproc). ok %% Subscribe to level changes for a gauge; only report if Level > 3 2> gproc_ps:subscribe_cond(l,gauge,ets:fun2ms(fun({level,N}) when N > 3 -> true end)). true 3> gproc_ps:publish_cond(l,gauge,{level,1}). ok 4> flush(). ok % no message delivered 5> gproc_ps:publish_cond(l,gauge,{level,4}). ok 6> flush(). Shell got {gproc_ps_event,gauge,{level,4}} ok Essentially, gproc attaches a match_spec to the subscription entry, and this match_spec is evaluated when each event is published. If ets:match_spec_run([Msg], ets:match_spec_compile(Condition)) == [true], the message is delivered; otherwise not. The match specs are stored un-compiled, partly because you can't pass compiled match specs between nodes (gproc's global scope), and partly because you can't read them when compiled, and it's useful to be able to inspect properties: 7> gproc:info(self()). [{gproc,[{{p,l,{gproc_ps_event,gauge}}, [{{level,'$1'},[{'>','$1',3}],[true]}]}]}, {current_function,{erl_eval,do_apply,6}}, {initial_call,{erlang,apply,2}}, {status,running}, ? Feedback is welcome. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From max.lapshin@REDACTED Fri Jun 15 21:09:58 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 15 Jun 2012 23:09:58 +0400 Subject: [erlang-questions] Reference binary comparison problem In-Reply-To: <20120615153005.GB23533@ioctl> References: <4FDB4865.9060708@erix.ericsson.se> <20120615153005.GB23533@ioctl> Message-ID: Maybe also use enif_make_resource_binary and put some unique data inside this binary? From msegalis@REDACTED Fri Jun 15 22:40:30 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 15 Jun 2012 22:40:30 +0200 Subject: [erlang-questions] What are the consequences of not closing the SSL connection ? Message-ID: A simple question really, I have a gen_server that is handling my SSL connection? Since now I just found out, that I did not closed the ssl connection handled by the gen_server when this one terminates? Of course I have added the close, now? But it got me curious, what would be the consequence if not explicitly closing the connection when the gen_server terminates ? Does it still lives, or does it get closed by itself? ? Thanks, Morgan From watson.timothy@REDACTED Sat Jun 16 00:17:56 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 15 Jun 2012 23:17:56 +0100 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: Message-ID: <4FDBB494.3000106@gmail.com> This is pretty cool - I also noticed await/1 for the first time, which looks pretty useful for synchronizing distributed startup. Looking at the code, gproc_dist seems to be using gen_leader. Is this something you'd be comfortable using in production (I assume yes) - I only ask because I've seen numerous threads talking about various incarnations of it, and I wondered whether it's considered stable now? I'm also very interested to know what kind of consistency guarantees I have when using this in a distributed scenario. Cheers, Tim On 15/06/2012 19:33, Ulf Wiger wrote: > I threw in a function for doing pub/sub with simple filter conditions in gproc. > > https://github.com/esl/gproc/blob/master/doc/gproc_ps.md#subscribe_cond-3 > > A simple example: > > Eshell V5.9 (abort with ^G) > 1> application:start(gproc). > ok > > %% Subscribe to level changes for a gauge; only report if Level> 3 > > 2> gproc_ps:subscribe_cond(l,gauge,ets:fun2ms(fun({level,N}) when N> 3 -> true end)). > true > > 3> gproc_ps:publish_cond(l,gauge,{level,1}). > ok > 4> flush(). > ok % no message delivered > > 5> gproc_ps:publish_cond(l,gauge,{level,4}). > ok > 6> flush(). > Shell got {gproc_ps_event,gauge,{level,4}} > ok > > Essentially, gproc attaches a match_spec to the subscription entry, and this > match_spec is evaluated when each event is published. > > If ets:match_spec_run([Msg], ets:match_spec_compile(Condition)) == [true], > the message is delivered; otherwise not. > > The match specs are stored un-compiled, partly because you can't pass > compiled match specs between nodes (gproc's global scope), and partly > because you can't read them when compiled, and it's useful to be able to > inspect properties: > > 7> gproc:info(self()). > [{gproc,[{{p,l,{gproc_ps_event,gauge}}, > [{{level,'$1'},[{'>','$1',3}],[true]}]}]}, > {current_function,{erl_eval,do_apply,6}}, > {initial_call,{erlang,apply,2}}, > {status,running}, > ? > > Feedback is welcome. > > BR, > Ulf W > > Ulf Wiger, Co-founder& Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf@REDACTED Sat Jun 16 00:31:06 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2012 15:31:06 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <4FDBB494.3000106@gmail.com> References: <4FDBB494.3000106@gmail.com> Message-ID: Personally, I don't use global gproc. The latest gen_leader versions have improved in partition tolerance, but gproc makes no attempt at deconflicting the registry after netsplits. My recommendation, if you want to use global scope in gproc, is to rig your system so that you can identify a minority island after netsplits, and restart those nodes. Otherwise, you can use gproc's local scope, perhaps also including the broadcast function. BR, Ulf W On 15 Jun 2012, at 15:17, Tim Watson wrote: > This is pretty cool - I also noticed await/1 for the first time, which looks pretty useful for synchronizing distributed startup. Looking at the code, gproc_dist seems to be using gen_leader. Is this something you'd be comfortable using in production (I assume yes) - I only ask because I've seen numerous threads talking about various incarnations of it, and I wondered whether it's considered stable now? I'm also very interested to know what kind of consistency guarantees I have when using this in a distributed scenario. > > Cheers, > Tim > > On 15/06/2012 19:33, Ulf Wiger wrote: >> I threw in a function for doing pub/sub with simple filter conditions in gproc. >> >> https://github.com/esl/gproc/blob/master/doc/gproc_ps.md#subscribe_cond-3 >> >> A simple example: >> >> Eshell V5.9 (abort with ^G) >> 1> application:start(gproc). >> ok >> >> %% Subscribe to level changes for a gauge; only report if Level> 3 >> >> 2> gproc_ps:subscribe_cond(l,gauge,ets:fun2ms(fun({level,N}) when N> 3 -> true end)). >> true >> >> 3> gproc_ps:publish_cond(l,gauge,{level,1}). >> ok >> 4> flush(). >> ok % no message delivered >> >> 5> gproc_ps:publish_cond(l,gauge,{level,4}). >> ok >> 6> flush(). >> Shell got {gproc_ps_event,gauge,{level,4}} >> ok >> >> Essentially, gproc attaches a match_spec to the subscription entry, and this >> match_spec is evaluated when each event is published. >> >> If ets:match_spec_run([Msg], ets:match_spec_compile(Condition)) == [true], >> the message is delivered; otherwise not. >> >> The match specs are stored un-compiled, partly because you can't pass >> compiled match specs between nodes (gproc's global scope), and partly >> because you can't read them when compiled, and it's useful to be able to >> inspect properties: >> >> 7> gproc:info(self()). >> [{gproc,[{{p,l,{gproc_ps_event,gauge}}, >> [{{level,'$1'},[{'>','$1',3}],[true]}]}]}, >> {current_function,{erl_eval,do_apply,6}}, >> {initial_call,{erlang,apply,2}}, >> {status,running}, >> ? >> >> Feedback is welcome. >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder& Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From watson.timothy@REDACTED Sat Jun 16 00:59:21 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 15 Jun 2012 23:59:21 +0100 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> Message-ID: <4CF6C43B-C392-4335-A148-4EC69329671E@gmail.com> Oops, looks like I forgot to cc the list - sorry about that. Thanks for the clarification Ulf. It sounds like local use is a safer bet for now. BTW I noticed some stdlib/kernel modules in the patches folder. What are these for? Cheers, Tim On 15 Jun 2012, at 23:31, Ulf Wiger wrote: > > Personally, I don't use global gproc. The latest gen_leader versions have improved in partition tolerance, but gproc makes no attempt at deconflicting the registry after netsplits. > > My recommendation, if you want to use global scope in gproc, is to rig your system so that you can identify a minority island after netsplits, and restart those nodes. > > Otherwise, you can use gproc's local scope, perhaps also including the broadcast function. > > BR, > Ulf W > > On 15 Jun 2012, at 15:17, Tim Watson wrote: > >> This is pretty cool - I also noticed await/1 for the first time, which looks pretty useful for synchronizing distributed startup. Looking at the code, gproc_dist seems to be using gen_leader. Is this something you'd be comfortable using in production (I assume yes) - I only ask because I've seen numerous threads talking about various incarnations of it, and I wondered whether it's considered stable now? I'm also very interested to know what kind of consistency guarantees I have when using this in a distributed scenario. >> >> Cheers, >> Tim >> >> On 15/06/2012 19:33, Ulf Wiger wrote: >>> I threw in a function for doing pub/sub with simple filter conditions in gproc. >>> >>> https://github.com/esl/gproc/blob/master/doc/gproc_ps.md#subscribe_cond-3 >>> >>> A simple example: >>> >>> Eshell V5.9 (abort with ^G) >>> 1> application:start(gproc). >>> ok >>> >>> %% Subscribe to level changes for a gauge; only report if Level> 3 >>> >>> 2> gproc_ps:subscribe_cond(l,gauge,ets:fun2ms(fun({level,N}) when N> 3 -> true end)). >>> true >>> >>> 3> gproc_ps:publish_cond(l,gauge,{level,1}). >>> ok >>> 4> flush(). >>> ok % no message delivered >>> >>> 5> gproc_ps:publish_cond(l,gauge,{level,4}). >>> ok >>> 6> flush(). >>> Shell got {gproc_ps_event,gauge,{level,4}} >>> ok >>> >>> Essentially, gproc attaches a match_spec to the subscription entry, and this >>> match_spec is evaluated when each event is published. >>> >>> If ets:match_spec_run([Msg], ets:match_spec_compile(Condition)) == [true], >>> the message is delivered; otherwise not. >>> >>> The match specs are stored un-compiled, partly because you can't pass >>> compiled match specs between nodes (gproc's global scope), and partly >>> because you can't read them when compiled, and it's useful to be able to >>> inspect properties: >>> >>> 7> gproc:info(self()). >>> [{gproc,[{{p,l,{gproc_ps_event,gauge}}, >>> [{{level,'$1'},[{'>','$1',3}],[true]}]}]}, >>> {current_function,{erl_eval,do_apply,6}}, >>> {initial_call,{erlang,apply,2}}, >>> {status,running}, >>> ? >>> >>> Feedback is welcome. >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder& Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From essen@REDACTED Sat Jun 16 01:10:04 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Sat, 16 Jun 2012 01:10:04 +0200 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> Message-ID: <4FDBC0CC.6080600@ninenines.eu> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? On 06/16/2012 12:31 AM, Ulf Wiger wrote: > > Personally, I don't use global gproc. The latest gen_leader versions have improved in partition tolerance, but gproc makes no attempt at deconflicting the registry after netsplits. > > My recommendation, if you want to use global scope in gproc, is to rig your system so that you can identify a minority island after netsplits, and restart those nodes. > > Otherwise, you can use gproc's local scope, perhaps also including the broadcast function. > > BR, > Ulf W > > On 15 Jun 2012, at 15:17, Tim Watson wrote: > >> This is pretty cool - I also noticed await/1 for the first time, which looks pretty useful for synchronizing distributed startup. Looking at the code, gproc_dist seems to be using gen_leader. Is this something you'd be comfortable using in production (I assume yes) - I only ask because I've seen numerous threads talking about various incarnations of it, and I wondered whether it's considered stable now? I'm also very interested to know what kind of consistency guarantees I have when using this in a distributed scenario. >> >> Cheers, >> Tim >> >> On 15/06/2012 19:33, Ulf Wiger wrote: >>> I threw in a function for doing pub/sub with simple filter conditions in gproc. >>> >>> https://github.com/esl/gproc/blob/master/doc/gproc_ps.md#subscribe_cond-3 >>> >>> A simple example: >>> >>> Eshell V5.9 (abort with ^G) >>> 1> application:start(gproc). >>> ok >>> >>> %% Subscribe to level changes for a gauge; only report if Level> 3 >>> >>> 2> gproc_ps:subscribe_cond(l,gauge,ets:fun2ms(fun({level,N}) when N> 3 -> true end)). >>> true >>> >>> 3> gproc_ps:publish_cond(l,gauge,{level,1}). >>> ok >>> 4> flush(). >>> ok % no message delivered >>> >>> 5> gproc_ps:publish_cond(l,gauge,{level,4}). >>> ok >>> 6> flush(). >>> Shell got {gproc_ps_event,gauge,{level,4}} >>> ok >>> >>> Essentially, gproc attaches a match_spec to the subscription entry, and this >>> match_spec is evaluated when each event is published. >>> >>> If ets:match_spec_run([Msg], ets:match_spec_compile(Condition)) == [true], >>> the message is delivered; otherwise not. >>> >>> The match specs are stored un-compiled, partly because you can't pass >>> compiled match specs between nodes (gproc's global scope), and partly >>> because you can't read them when compiled, and it's useful to be able to >>> inspect properties: >>> >>> 7> gproc:info(self()). >>> [{gproc,[{{p,l,{gproc_ps_event,gauge}}, >>> [{{level,'$1'},[{'>','$1',3}],[true]}]}]}, >>> {current_function,{erl_eval,do_apply,6}}, >>> {initial_call,{erlang,apply,2}}, >>> {status,running}, >>> ? >>> >>> Feedback is welcome. >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder& Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines From ulf@REDACTED Sat Jun 16 01:15:00 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2012 16:15:00 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <4CF6C43B-C392-4335-A148-4EC69329671E@gmail.com> References: <4FDBB494.3000106@gmail.com> <4CF6C43B-C392-4335-A148-4EC69329671E@gmail.com> Message-ID: <9374A62D-ABC4-4982-88A0-22C7A9AB6BE2@feuerlabs.com> On 15 Jun 2012, at 15:59, Tim Watson wrote: > Oops, looks like I forgot to cc the list - sorry about that. Thanks for the clarification Ulf. It sounds like local use is a safer bet for now. > > BTW I noticed some stdlib/kernel modules in the patches folder. What are these for? Oh, they're ancient, but they go with the Erlang Workshop paper which is also included (doc/erlang07-wiger.pdf), and are still there for illustration of some of the chapters in the paper. What I did was integrate gproc into OTP, instrumenting the OTP behaviors so that they would register properties informing about their characteristics. I also patched gen_leader so that it would handle dynamic addition and removal of nodes, but that patch is not to be taken seriously! It was just for demo purposes. A problem with that approach was that many people thought gproc required patches to both OTP and gen_leader, which was never true. :) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From ulf@REDACTED Sat Jun 16 01:18:36 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2012 16:18:36 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <4FDBC0CC.6080600@ninenines.eu> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> Message-ID: On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: > That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? Uhm, those functions work perfectly fine in a local context? Can you expand on that? Global gproc (gproc_dist) is disabled by default. My assumption is that most people leave it that way. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From essen@REDACTED Sat Jun 16 01:25:33 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Sat, 16 Jun 2012 01:25:33 +0200 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> Message-ID: <4FDBC46D.3070305@ninenines.eu> Sorry I should have been clearer. I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. On 06/16/2012 01:18 AM, Ulf Wiger wrote: > > On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: > >> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? > > Uhm, those functions work perfectly fine in a local context? > > Can you expand on that? > > Global gproc (gproc_dist) is disabled by default. > My assumption is that most people leave it that way. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From ulf@REDACTED Sat Jun 16 02:58:33 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2012 17:58:33 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <4FDBC46D.3070305@ninenines.eu> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> Message-ID: <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> Good points. Please try out the latest version, which includes a few new functions: gproc:await(Node, Key, Timeout) https://github.com/esl/gproc/blob/master/doc/gproc.md#await-3 gproc:wide_await(Nodes, Key, Timeout) https://github.com/esl/gproc/blob/master/doc/gproc.md#wide_await3 gproc:nb_wait(Node, Key) https://github.com/esl/gproc/blob/master/doc/gproc.md#nb_wait-2 gproc:cancel_wait(Node, Key, Ref) https://github.com/esl/gproc/blob/master/doc/gproc.md#cancel_wait-3 Note that await/3 also returns the Value, so can be throught of as a distributed get_value() function. BR, Ulf W On 15 Jun 2012, at 16:25, Lo?c Hoguin wrote: > Sorry I should have been clearer. > > I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. > > Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. > > It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. > > On 06/16/2012 01:18 AM, Ulf Wiger wrote: >> >> On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: >> >>> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? >> >> Uhm, those functions work perfectly fine in a local context? >> >> Can you expand on that? >> >> Global gproc (gproc_dist) is disabled by default. >> My assumption is that most people leave it that way. >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From essen@REDACTED Sat Jun 16 03:08:18 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Sat, 16 Jun 2012 03:08:18 +0200 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> Message-ID: <4FDBDC82.3060103@ninenines.eu> Awesome! This is exactly what I need. I'll try to give feedback on monday, if I can. Otherwise the week after. If no issues arise during testing this will be quickly deployed. Thanks a lot for always being so helpful! On 06/16/2012 02:58 AM, Ulf Wiger wrote: > > Good points. > > Please try out the latest version, which includes a few new functions: > > gproc:await(Node, Key, Timeout) > https://github.com/esl/gproc/blob/master/doc/gproc.md#await-3 > > gproc:wide_await(Nodes, Key, Timeout) > https://github.com/esl/gproc/blob/master/doc/gproc.md#wide_await3 > > gproc:nb_wait(Node, Key) > https://github.com/esl/gproc/blob/master/doc/gproc.md#nb_wait-2 > > gproc:cancel_wait(Node, Key, Ref) > https://github.com/esl/gproc/blob/master/doc/gproc.md#cancel_wait-3 > > > Note that await/3 also returns the Value, so can be throught of as a > distributed get_value() function. > > BR, > Ulf W > > > On 15 Jun 2012, at 16:25, Lo?c Hoguin wrote: > >> Sorry I should have been clearer. >> >> I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. >> >> Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. >> >> It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. >> >> On 06/16/2012 01:18 AM, Ulf Wiger wrote: >>> >>> On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: >>> >>>> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? >>> >>> Uhm, those functions work perfectly fine in a local context? >>> >>> Can you expand on that? >>> >>> Global gproc (gproc_dist) is disabled by default. >>> My assumption is that most people leave it that way. >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From corticalcomputer@REDACTED Sat Jun 16 07:01:00 2012 From: corticalcomputer@REDACTED (G.S.) Date: Sat, 16 Jun 2012 01:01:00 -0400 Subject: [erlang-questions] DXNN: Topology and Parameter Evolving Universal Learning Network system/platform, released to GitHub. In-Reply-To: References: Message-ID: Hello Shahrdad, Thanks. The reason for the concentration on Neuroevolution is because the goal of my research is to develop computational intelligence systems that are flexible, scalable, general, and capable of learning and self modification. Seed computational intelligence... At this time we know with absolute certainty of one approach that has worked and is capable of producing such systems; the method is Evolution, the evolution of neural networks, and the proof that it works, is us, biological thinking machines. Our brains are the result of billions of years of evolution, which has carved out in flesh our neural circuitry through trial and error. Because DXNN MK2 is fully decoupled, it allows for other search methods, selection, mutation... to be implemented and simply plugged in. As others begin using the system, they will most likely modify and try out and create new modules and functions with regards to search optimisation (artificial immune system, ant colony, swarm, CMA-CS...), selection, mutation..., and hopefully contribute those new functions and modules, thus making them selectable as options within the DXNN, and make the system even more useful within the field, and in general. Best regards, -Gene On Fri, Jun 15, 2012 at 9:52 AM, Shahrdad Shadab wrote: > Grate work! I cannot wait to read your book. > I am also working on using Erlang in statistical machine learning, this > requires of mathematical/statistical > library functions (like linear algebra / statistical libraries and so on) > to be implemented in Erlang which is taking a lot of time from me. > I wonder if you ever looked at statistical approach to AI and why you > didn't follow that path as opposed to neuron-genetic approach. > > Thanks a lot > Best regards > Shahrdad > > > > On Tue, Jun 12, 2012 at 1:31 PM, G.S. wrote: > >> Hello all, >> >> DXNN [1,4] is Topology and Parameter Evolving Universal Learning Network >> (TPEULN) system, similar to topology and weight evolving artificial neural >> network, but more general, and not constrained to the use of only sigmoid >> based activation function neurons. Erlang was chosen because of its perfect >> and complete mapping to the neural network architecture. >> >> DXNN is a TPEULN platform that uses direct and indirect encoding (neural >> and substrate respectively [5]), has a cross-validation system for >> experimentation, decoupled sensor/actuator systems, decoupled >> learning/selection/... algorithms (in MK2), a built in 2d world simulator >> called flatland for ALife experiments (all in gs()). >> >> The second generation (mk2) DXNN is available as a branch of the original >> project, and is a clean implementation of this computational intelligence >> evolving system. It is also the system explained and created in my Springer >> book: Handbook of Neuroevolution Through Erlang [2,3], with a foreword >> written by Joe Armstrong. The book will go into print this September. >> >> There are not a lot of comments within the source code on github, but I >> will continue to add more comments as time permits. >> >> Upcoming features: >> 1. Visualisation system. >> 2. New selection algorithm modules. >> 3. New speciation and diversification functions. >> 4. An improved cross-validation system for the experiment database. >> 5. Full population backup, so that all agents are saved, and only >> manually deleted at the researcher's request (they don't take much space, >> and it would make for an interesting visualisation, and ability to traverse >> from the seed agent to the current agent). >> >> -Gene >> [1] https://github.com/CorticalComputer/DXNN First generation DXNN has a >> convoluted implementation. DXNN mk2 is a very clean implementation and is >> currently on the non master branch, it will eventually overwrite the master >> branch but both have the same features (almost) at this time. >> [2] http://www.springer.com/computer/swe/book/978-1-4614-4462-6 >> [3] >> http://www.amazon.com/Handbook-Neuroevolution-through-Erlang-Gene/dp/1461444624/ref=sr_1_1?ie=UTF8&qid=1338163875&sr=8-1 >> [4] http://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher >> [5] http://arxiv.org/abs/1111.5892 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Software Architect & Computer Scientist > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sat Jun 16 17:28:34 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Sat, 16 Jun 2012 17:28:34 +0200 Subject: [erlang-questions] Let's get rid of packages Message-ID: Hello everyone, I have heard (delightful) rumors that packages are being removed. If this is true, can it be confirmed? Otherwise I would like to provide some patches because embedded mode is not working correctly with packages (but everything else seems to be working properly in my experience). * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Sat Jun 16 18:57:47 2012 From: jon@REDACTED (jon trembley) Date: Sat, 16 Jun 2012 09:57:47 -0700 Subject: [erlang-questions] DXNN: Topology and Parameter Evolving Universal Learning Network system/platform, released to GitHub. In-Reply-To: References: Message-ID: <20120616165747.GA1591@deathray.tv> This is great! I`ve been waiting for something like this in Erlang for awhile. Can`t wait to buy the book. thanks, jon On Tue, Jun 12, 2012 at 01:31:21PM -0400, G.S. wrote: > Hello all, > > DXNN [1,4]* is Topology and Parameter Evolving Universal Learning Network > (TPEULN) system, similar to topology and weight evolving artificial neural > network, but more general, and not constrained to the use of only sigmoid > based activation function neurons. Erlang was chosen because of its > perfect and complete mapping to the neural network architecture. > > DXNN is a TPEULN platform that uses direct and indirect encoding (neural > and substrate respectively [5]), has a cross-validation system for > experimentation, decoupled sensor/actuator systems, decoupled > learning/selection/... algorithms (in MK2), a built in 2d world simulator > called flatland for ALife experiments (all in gs()). > > The second generation (mk2) DXNN is available as a branch of the original > project, and is a clean implementation of this computational intelligence > evolving system. It is also the system explained and created in my > Springer book: Handbook of Neuroevolution Through Erlang [2,3], with a > foreword written by Joe Armstrong. The book will go into print this > September. > > There are not a lot of comments within the source code on github, but I > will continue to add more comments as time permits. > > Upcoming features: > 1. Visualisation system. > 2. New selection algorithm modules. > 3. New speciation and diversification functions. > 4. An improved cross-validation system for the experiment database. > 5. Full population backup, so that all agents are saved, and only manually > deleted at the researcher's request (they don't take much space, and it > would make for an interesting visualisation, and ability to traverse from > the seed agent to the current agent). > > -Gene > [1] [1]https://github.com/CorticalComputer/DXNN First generation DXNN has > a convoluted implementation. DXNN mk2 is a very clean implementation and > is currently on the non master branch, it will eventually overwrite the > master branch but both have the same features (almost) at this time. > [2] [2]http://www.springer.com/computer/swe/book/978-1-4614-4462-6 > [3] > [3]http://www.amazon.com/Handbook-Neuroevolution-through-Erlang-Gene/dp/1461444624/ref=sr_1_1?ie=UTF8&qid=1338163875&sr=8-1 > [4] > [4]http://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher > [5] [5]http://arxiv.org/abs/1111.5892 > > References > > Visible links > 1. https://github.com/CorticalComputer/DXNN > 2. http://www.springer.com/computer/swe/book/978-1-4614-4462-6 > 3. http://www.amazon.com/Handbook-Neuroevolution-through-Erlang-Gene/dp/1461444624/ref=sr_1_1?ie=UTF8&qid=1338163875&sr=8-1 > 4. http://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher > 5. http://arxiv.org/abs/1111.5892 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Sat Jun 16 22:43:07 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 16 Jun 2012 21:43:07 +0100 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> Message-ID: <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> Guys - is there a good discussion/description of the outstanding issues with gen_leader? I'd like to understand its limitations a bit better. Cheers, Tim On 16 Jun 2012, at 01:58, Ulf Wiger wrote: > > Good points. > > Please try out the latest version, which includes a few new functions: > > gproc:await(Node, Key, Timeout) > https://github.com/esl/gproc/blob/master/doc/gproc.md#await-3 > > gproc:wide_await(Nodes, Key, Timeout) > https://github.com/esl/gproc/blob/master/doc/gproc.md#wide_await3 > > gproc:nb_wait(Node, Key) > https://github.com/esl/gproc/blob/master/doc/gproc.md#nb_wait-2 > > gproc:cancel_wait(Node, Key, Ref) > https://github.com/esl/gproc/blob/master/doc/gproc.md#cancel_wait-3 > > > Note that await/3 also returns the Value, so can be throught of as a > distributed get_value() function. > > BR, > Ulf W > > > On 15 Jun 2012, at 16:25, Lo?c Hoguin wrote: > >> Sorry I should have been clearer. >> >> I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. >> >> Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. >> >> It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. >> >> On 06/16/2012 01:18 AM, Ulf Wiger wrote: >>> >>> On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: >>> >>>> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? >>> >>> Uhm, those functions work perfectly fine in a local context? >>> >>> Can you expand on that? >>> >>> Global gproc (gproc_dist) is disabled by default. >>> My assumption is that most people leave it that way. >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto@REDACTED Sat Jun 16 23:48:45 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 16 Jun 2012 14:48:45 -0700 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion Message-ID: Dear all, SublimErl, the Sublime Text 2 editor Erlang plugin, now supports code completion which includes parameter names. An animated GIF showing how it works can be seen here: http://www.ostinelli.net/_out_images/code_completion_full.gif It has now better support for testing [screenshot]: http://www.ostinelli.net/_out_images/running_test.jpeg SublimErl can be found in Github here: https://github.com/ostinelli/SublimErl As usual, comments / criticism / feedback welcome. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.hillsborough@REDACTED Sun Jun 17 03:34:25 2012 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Sat, 16 Jun 2012 21:34:25 -0400 Subject: [erlang-questions] rebar release not starting with an automated start script Message-ID: Hi all, I am using rebar in order to make releases of my Erlang OTP application. The application runs rock solid thus far. I'm having one complication where I am trying to script an automated start of the Erlang application after a reboot. I trimmed down the script to the ultimate basics and am running something like this, completely headless (via Jenkins and/or RightScale). ========== #!/bin/bash -ex chmod u+x /home/erlang/erlapp/bin/erlapp /home/erlang/erlapp/bin/erlapp start ========== The odd part is that it does listen on the TCP port I have it built to listen and accept connections on. However, no connections make it through and I cannot attach to the console using ``/home/erlang/erlapp/bin/erlapp attach``. If I manually execute the above in login shell, everything works just great. Server starts, I can attach to the console. Server accepts connections from clients, etc. Any idea why I can't script start with a rebar release? Thanks all. Matthew From ulf@REDACTED Sun Jun 17 03:58:06 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 16 Jun 2012 18:58:06 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> Message-ID: The main problem is that, while gen_leader is a very well verified implementation of Stoller's leader election algorithm (Hans Svensson got his PhD on the subject), the algorithm doesn't handle netsplits. One can of course "hack" the algorithm, but leader election algos all have a specific band of circumstances within which they are valid. The Stoller algo fits Erlang well, save for the netsplit problem. (Gen_leader was originally based on a different algo, but Hans proved it unsound given the failure modes that exist in Erlang). I think the latest mods to gen_leader seem sound, but there need to be hooks to userland, allowing the leader to merge data and establish a new baseline. My own secret plan is to finish my locking framework that does distributed deadlock detection in a minimal way without a central dependency graph. I've been sitting on that algo since 1993, and Thomas Arts helped me verify it some years ago. Only recently did it occur to me that if two leaders would grab for an election lock after a netsplit, they would deadlock and my algo would automatically resolve it and grant the lock to one of them. Lots to do right now, but one if these days I may release it. :) BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com 16 jun 2012 kl. 13:43 skrev Tim Watson : > Guys - is there a good discussion/description of the outstanding issues with gen_leader? I'd like to understand its limitations a bit better. > > Cheers, > Tim > > On 16 Jun 2012, at 01:58, Ulf Wiger wrote: > >> >> Good points. >> >> Please try out the latest version, which includes a few new functions: >> >> gproc:await(Node, Key, Timeout) >> https://github.com/esl/gproc/blob/master/doc/gproc.md#await-3 >> >> gproc:wide_await(Nodes, Key, Timeout) >> https://github.com/esl/gproc/blob/master/doc/gproc.md#wide_await3 >> >> gproc:nb_wait(Node, Key) >> https://github.com/esl/gproc/blob/master/doc/gproc.md#nb_wait-2 >> >> gproc:cancel_wait(Node, Key, Ref) >> https://github.com/esl/gproc/blob/master/doc/gproc.md#cancel_wait-3 >> >> >> Note that await/3 also returns the Value, so can be throught of as a >> distributed get_value() function. >> >> BR, >> Ulf W >> >> >> On 15 Jun 2012, at 16:25, Lo?c Hoguin wrote: >> >>> Sorry I should have been clearer. >>> >>> I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. >>> >>> Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. >>> >>> It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. >>> >>> On 06/16/2012 01:18 AM, Ulf Wiger wrote: >>>> >>>> On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: >>>> >>>>> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? >>>> >>>> Uhm, those functions work perfectly fine in a local context? >>>> >>>> Can you expand on that? >>>> >>>> Global gproc (gproc_dist) is disabled by default. >>>> My assumption is that most people leave it that way. >>>> >>>> BR, >>>> Ulf W >>>> >>>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>>> http://feuerlabs.com >>>> >>>> >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> >>> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From norton@REDACTED Sun Jun 17 06:38:54 2012 From: norton@REDACTED (Joseph Wayne Norton) Date: Sun, 17 Jun 2012 13:38:54 +0900 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> Message-ID: <3CE11D4D-FDD5-44E8-8D53-38F3AE6B1EE8@lovely.email.ne.jp> What about Mnesia-based storage for gproc? On 2012/06/17, at 10:58, Ulf Wiger wrote: > Lots to do right now, but one if these days I may release it. :) From ulf@REDACTED Sun Jun 17 07:15:54 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 16 Jun 2012 22:15:54 -0700 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: <3CE11D4D-FDD5-44E8-8D53-38F3AE6B1EE8@lovely.email.ne.jp> References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> <3CE11D4D-FDD5-44E8-8D53-38F3AE6B1EE8@lovely.email.ne.jp> Message-ID: Gproc does support mnesia-based storage for gproc:get_env(). https://github.com/esl/gproc/blob/master/doc/gproc.md#get_env4 However, as a replacement for gen_leader, mnesia, unfortunately doesn't handle netsplits all that well either*. :) It wouldn't be all that difficult to modify gproc to use something other than gen_leader, but demand for a mnesia-based gproc hasn't exactly been high so far... BR, Ulf W * Although using http:///github.com/esl/unsplit, it's not half-bad. On 16 Jun 2012, at 21:38, Joseph Wayne Norton wrote: > What about Mnesia-based storage for gproc? > > On 2012/06/17, at 10:58, Ulf Wiger wrote: > >> Lots to do right now, but one if these days I may release it. :) Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From desired.mta@REDACTED Sun Jun 17 09:45:17 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Sun, 17 Jun 2012 09:45:17 +0200 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: Message-ID: <20120617074517.GA15191@localhost> On Sat, Jun 16, 2012 at 10:15:54PM -0700, Ulf Wiger wrote: > > Gproc does support mnesia-based storage for gproc:get_env(). > > https://github.com/esl/gproc/blob/master/doc/gproc.md#get_env4 > > However, as a replacement for gen_leader, mnesia, unfortunately > doesn't handle netsplits all that well either*. :) > > It wouldn't be all that difficult to modify gproc to use something other > than gen_leader, but demand for a mnesia-based gproc hasn't > exactly been high so far... Well, I am just about to write something like this. Use case: * need a (global) mapping from Gid::binary() to Pid::pid(). Every time application wants to send something to Gid, it does a lookup. * if a process crashes, a new one is spawned on monitoring node. * the spawned process registers itself with the Gid. * in case of netsplit, we have two processes with the same key. Damn. * during netsplit, application will be querying Gid to Pid. Only one of the Gids will be returned. We cache somewhere, which one (more on this later). * when nodes are back together, we must decide which process should live or die. * in my case the process which Pid was returned more recently wins (at least for the 0.1 version). Then let the other process know that it should kill itself. So.. Gproc with ways to resolve netsplit is definitely a match here. I am not yet sure how would I keep the "most recent lookup" (probably dirty_write the timestamp on every lookup, but I will have to think more about it). Though gproc counters look not too half-bad here, as well. More ideas welcome. :) Oh, and if we (Spil Games) choose to use it in production, we would test The Gproc With The Mnesia, thoroughly. The whole thing I am building will be open sourced and quite useful IMO, but it's just too early yet to announce it yet (v0.1 on the way). Wild cheers and enthusiasm! ;-) Regards, Motiejus Jak?tys From norton@REDACTED Sun Jun 17 11:12:41 2012 From: norton@REDACTED (Joseph Wayne Norton) Date: Sun, 17 Jun 2012 18:12:41 +0900 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> <3CE11D4D-FDD5-44E8-8D53-38F3AE6B1EE8@lovely.email.ne.jp> Message-ID: <740CB6F1-6FD8-4904-9331-F6E216D554B4@lovely.email.ne.jp> Yes, Mnesia with unsplit seems like an option. How about Mnesia with majority tables? Are there use cases of gproc that could work with Mnesia dirty reads? Thanks. On Jun 17, 2012, at 2:15 PM, Ulf Wiger wrote: > > Gproc does support mnesia-based storage for gproc:get_env(). > > https://github.com/esl/gproc/blob/master/doc/gproc.md#get_env4 > > However, as a replacement for gen_leader, mnesia, unfortunately > doesn't handle netsplits all that well either*. :) > > It wouldn't be all that difficult to modify gproc to use something other > than gen_leader, but demand for a mnesia-based gproc hasn't > exactly been high so far... > > BR, > Ulf W > > * Although using http:///github.com/esl/unsplit, it's not half-bad. > > On 16 Jun 2012, at 21:38, Joseph Wayne Norton wrote: > >> What about Mnesia-based storage for gproc? >> >> On 2012/06/17, at 10:58, Ulf Wiger wrote: >> >>> Lots to do right now, but one if these days I may release it. :) > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From lambdadmitry@REDACTED Sun Jun 17 11:41:55 2012 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Sun, 17 Jun 2012 13:41:55 +0400 Subject: [erlang-questions] Reference binary comparison problem In-Reply-To: References: <4FDB4865.9060708@erix.ericsson.se> <20120615153005.GB23533@ioctl> Message-ID: Thanks for answering. On Fri, Jun 15, 2012 at 11:09 PM, Max Lapshin wrote: > Maybe also use enif_make_resource_binary and put some unique data > inside this binary? From max.lapshin@REDACTED Sun Jun 17 12:40:49 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 17 Jun 2012 14:40:49 +0400 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: Very nice! I'll try Sublime Text 2 for it! On Sun, Jun 17, 2012 at 1:48 AM, Roberto Ostinelli wrote: > Dear all, > > SublimErl, the Sublime Text 2 editor Erlang plugin, now supports > code?completion which includes parameter names. > > An animated GIF showing how it works can be seen here: > http://www.ostinelli.net/_out_images/code_completion_full.gif > > It has now better support for testing [screenshot]: > http://www.ostinelli.net/_out_images/running_test.jpeg > > SublimErl can be found in Github here: > https://github.com/ostinelli/SublimErl > > As usual, comments / criticism / feedback welcome. > > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Sun Jun 17 14:12:28 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 17 Jun 2012 13:12:28 +0100 Subject: [erlang-questions] conditional pub-sub in gproc In-Reply-To: References: <4FDBB494.3000106@gmail.com> <4FDBC0CC.6080600@ninenines.eu> <4FDBC46D.3070305@ninenines.eu> <0C235F84-F92B-414D-A6B3-AB8B23A9CF2D@feuerlabs.com> <2A42E09C-69E8-487D-9262-2211CE6D5E20@gmail.com> Message-ID: <8FA69B2D-7A82-428B-9F9A-BE859D1EF6EF@gmail.com> On 17 Jun 2012, at 02:58, Ulf Wiger wrote: > The main problem is that, while gen_leader is a very well verified implementation of Stoller's leader election algorithm (Hans Svensson got his PhD on the subject), the algorithm doesn't handle netsplits. One can of course "hack" the algorithm, but leader election algos all have a specific band of circumstances within which they are valid. The Stoller algo fits Erlang well, save for the netsplit problem. > > (Gen_leader was originally based on a different algo, but Hans proved it unsound given the failure modes that exist in Erlang). > > I think the latest mods to gen_leader seem sound, but there need to be hooks to userland, allowing the leader to merge data and establish a new baseline. > Ok that's an interesting one to look at. > My own secret plan is to finish my locking framework that does distributed deadlock detection in a minimal way without a central dependency graph. I've been sitting on that algo since 1993, and Thomas Arts helped me verify it some years ago. Only recently did it occur to me that if two leaders would grab for an election lock after a netsplit, they would deadlock and my algo would automatically resolve it and grant the lock to one of them. > > Lots to do right now, but one if these days I may release it. :) > Yes I've heard you mention this in passing before - it's eagerly awaited! :) > BR, > Ulf W > > Ulf Wiger, Feuerlabs, Inc. > http://www.feuerlabs.com > > 16 jun 2012 kl. 13:43 skrev Tim Watson : > >> Guys - is there a good discussion/description of the outstanding issues with gen_leader? I'd like to understand its limitations a bit better. >> >> Cheers, >> Tim >> >> On 16 Jun 2012, at 01:58, Ulf Wiger wrote: >> >>> >>> Good points. >>> >>> Please try out the latest version, which includes a few new functions: >>> >>> gproc:await(Node, Key, Timeout) >>> https://github.com/esl/gproc/blob/master/doc/gproc.md#await-3 >>> >>> gproc:wide_await(Nodes, Key, Timeout) >>> https://github.com/esl/gproc/blob/master/doc/gproc.md#wide_await3 >>> >>> gproc:nb_wait(Node, Key) >>> https://github.com/esl/gproc/blob/master/doc/gproc.md#nb_wait-2 >>> >>> gproc:cancel_wait(Node, Key, Ref) >>> https://github.com/esl/gproc/blob/master/doc/gproc.md#cancel_wait-3 >>> >>> >>> Note that await/3 also returns the Value, so can be throught of as a >>> distributed get_value() function. >>> >>> BR, >>> Ulf W >>> >>> >>> On 15 Jun 2012, at 16:25, Lo?c Hoguin wrote: >>> >>>> Sorry I should have been clearer. >>>> >>>> I'm using global gproc to locate processes that may be on any node of the cluster. But sometimes I run into gen_leader's issues (for example if a node crashes) so I was wondering if I could manage still using gproc for my purposes without gen_leader. >>>> >>>> Currently it's easy to broadcast information to other local gprocs but not so much to retrieve remote data, similar to what get_value/2 or where/1 would do in a distributed context. >>>> >>>> It would be nice to be able to have a cluster of local gprocs and easily access them from any remote node, similar to how bcast works for broadcasting. >>>> >>>> On 06/16/2012 01:18 AM, Ulf Wiger wrote: >>>>> >>>>> On 15 Jun 2012, at 16:10, Lo?c Hoguin wrote: >>>>> >>>>>> That removes some functionality though, doesn't it. Like :get_value/1 and where/1. How would you suggest using these without global gproc? >>>>> >>>>> Uhm, those functions work perfectly fine in a local context? >>>>> >>>>> Can you expand on that? >>>>> >>>>> Global gproc (gproc_dist) is disabled by default. >>>>> My assumption is that most people leave it that way. >>>>> >>>>> BR, >>>>> Ulf W >>>>> >>>>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>>>> http://feuerlabs.com >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Lo?c Hoguin >>>> Erlang Cowboy >>>> Nine Nines >>>> >>>> >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> From erisa85d@REDACTED Sun Jun 17 15:11:57 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Sun, 17 Jun 2012 15:11:57 +0200 Subject: [erlang-questions] Difference btw Erlang ODBC and emysql driver for accessing MySQL Message-ID: Hello, Could somebody take the trouble and explain to me (just the big picture) why should I use emysql driver (https://github.com/Eonblast/Emysql) instead of using the Erlang ODBC application for accessing Mysql Database? Thanks a lot, Erisa From marcel.meyer@REDACTED Sun Jun 17 15:25:35 2012 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Sun, 17 Jun 2012 09:25:35 -0400 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: Amazing, thanks! Sublime has been my weapon if choice for a while now. On Sunday, June 17, 2012, Max Lapshin wrote: > Very nice! > > I'll try Sublime Text 2 for it! > > On Sun, Jun 17, 2012 at 1:48 AM, Roberto Ostinelli > > wrote: > > Dear all, > > > > SublimErl, the Sublime Text 2 editor Erlang plugin, now supports > > code completion which includes parameter names. > > > > An animated GIF showing how it works can be seen here: > > http://www.ostinelli.net/_out_images/code_completion_full.gif > > > > It has now better support for testing [screenshot]: > > http://www.ostinelli.net/_out_images/running_test.jpeg > > > > SublimErl can be found in Github here: > > https://github.com/ostinelli/SublimErl > > > > As usual, comments / criticism / feedback welcome. > > > > r. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Sun Jun 17 15:49:26 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sun, 17 Jun 2012 15:49:26 +0200 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: <7CC7FC29-14CB-4BE4-89E4-223C41E52327@gmail.com> What about this warning message Roberto : Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sublimtext2.png Type: image/png Size: 33326 bytes Desc: not available URL: From ciprian.craciun@REDACTED Sun Jun 17 17:10:11 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Sun, 17 Jun 2012 18:10:11 +0300 Subject: [erlang-questions] Automatic nodes discovering In-Reply-To: References: Message-ID: On Thu, Jun 7, 2012 at 2:06 PM, Maxim Treskin wrote: > Hello > > How I can make automatically discovering new nodes in cluster? When new node > started up it is not listed in output of nodes() on other nodes without > explicit ping of them. There is way to do this using RabbitMQ or ZeroMQ, but > it is some redundant. Yet another way is ping of one known node in cluster, > but it is constrained by nodes startup order. > > May be some interaction with epmd can helps? > > Thank you We had exactly this problem. Thus we've implemented something along these lines: * in case you have multicast, each node sends an "advertisement" on a well established group IP and port; * in case you don't have multicast (as in EC2's case), each node resolves a well known DNS A record and sends there an "advertisement" over TCP to all resolved IP's; (we could have used UDP;) The advertisement is actually an Erlang term in binary format, which is authenticated with an HMAC with the cookie as the secret. This way we are sure nobody tries to "join" if it doesn't know already the cookie. (This advertiement is actually generic and could be used for other things also.) The code -- although experimental -- is open-source under Apache. Unfortunately it's not factored into an independent project, but it has few dependencies. (The TCP code isn't very efficient as for every advertisement we make a new connection... It was done in a hurry.) * https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-node/sources/mosaic_discovery_agent_udp.erl * https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-node/sources/mosaic_discovery_agent_tcp.erl * https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-node/sources/mosaic_discovery_events.erl See `start_discovery` for an example: * https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-node/sources/mosaic_node_app.erl Hope it helps, Ciprian. P.S.: We use it together with `riak_core`, but it is not a dependency. From roberto@REDACTED Sun Jun 17 17:18:44 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 17 Jun 2012 08:18:44 -0700 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: <7CC7FC29-14CB-4BE4-89E4-223C41E52327@gmail.com> References: <7CC7FC29-14CB-4BE4-89E4-223C41E52327@gmail.com> Message-ID: This is a very sporadic message, haven't seen that in months. I've reopened this old issue https://github.com/ostinelli/SublimErl/issues/10 You can safely ignore the warning, you'll probably see this only once when you first start sublime text. r. On Sun, Jun 17, 2012 at 6:49 AM, Zabrane Mickael wrote: > What about this warning message Roberto : > > > Regards, > Zabrane > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sublimtext2.png Type: image/png Size: 33326 bytes Desc: not available URL: From alessandro.sivieri@REDACTED Sun Jun 17 17:31:21 2012 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Sun, 17 Jun 2012 17:31:21 +0200 Subject: [erlang-questions] Statistical modules Message-ID: Hi all, given the fact that searching this on the Web is quite hard, because Erlang is also the name of a probability distribution: do you know if there are modules for performing statistical operations on sets of data (like average, median, standard deviation and such) in Erlang? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Jun 17 17:56:44 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 17 Jun 2012 08:56:44 -0700 Subject: [erlang-questions] Automatic nodes discovering In-Reply-To: References: Message-ID: <4FDDFE3C.2020509@gmail.com> On 06/07/2012 04:06 AM, Maxim Treskin wrote: > How I can make automatically discovering new nodes in cluster? When new node started up it is not listed in output of nodes() on other nodes without explicit ping of them. There is way to do this using RabbitMQ or ZeroMQ, but it is some redundant. Yet another way is ping of one known node in cluster, but it is constrained by nodes startup order. > > May be some interaction with epmd can helps? > I think a project which should help you is nodefinder (http://code.google.com/p/nodefinder/) . I haven't used it yet, but it seems like the best option for doing automatic discovery that also works in ec2, and it is done by the Dukes of Erl, so it should be decent. - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Jun 17 18:11:30 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 17 Jun 2012 09:11:30 -0700 Subject: [erlang-questions] reference a fun from its atom name Message-ID: Dear list, is it possible to reference a fun from its atom name? For instance: 1> eunit:test({generator, fun hello_test_/0}). -> everything is ok. 2> FunName = 'hello_test_/0'. 'hello_test_/0' 3> eunit:test({generator, fun FunName}). * 1: syntax error before: '}' Any ideas? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Sun Jun 17 18:29:17 2012 From: eric@REDACTED (Eric Moritz) Date: Sun, 17 Jun 2012 12:29:17 -0400 Subject: [erlang-questions] Statistical modules In-Reply-To: References: Message-ID: There's two projects that I'm aware of: https://github.com/boundary/bear is used by https://github.com/boundary/folsom And https://github.com/basho/basho_stats Which is used by https://github.com/basho/basho_bench Eric. On Jun 17, 2012 11:31 AM, "Alessandro Sivieri" wrote: > Hi all, > > given the fact that searching this on the Web is quite hard, because > Erlang is also the name of a probability distribution: do you know if there > are modules for performing statistical operations on sets of data (like > average, median, standard deviation and such) in Erlang? > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://sivieri.wordpress.com/ > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Sun Jun 17 19:18:53 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Sun, 17 Jun 2012 19:18:53 +0200 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: Somehow, it complains about tests failures. Is there a log some place to see what the plugin does not like. I provided all the executable paths, still no success. If run rebar eunit from the command line all works as expected. How does one troubleshoot this? Cheers, Alex. On Sat, Jun 16, 2012 at 11:48 PM, Roberto Ostinelli wrote: > Dear all, > > SublimErl, the Sublime Text 2 editor Erlang plugin, now supports > code?completion which includes parameter names. > > An animated GIF showing how it works can be seen here: > http://www.ostinelli.net/_out_images/code_completion_full.gif > > It has now better support for testing [screenshot]: > http://www.ostinelli.net/_out_images/running_test.jpeg > > SublimErl can be found in Github here: > https://github.com/ostinelli/SublimErl > > As usual, comments / criticism / feedback welcome. > > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From freeakk@REDACTED Sun Jun 17 19:20:14 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Sun, 17 Jun 2012 21:20:14 +0400 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: Use fun M:F/A, where M is a module name, F is a function name and A is arity. M and F are atoms, A is non_neg_integer, 17> A = 1. 1 18> X = fun lists:reverse/A. #Fun 19> X([1,2,3]). [3,2,1] From freeakk@REDACTED Sun Jun 17 19:24:04 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Sun, 17 Jun 2012 21:24:04 +0400 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: F = hello_test_, Fun = fun ?MODULE:F/0. From tilman.holschuh@REDACTED Sun Jun 17 19:40:32 2012 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Sun, 17 Jun 2012 10:40:32 -0700 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: Did you try? fun() -> apply(?MODULE, hello_test_, []) end. Cheers - Tilman On 2012-06-17, at 9:11 AM, Roberto Ostinelli wrote: > Dear list, > > is it possible to reference a fun from its atom name? For instance: > > 1> eunit:test({generator, fun hello_test_/0}). > > -> everything is ok. > > 2> FunName = 'hello_test_/0'. > 'hello_test_/0' > 3> eunit:test({generator, fun FunName}). > * 1: syntax error before: '}' > > Any ideas? > > Thank you, > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Sun Jun 17 19:44:54 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sun, 17 Jun 2012 19:44:54 +0200 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: <46E4FE8B-0BEB-4858-9EEC-6ACDFDDEF792@gmail.com> fun M:F/A Sent from my iPhone Le 17 juin 2012 ? 18:11, Roberto Ostinelli a ?crit : > Dear list, > > is it possible to reference a fun from its atom name? For instance: > > 1> eunit:test({generator, fun hello_test_/0}). > > -> everything is ok. > > 2> FunName = 'hello_test_/0'. > 'hello_test_/0' > 3> eunit:test({generator, fun FunName}). > * 1: syntax error before: '}' > > Any ideas? > > Thank you, > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Sun Jun 17 20:24:53 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 17 Jun 2012 12:24:53 -0600 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: On Sat, Jun 16, 2012 at 3:48 PM, Roberto Ostinelli wrote: > Dear all, > > SublimErl, the Sublime Text 2 editor Erlang plugin, now supports > code?completion which includes parameter names. > > An animated GIF showing how it works can be seen here: > http://www.ostinelli.net/_out_images/code_completion_full.gif > > It has now better support for testing [screenshot]: > http://www.ostinelli.net/_out_images/running_test.jpeg > > SublimErl can be found in Github here: > https://github.com/ostinelli/SublimErl > > As usual, comments / criticism / feedback welcome. Never used Sublime Text 2 before, and at $60 it seems pricey for a mostly-text editor, but that depends on what it does, and I will not have a chance to give it a try for a few days. I run a Debian type Linux distribution, and considering that are its features up to par? Why would I use Sublime Text 2 over Eclipse/ErlIDE or so? I really, like really cannot stand Eclipse, tends to start eating a few gigs of memory then saying it is out of handles then just kind of vanishes after a time (on windows, linux, multiple variations of each, no plugins installed, multiple java versions, etc...), so looking for a replacement would be nice. The code-completion alone is the one of the largest reasons I do not use Emacs (yes I know Emacs has code completion but it is rather clunky), and Eclipse code completion is not very quick (cannot keep up with my typing speed), and it appears to have tabbed support, so I am curious what else is there that makes it a good replacement, such as does it handle any sort of code loading, debugging, etc... (or is it just a rather expensive text editor with code completion)? Or does anyone else know of a good Erlang IDE replacement for ErlIDE, Eclipse itself just has far too many issues to use well... From roberto@REDACTED Sun Jun 17 21:12:03 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 17 Jun 2012 12:12:03 -0700 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: Thank you all, I ended up using this one. r. On Sun, Jun 17, 2012 at 10:24 AM, Michael Uvarov wrote: > F = hello_test_, Fun = fun ?MODULE:F/0. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun Jun 17 21:54:33 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 17 Jun 2012 21:54:33 +0200 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: Hi! On Sun, Jun 17, 2012 at 8:24 PM, OvermindDL1 wrote: > Why would I use Sublime Text 2 over Eclipse/ErlIDE or so? ?I really, > like really cannot stand Eclipse, tends to start eating a few gigs of > memory then saying it is out of handles then just kind of vanishes > after a time (on windows, linux, multiple variations of each, no > plugins installed, multiple java versions, etc...), so looking for a > replacement would be nice. ?The code-completion alone is the one of > the largest reasons I do not use Emacs (yes I know Emacs has code > completion but it is rather clunky), and Eclipse code completion is > not very quick (cannot keep up with my typing speed), and it appears > to have tabbed support, so I am curious what else is there that makes > it a good replacement, such as does it handle any sort of code > loading, debugging, etc... (or is it just a rather expensive text > editor with code completion)? This gets a little out of topic, but you might want to know that we are working hard to improve the response time for Erlide and the plan is to get a new engine somewhere at the end of the summer. Also, the completion uses Ctrl-Space instead of tab; you can bind another key like Tab but that may disable other functionality and you need to do it for every new workspace. Regarding Eclipse as a memory and resources hog, I fully agree. It's far from optimal, but for me the advantages are balancing this. I will confess that part of these advantages are things that I am planning to implement, so I am aware that few others have the same attitude :-) best regards, Vlad From overminddl1@REDACTED Mon Jun 18 00:57:27 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 17 Jun 2012 16:57:27 -0600 Subject: [erlang-questions] [ANN] SublimErl v0.2 - with code completion In-Reply-To: References: Message-ID: On Sun, Jun 17, 2012 at 1:54 PM, Vlad Dumitrescu wrote: > Hi! > > On Sun, Jun 17, 2012 at 8:24 PM, OvermindDL1 wrote: >> Why would I use Sublime Text 2 over Eclipse/ErlIDE or so? ?I really, >> like really cannot stand Eclipse, tends to start eating a few gigs of >> memory then saying it is out of handles then just kind of vanishes >> after a time (on windows, linux, multiple variations of each, no >> plugins installed, multiple java versions, etc...), so looking for a >> replacement would be nice. ?The code-completion alone is the one of >> the largest reasons I do not use Emacs (yes I know Emacs has code >> completion but it is rather clunky), and Eclipse code completion is >> not very quick (cannot keep up with my typing speed), and it appears >> to have tabbed support, so I am curious what else is there that makes >> it a good replacement, such as does it handle any sort of code >> loading, debugging, etc... (or is it just a rather expensive text >> editor with code completion)? > > This gets a little out of topic, but you might want to know that we > are working hard to improve the response time for Erlide and the plan > is to get a new engine somewhere at the end of the summer. Also, the > completion uses Ctrl-Space instead of tab; you can bind another key > like Tab but that may disable other functionality and you need to do > it for every new workspace. > > Regarding Eclipse as a memory and resources hog, I fully agree. It's > far from optimal, but for me the advantages are balancing this. I will > confess that part of these advantages are things that I am planning to > implement, so I am aware that few others have the same attitude :-) > > best regards, > Vlad I look forward to seeing ErlIDE grow, but I still hate hitting Ctrl+Space to bring up completion, I like it automatic (too used to other IDEs) and fast enough to keep up with my typing (that can well exceed 100wpm at times, not so much in programming, but when the interfaces freezes for a second that just kills the 'flow', Eclipse problem, not ErlIDE). But still, just using Eclipse for Java work, with no secondary plugins installed at all, just what comes with it, Eclipse eventually *always* dies in the above situation I stated, and it still does with ErlIDE, or C++, or a few other things I tried. Eclipse is horrendously unstable for long-term use. Netbeans has always been stable for me for Java work, but the tools I use for Java do not support it, for Erlang I mostly use emacs (again, code completion issues there, and more), I use KDevelop for C++ (Visual Studio with the Visual Assist plugin installed in the past, Visual Assist is such bliss, I wish it supported KDevelop, or really anything that runs on linux). I am pretty sure I know why Eclipse dies, it seems to be related to its process spawning for debugging leaking handles, if I use Eclipse without ever debugging (so basically not using it beyond as a text editor) then I have ran it for a week without it crashing. Thus brings me to really looking toward a KDevelop level IDE for C++, but for Erlang. I want real-time code completion that keeps up with how I type, tabs, being able to see multiple tabs at once, debugging utilities (tracing things for Erlang specific debugging would be *amazing*), and so forth few other things as the very basics. Hence I have hope for this SublimErl, if it can become a wonderfully response little IDE for Erlang with the above features, even with its cost I would buy and use the parent program. :) If you have ideas for fixing Eclipse so it is actually stable for once, and I have tried a *lot* so far, I would be happy to try. From seth@REDACTED Mon Jun 18 05:18:03 2012 From: seth@REDACTED (Seth Falcon) Date: Sun, 17 Jun 2012 20:18:03 -0700 Subject: [erlang-questions] Difference btw Erlang ODBC and emysql driver for accessing MySQL In-Reply-To: References: Message-ID: Hi Erisa, On Sun, Jun 17, 2012 at 6:11 AM, Erisa Dervishi wrote: > Could somebody take the trouble and explain to me (just the big > picture) why should I use emysql driver > (https://github.com/Eonblast/Emysql) instead of using the Erlang ODBC > application for accessing Mysql Database? You might want to use emysql instead of ODBC because... To use ODBC will require that you have the appropriate odbc driver code installed on your platform. If you use emysql, that's all you need. It implements the MySQL wire protocol, no additional non-erlang software needed. In theory, using a native driver (emysql) can be faster, though that is worth testing if that is of particular importance. A native driver is more likely to expose RDBMS specific features. This is good if you want the features and can be bad if you use them without realizing they are special and want to change backend DBs later on. Based on a quick scan of the ODBC docs, you will get a richer experience using prepared queries in emysql. In particular, you will be able to re-use a prepared query on a given connection, whereas with ODBC it looks like such queries will require parsing by the db engine on each call. Maybe that helps some? + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ From avinash@REDACTED Mon Jun 18 06:45:50 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Mon, 18 Jun 2012 10:15:50 +0530 Subject: [erlang-questions] Difference btw Erlang ODBC and emysql driver for accessing MySQL In-Reply-To: References: Message-ID: Another aspect and consideration - applicable when using commercial DBMS (and, may not be for this specific thread): For commercial deployment of the erlang-solution for integration with DBMS, ODBC of the erlang standard library maybe relied on. Usually ODBC driver is not an issue because customer gets the certified driver from the DBMS vendor. We have taken ODBC application of erlang standard library to the DBMSs (like AS/400, Netezza) not tested (as per the erlang documentation) by Erlang/OTP team, and it just works! On Mon, 18 Jun 2012 08:48:03 +0530, Seth Falcon wrote: > Hi Erisa, > > On Sun, Jun 17, 2012 at 6:11 AM, Erisa Dervishi > wrote: >> Could somebody take the trouble and explain to me (just the big >> picture) why should I use emysql driver >> (https://github.com/Eonblast/Emysql) instead of using the Erlang ODBC >> application for accessing Mysql Database? > > You might want to use emysql instead of ODBC because... > > To use ODBC will require that you have the appropriate odbc driver > code installed on your platform. If you use emysql, that's all you > need. It implements the MySQL wire protocol, no additional non-erlang > software needed. > > In theory, using a native driver (emysql) can be faster, though that > is worth testing if that is of particular importance. > > A native driver is more likely to expose RDBMS specific features. This > is good if you want the features and can be bad if you use them > without realizing they are special and want to change backend DBs > later on. > > Based on a quick scan of the ODBC docs, you will get a richer > experience using prepared queries in emysql. In particular, you will > be able to re-use a prepared query on a given connection, whereas with > ODBC it looks like such queries will require parsing by the db engine > on each call. > > Maybe that helps some? > > + seth From samuelrivas@REDACTED Mon Jun 18 09:11:15 2012 From: samuelrivas@REDACTED (Samuel) Date: Mon, 18 Jun 2012 09:11:15 +0200 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: > Thank you all, I ended up using this one. > >> F = hello_test_, Fun = fun ?MODULE:F/0. Note that this is not exactly what you were trying to do. I guess you were trying to create a fun to call a local (possibly unexported function). The construction above creates a fun that does a fully qualified call (thus, the function must be exported). Your problem was that instead of a fun you defined an atom: 2> FunName = 'hello_test_/0'. 'hello_test_/0' Anything enclosed in '' is an atom. What you were trying to do is probably 2> FunName = fun hello_test_/0. That's the valid syntax to define a fun, and is roughly (not exactly) equivalent to 2> FunName = fun() -> hello_test() end. However the first construct will fail in the shell, because you are in the erl_eval module context there: 2> FunName = fun hello_test_/0. ** exception error: undefined function erl_eval:hello_test_/0 But it will work if you use it in a module that defines hello_test_() Regards -- Samuel From samuelrivas@REDACTED Mon Jun 18 09:16:21 2012 From: samuelrivas@REDACTED (Samuel) Date: Mon, 18 Jun 2012 09:16:21 +0200 Subject: [erlang-questions] Check for file used by any process In-Reply-To: References: Message-ID: Using lsof for that kind of thing is not safe, you can still end up in situations with several process writing the same file. The right solution depends on the problem (as usual :) ). If all the access to those files are done by your erlang application maybe you need to implement a server that ensures only one worker has write access to the file. If there are external applications that can access those files you will need to implement some kind of locking. Regards -- Samuel From dm.klionsky@REDACTED Mon Jun 18 09:58:42 2012 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Mon, 18 Jun 2012 10:58:42 +0300 Subject: [erlang-questions] Difference btw Erlang ODBC and emysql driver for accessing MySQL In-Reply-To: References: Message-ID: <4FDEDFB2.7080600@gmail.com> Hello, The number one reason, as it was already mentioned, is speed. I suggest you first have a look at http://erlang.org/pipermail/erlang-questions/2008-September/038010.html. I've recently benchmarked speed for MySQL and PostgreSQL using both ODBC and native drivers and the ODBC tests were an order of magnitude slower. On 06/17/2012 04:11 PM, Erisa Dervishi wrote: > Hello, > Could somebody take the trouble and explain to me (just the big > picture) why should I use emysql driver > (https://github.com/Eonblast/Emysql) instead of using the Erlang ODBC > application for accessing Mysql Database? > > Thanks a lot, > Erisa > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Dmitry Klionsky From clist@REDACTED Mon Jun 18 10:24:26 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 18 Jun 2012 10:24:26 +0200 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: Message-ID: <201206181024.26898.clist@uah.es> Hello Just saw this, and it is also perfect for my needs I need to load a potentially huge hessian data file and serve arbitrary number of lines to many processes in a molecular simulator. now im reading entire file in memory as testing doesnt involve large molecules but it is getting bigger day by day. I was thinking on read data on start and store it in a dets but this seems better. thanks /Angel On Martes, 12 de Junio de 2012 09:19:43 Max Lapshin escribi?: > Read, 10 KBytes, split them with binary:split and parse 10 lines. > > Remember their byte size and calculate offset of next 10 lines. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon Jun 18 10:49:22 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 18 Jun 2012 12:49:22 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: <201206181024.26898.clist@uah.es> References: <201206181024.26898.clist@uah.es> Message-ID: I believe, that my way is fastest: import CSV once by reading it in chunks and store it serialized with file:write_file(Path, erlang:term_to_binary(Rows, [compressed])) I've promised to fix benchmark and I remember about it. From bob@REDACTED Mon Jun 18 13:58:27 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 18 Jun 2012 04:58:27 -0700 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <201206181024.26898.clist@uah.es> Message-ID: If your use case is streaming and the files are big you might be better off serializing with disk_log, although you won't get any compression that way. On Monday, June 18, 2012, Max Lapshin wrote: > I believe, that my way is fastest: import CSV once by reading it in > chunks and store it serialized with file:write_file(Path, > erlang:term_to_binary(Rows, [compressed])) > > I've promised to fix benchmark and I remember about it. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 18 14:02:58 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 18 Jun 2012 16:02:58 +0400 Subject: [erlang-questions] Large Flat file operations - optimization. In-Reply-To: References: <201206181024.26898.clist@uah.es> Message-ID: On Mon, Jun 18, 2012 at 3:58 PM, Bob Ippolito wrote: > If your use case is streaming and the files are big you might be better off > serializing with disk_log, although you won't get any compression that way. > Using option [compressed] in erlang:term_to_binary can compress my data 100 times From mscame@REDACTED Mon Jun 18 14:08:52 2012 From: mscame@REDACTED (William Yangzhuwei) Date: Mon, 18 Jun 2012 14:08:52 +0200 Subject: [erlang-questions] ip_v6_disabled cannot be set for ssh connection in OTP R15B01 Message-ID: hello, Once I try to connect with ssh with ip_v6_disabled = true, like following ssh:connect("some hostname",2022,[{user,"user"}, {password,"pwd"}, {user_interaction,false}, {user_dir,"/tmp/tmp.6xqteJfWII"}, {silently_accept_hosts,true}, {ip_v6_disabled,true}],infinity) I will got some error return {error, {eoptions,{ip_v6_disabled,true}}} I have check the code of OTP, These two lines looks strange. handle_ssh_option({ip_v6_disabled, Value} = Opt) when is_function(Value) -> Opt; Why it must be a function? BTW, ip_v6_disabled = true works in R14B -- Best Regards //William YangZhuwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Mon Jun 18 14:19:46 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 18 Jun 2012 14:19:46 +0200 Subject: [erlang-questions] ip_v6_disabled cannot be set for ssh connection in OTP R15B01 In-Reply-To: References: Message-ID: 2012/6/18 William Yangzhuwei : > hello, > > Once I try to connect with ssh with ip_v6_disabled = true, like following > > ?ssh:connect("some hostname",2022,[{user,"user"}, > ?{password,"pwd"}, > ?{user_interaction,false}, > ?{user_dir,"/tmp/tmp.6xqteJfWII"}, > ?{silently_accept_hosts,true}, > ?{ip_v6_disabled,true}],infinity) > > I will got some error return > > {error, > > {eoptions,{ip_v6_disabled,true}}} > > > I have check the code of OTP, > These two lines looks strange. > > handle_ssh_option({ip_v6_disabled, Value} = Opt) when is_function(Value) -> > ??? Opt; > > Why it must be a function? That's a copy&paste error, fixed in this commit: https://github.com/erlang/otp/commit/b85e884f14e1dce52af7157045625f1ef5441ed1 From mscame@REDACTED Mon Jun 18 14:42:02 2012 From: mscame@REDACTED (William Yangzhuwei) Date: Mon, 18 Jun 2012 14:42:02 +0200 Subject: [erlang-questions] ip_v6_disabled cannot be set for ssh connection in OTP R15B01 In-Reply-To: References: Message-ID: Thanks for your reply. This fix will be included in next release then, I guess. BRs//William On Mon, Jun 18, 2012 at 2:19 PM, Attila Rajmund Nohl < attila.r.nohl@REDACTED> wrote: > 2012/6/18 William Yangzhuwei : > > hello, > > > > Once I try to connect with ssh with ip_v6_disabled = true, like following > > > > ssh:connect("some hostname",2022,[{user,"user"}, > > {password,"pwd"}, > > {user_interaction,false}, > > {user_dir,"/tmp/tmp.6xqteJfWII"}, > > {silently_accept_hosts,true}, > > {ip_v6_disabled,true}],infinity) > > > > I will got some error return > > > > {error, > > > > {eoptions,{ip_v6_disabled,true}}} > > > > > > I have check the code of OTP, > > These two lines looks strange. > > > > handle_ssh_option({ip_v6_disabled, Value} = Opt) when is_function(Value) > -> > > Opt; > > > > Why it must be a function? > > That's a copy&paste error, fixed in this commit: > > https://github.com/erlang/otp/commit/b85e884f14e1dce52af7157045625f1ef5441ed1 > -- Best Regards //William YangZhuwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From erisa85d@REDACTED Mon Jun 18 16:42:02 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Mon, 18 Jun 2012 16:42:02 +0200 Subject: [erlang-questions] Difference btw Erlang ODBC and emysql driver for accessing MySQL In-Reply-To: <4FDEDFB2.7080600@gmail.com> References: <4FDEDFB2.7080600@gmail.com> Message-ID: Thanks guys for your answers! On Mon, Jun 18, 2012 at 9:58 AM, Dmitry Klionsky wrote: > Hello, > > The number one reason, as it was already mentioned, is speed. > I suggest you first have a look at > http://erlang.org/pipermail/erlang-questions/2008-September/038010.html. > I've recently benchmarked speed for MySQL and PostgreSQL using both ODBC and > native drivers and > the ODBC tests were an order of magnitude slower. > > > > On 06/17/2012 04:11 PM, Erisa Dervishi wrote: >> >> Hello, >> Could somebody take the trouble and explain to me (just the big >> picture) why should I use emysql driver >> (https://github.com/Eonblast/Emysql) instead of using the Erlang ODBC >> application for accessing Mysql Database? >> >> Thanks a lot, >> Erisa >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best regards, > Dmitry Klionsky > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto@REDACTED Mon Jun 18 17:04:48 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 18 Jun 2012 08:04:48 -0700 Subject: [erlang-questions] reference a fun from its atom name In-Reply-To: References: Message-ID: On Monday, June 18, 2012, Samuel wrote: > > Thank you all, I ended up using this one. > > > >> F = hello_test_, Fun = fun ?MODULE:F/0. > > Note that this is not exactly what you were trying to do. I guess you > were trying to create a fun to call a local (possibly unexported > function). no, this is exactly what I wanted to do. I have a function name as an atom in a variable, and I needed a Fun reference out of it. Your problem was that instead of a fun you defined an atom: > > 2> FunName = 'hello_test_/0'. > 'hello_test_/0' > > Anything enclosed in '' is an atom. What you were trying to do is probably > > 2> FunName = fun hello_test_/0. no, believe it or not but I actually may know the difference between an atom and a function. no matter how incredible this may sound. :) ...r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Jun 18 21:17:33 2012 From: ingela@REDACTED (Ingela Andin) Date: Mon, 18 Jun 2012 21:17:33 +0200 Subject: [erlang-questions] What are the consequences of not closing the SSL connection ? In-Reply-To: References: Message-ID: Hi! 2012/6/15 Morgan Segalis : > A simple question really, > > I have a gen_server that is handling my SSL connection? > Since now I just found out, that I did not closed the ssl connection handled by the gen_server when this one terminates? > > Of course I have added the close, now? > > But it got me curious, what would be the consequence if not explicitly closing the connection when the gen_server terminates ? > Does it still lives, or does it get closed by itself? ? > If the process that controlls the ssl-socket terminates the ssl-connection process will terminate too. The controlling process is the process that started the ssl connection unless you transfered the control by calling ssl:controlling_process/2. Regards Ingela Erlang/OTP team - Ericsson AB From hvjunk@REDACTED Mon Jun 18 21:24:24 2012 From: hvjunk@REDACTED (Hendrik Visage) Date: Mon, 18 Jun 2012 21:24:24 +0200 Subject: [erlang-questions] Erlang creator please grow a beard!!! Message-ID: http://www.wired.com/wiredenterprise/2012/06/beard-gallery/?viewall=true ;) From ingela.andin@REDACTED Mon Jun 18 21:26:32 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 18 Jun 2012 21:26:32 +0200 Subject: [erlang-questions] ip_v6_disabled cannot be set for ssh connection in OTP R15B01 In-Reply-To: References: Message-ID: Hi! 2012/6/18 William Yangzhuwei : > hello, > > Once I try to connect with ssh with ip_v6_disabled = true, like following > > ?ssh:connect("some hostname",2022,[{user,"user"}, > ?{password,"pwd"}, > ?{user_interaction,false}, > ?{user_dir,"/tmp/tmp.6xqteJfWII"}, > ?{silently_accept_hosts,true}, > ?{ip_v6_disabled,true}],infinity) > > I will got some error return > > {error, > > {eoptions,{ip_v6_disabled,true}}} > > > I have check the code of OTP, > These two lines looks strange. > > handle_ssh_option({ip_v6_disabled, Value} = Opt) when is_function(Value) -> > ??? Opt; > > Why it must be a function? That is a bug, it has been fixed in the upcoming version. Regards Ingela Erlang/OTP team - Ericsson AB From gleber.p@REDACTED Mon Jun 18 21:42:57 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 18 Jun 2012 21:42:57 +0200 Subject: [erlang-questions] What are the consequences of not closing the SSL connection ? In-Reply-To: References: Message-ID: I've spent like 3 minutes looking for this information in eel -man {ssl,inet,gen_tcp} and found nothing about termination of controlling process. Was I looking not carefully enough or is it missing from docs? On Monday, June 18, 2012, Ingela Andin wrote: > Hi! > > 2012/6/15 Morgan Segalis >: > > A simple question really, > > > > I have a gen_server that is handling my SSL connection? > > Since now I just found out, that I did not closed the ssl connection > handled by the gen_server when this one terminates? > > > > Of course I have added the close, now? > > > > But it got me curious, what would be the consequence if not explicitly > closing the connection when the gen_server terminates ? > > Does it still lives, or does it get closed by itself? ? > > > > If the process that controlls the ssl-socket terminates the > ssl-connection process will terminate too. The controlling process is > the process that started the ssl connection > unless you transfered the control by calling ssl:controlling_process/2. > > Regards Ingela Erlang/OTP team - Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwilberding@REDACTED Mon Jun 18 22:08:35 2012 From: jwilberding@REDACTED (Jordan Wilberding) Date: Mon, 18 Jun 2012 16:08:35 -0400 Subject: [erlang-questions] [ANN] ErlangCamp Coruna, Spain 2012 Message-ID: *Greetings all, We are happy to announce ErlangCamp 2012 is going to be in Coru?a, Spain this Fall. ErlangCamp is a two day hands on workshop for those wanting to go from novice or even an experienced level of programming Erlang to being able to confidently write production grade Erlang/OTP applications. Erlang Camp Coru?a is an opportunity to learn from Erlang/OTP experts. Engineers who have put many many lines of code into production at companies ranging in all sizes. Two of the presenters are the authors of ?Erlang and OTP in Action?. The curriculum will roughly follow the progression from the book (found at http://manning.com/logan) but present a great deal of new material from different angles. When you leave you will know how to confidently put massively parallel, fault tolerant, distributed Erlang/OTP applications into production and then manage them from there. For more information and early-bird registration price of only 80 euros (100 dollars), please go to: http://erlangcamp.com To keep up with what is going on with the Camp you can follow us on twitter here http://twitter.com/erlangcamp or register for an email update on the site. For any questions, please e-mail us organizers@REDACTED Thanks, The Erlang Camp team* -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Mon Jun 18 22:48:12 2012 From: eric@REDACTED (Eric Moritz) Date: Mon, 18 Jun 2012 16:48:12 -0400 Subject: [erlang-questions] Statistical modules In-Reply-To: References: Message-ID: I believe folsom's histograms only keep 60 second windows on the histogram data. That may lead to surprises if he's not using it for realtime histograms. He can use bear directly to avoid what could become a subtle bug if things take longer than 60 to load. Eric. On Mon, Jun 18, 2012 at 2:15 PM, Dave Smith wrote: > Use folsom -- we'll eventually deprecate basho_stats in favor of that. > > D. > > On Sun, Jun 17, 2012 at 10:29 AM, Eric Moritz wrote: >> There's two projects that I'm aware of: >> >> https://github.com/boundary/bear is used by >> https://github.com/boundary/folsom >> >> And >> https://github.com/basho/basho_stats >> >> Which is used by https://github.com/basho/basho_bench >> >> Eric. >> >> On Jun 17, 2012 11:31 AM, "Alessandro Sivieri" >> wrote: >>> >>> Hi all, >>> >>> given the fact that searching this on the Web is quite hard, because >>> Erlang is also the name of a probability distribution: do you know if there >>> are modules for performing statistical operations on sets of data (like >>> average, median, standard deviation and such) in Erlang? >>> >>> -- >>> Sivieri Alessandro >>> alessandro.sivieri@REDACTED >>> http://sivieri.wordpress.com/ >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From paulperegud@REDACTED Mon Jun 18 23:24:53 2012 From: paulperegud@REDACTED (Paul Peregud) Date: Mon, 18 Jun 2012 23:24:53 +0200 Subject: [erlang-questions] Fwd: Re: Statistical modules In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: "Paul Peregud" Date: Jun 18, 2012 11:14 PM Subject: Re: [erlang-questions] Statistical modules To: "Eric Moritz" No, this is true only for recently introduced "sliding window" type of histograms. Both uniform and normal types sample from whole population. On Jun 18, 2012 10:48 PM, "Eric Moritz" wrote: > I believe folsom's histograms only keep 60 second windows on the > histogram data. That may lead to surprises if he's not using it for > realtime histograms. He can use bear directly to avoid what could > become a subtle bug if things take longer than 60 to load. > > Eric. > > On Mon, Jun 18, 2012 at 2:15 PM, Dave Smith wrote: > > Use folsom -- we'll eventually deprecate basho_stats in favor of that. > > > > D. > > > > On Sun, Jun 17, 2012 at 10:29 AM, Eric Moritz > wrote: > >> There's two projects that I'm aware of: > >> > >> https://github.com/boundary/bear is used by > >> https://github.com/boundary/folsom > >> > >> And > >> https://github.com/basho/basho_stats > >> > >> Which is used by https://github.com/basho/basho_bench > >> > >> Eric. > >> > >> On Jun 17, 2012 11:31 AM, "Alessandro Sivieri" > >> wrote: > >>> > >>> Hi all, > >>> > >>> given the fact that searching this on the Web is quite hard, because > >>> Erlang is also the name of a probability distribution: do you know if > there > >>> are modules for performing statistical operations on sets of data (like > >>> average, median, standard deviation and such) in Erlang? > >>> > >>> -- > >>> Sivieri Alessandro > >>> alessandro.sivieri@REDACTED > >>> http://sivieri.wordpress.com/ > >>> > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Tue Jun 19 03:13:27 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 18 Jun 2012 18:13:27 -0700 Subject: [erlang-questions] CQRS, Event Sourcing and Erlang Message-ID: I have been reading up on CQRS and Event Sourcing ( http://martinfowler.com/bliki/CQRS.html) and I was wondering if anyone has tried to tackle this in Erlang. It seems like it would be a good fit (even though it's geared more toward OO languages) because of Erlang's ease of handling async events and messages. Most examples I've seen seem to be overly complicated to handle the async messages since most languages don't do it like Erlang. Does anyone have a sample they could share to get me started or maybe point me in the right direction to build a system in Erlang using these concepts? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Tue Jun 19 12:01:33 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 19 Jun 2012 12:01:33 +0200 Subject: [erlang-questions] how: I would like to write a good bug report Message-ID: <1340100093.7328.65.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, After the switch to R15B01 I get the following crash(*) when I run a test. How should I make a good bug report? Is is sufficient with: "ct_line module is missing line/1 function" ? bengt === Ended at 2012-06-19 10:36:01 === location [{ct_line,line}, {sbg_st_legacy_char_SUITE,init_per_suite,0}, {test_server,ts_tc,1834}, {test_server,run_test_case_eval1,1434}, {test_server,run_test_case_eval,1302}] === reason = undefined function ct_line:line/1 in function sbg_st_legacy_char_SUITE:init_per_suite/1 in call from test_server:ts_tc/3 (test_server.erl, line 1834) in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1434) in call from test_server:run_test_case_eval/9 (test_server.erl, line 1302) === *** init_per_suite failed. Skipping all cases. From lukas@REDACTED Tue Jun 19 16:28:46 2012 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 19 Jun 2012 16:28:46 +0200 Subject: [erlang-questions] how: I would like to write a good bug report In-Reply-To: <1340100093.7328.65.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> References: <1340100093.7328.65.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: Hi, The stacktrace below and a description of which version of Erlang you are running should be enough for a basic bug report. If more info is needed, we'll get back to you. As for the error, I suspect that the SUITE has been compiled with a pre R15B compiler and is being run with a post R15B runtime. Recomping the SUITE with a new compiler should solve the problem. We try to be backwards compatible in the interfaces, but it seems we missed this scenario. Lukas On Tue, Jun 19, 2012 at 12:01 PM, Bengt Kleberg wrote: > Greetings, > > After the switch to R15B01 I get the following crash(*) when I run a > test. How should I make a good bug report? Is is sufficient with: > "ct_line module is missing line/1 function" > ? > > > bengt > > === Ended at 2012-06-19 10:36:01 > === location [{ct_line,line}, > ?{sbg_st_legacy_char_SUITE,init_per_suite,0}, > ?{test_server,ts_tc,1834}, > ?{test_server,run_test_case_eval1,1434}, > ?{test_server,run_test_case_eval,1302}] > === reason = undefined function ct_line:line/1 > ?in function ?sbg_st_legacy_char_SUITE:init_per_suite/1 > ?in call from test_server:ts_tc/3 (test_server.erl, line 1834) > ?in call from test_server:run_test_case_eval1/6 (test_server.erl, line > 1434) > ?in call from test_server:run_test_case_eval/9 (test_server.erl, line > 1302) > === > *** init_per_suite failed. > ? ?Skipping all cases. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Tue Jun 19 16:34:32 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 19 Jun 2012 16:34:32 +0200 Subject: [erlang-questions] how: I would like to write a good bug report In-Reply-To: References: <1340100093.7328.65.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: <1340116472.7328.129.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> I am running with the ct_run command. It should recompile the suite for me. bengt On Tue, 2012-06-19 at 16:28 +0200, Lukas Larsson wrote: > Hi, > > The stacktrace below and a description of which version of Erlang you > are running should be enough for a basic bug report. If more info is > needed, we'll get back to you. > > As for the error, I suspect that the SUITE has been compiled with a > pre R15B compiler and is being run with a post R15B runtime. Recomping > the SUITE with a new compiler should solve the problem. > > We try to be backwards compatible in the interfaces, but it seems we > missed this scenario. > > Lukas > > On Tue, Jun 19, 2012 at 12:01 PM, Bengt Kleberg > wrote: > > Greetings, > > > > After the switch to R15B01 I get the following crash(*) when I run a > > test. How should I make a good bug report? Is is sufficient with: > > "ct_line module is missing line/1 function" > > ? > > > > > > bengt > > > > === Ended at 2012-06-19 10:36:01 > > === location [{ct_line,line}, > > {sbg_st_legacy_char_SUITE,init_per_suite,0}, > > {test_server,ts_tc,1834}, > > {test_server,run_test_case_eval1,1434}, > > {test_server,run_test_case_eval,1302}] > > === reason = undefined function ct_line:line/1 > > in function sbg_st_legacy_char_SUITE:init_per_suite/1 > > in call from test_server:ts_tc/3 (test_server.erl, line 1834) > > in call from test_server:run_test_case_eval1/6 (test_server.erl, line > > 1434) > > in call from test_server:run_test_case_eval/9 (test_server.erl, line > > 1302) > > === > > *** init_per_suite failed. > > Skipping all cases. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From raould@REDACTED Tue Jun 19 18:17:40 2012 From: raould@REDACTED (Raoul Duke) Date: Tue, 19 Jun 2012 09:17:40 -0700 Subject: [erlang-questions] CQRS, Event Sourcing and Erlang In-Reply-To: References: Message-ID: i should think just look for http/rest send-and-forget examples. From hd2010@REDACTED Tue Jun 19 21:05:52 2012 From: hd2010@REDACTED (H. Diedrich) Date: Tue, 19 Jun 2012 21:05:52 +0200 Subject: [erlang-questions] Missing SASL Crash Report Message-ID: <4FE0CD90.6090507@eonblast.com> Hi, why would I NOT get a CRASH REPORT listed out (although there is a crash) in cases where I start a program directly, like $ erl -boot start_sasl -pa ./ebin ./lib/Emysql/ebin -s run run -s init stop -noshell -module(run). -export([run/0]). run() -> crypto:start(), emysql:start(), mysrv:start(), pong = mysrv:ping(), pong = mysrv:ping(gen_server), io:format("start up and pings complete.~n"), ok. But if I do that in the shell, I get a CRASH REPORT (which is what I want to find a bug)? $ erl -boot start_sasl -pa ./ebin ./lib/Emysql/ebin > crypto:start(), emysql:start(), nbtsrv:start(). Thanks, Henning From romanshestakov@REDACTED Tue Jun 19 21:50:10 2012 From: romanshestakov@REDACTED (Roman Shestakov) Date: Tue, 19 Jun 2012 20:50:10 +0100 (BST) Subject: [erlang-questions] (no subject) Message-ID: <1340135410.28209.YahooMailNeo@web171305.mail.ir2.yahoo.com> http://kaycemcgehee.com/wp-content/themes/twentyeleven/googles.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From snarayan@REDACTED Wed Jun 20 07:23:04 2012 From: snarayan@REDACTED (Sandhya Narayan) Date: Wed, 20 Jun 2012 05:23:04 +0000 Subject: [erlang-questions] Requesting a skype call between 6:30 - 8 AM Message-ID: Hi Konrad, We have several questions about LINC and our Xen environment. 1. We have heap allocation problem when using LINC even after increasing Dom0 main memory. 2. We could not use LINC in xenovs, and cannot ping to VMs on xenovs. This is after increasing Dom0 main memory, and rebooting. 3. Need to discuss flow-mods and sys.config. Need some examples. Please let me if we can connect with you on skype tomorrow morning sometime between 6:30 AM - 8 AM PDT. Thanks Sandhya -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Wed Jun 20 13:45:13 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 13:45:13 +0200 Subject: [erlang-questions] how to seek Leaking processes ? Message-ID: Hi everyone ! My Erlang App seems to have a process leak? I would like to know if there is a technic in order to seek any process leak in Erlang ? (pinpoint the module that leaks, or get the module of the leaked process) I cannot seems to be able to reproduce the issue manually, I only see it on production servers. Any help, or hint would be greatly appreciated. Thank you, Morgan. From max.lapshin@REDACTED Wed Jun 20 14:00:49 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Jun 2012 16:00:49 +0400 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: Message-ID: 1) always use OTP: simple_one_for_one supervisors and gen_server 2) launch appmon 3) profit From attila.r.nohl@REDACTED Wed Jun 20 14:02:18 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 20 Jun 2012 14:02:18 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: Message-ID: 2012/6/20 Morgan Segalis : > Hi everyone ! > > My Erlang App seems to have a process leak? > I would like to know if there is a technic in order to seek any process leak in Erlang ? (pinpoint the module that leaks, or get the module of the leaked process) > > I cannot seems to be able to reproduce the issue manually, I only see it on production servers. > > Any help, or hint would be greatly appreciated. Periodically run erlang:processes() and check what are the extra processes? From msegalis@REDACTED Wed Jun 20 14:05:06 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 14:05:06 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: Message-ID: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> Hi, Actually I am running erlang:processes(), more precisely length(erlang:processes()). erlang:processes() only gives me Pid, doesn't really give me any hint about which module is leaking? Thanks, Morgan. Le 20 juin 2012 ? 14:02, Attila Rajmund Nohl a ?crit : > 2012/6/20 Morgan Segalis : >> Hi everyone ! >> >> My Erlang App seems to have a process leak? >> I would like to know if there is a technic in order to seek any process leak in Erlang ? (pinpoint the module that leaks, or get the module of the leaked process) >> >> I cannot seems to be able to reproduce the issue manually, I only see it on production servers. >> >> Any help, or hint would be greatly appreciated. > > Periodically run erlang:processes() and check what are the extra processes? From msegalis@REDACTED Wed Jun 20 14:08:57 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 14:08:57 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: Message-ID: I'm already using simple_one_for_one supervisors and gen_server only. (But why is this important ?) But I'm also running third party module, like emysql, thrift, gen_server_pool. Le 20 juin 2012 ? 14:00, Max Lapshin a ?crit : > 1) always use OTP: simple_one_for_one supervisors and gen_server > 2) launch appmon > 3) profit From gleber.p@REDACTED Wed Jun 20 14:09:11 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 20 Jun 2012 14:09:11 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> References: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> Message-ID: On Wed, Jun 20, 2012 at 2:05 PM, Morgan Segalis wrote: > Hi, > > Actually I am running erlang:processes(), more precisely length(erlang:processes()). > > erlang:processes() only gives me Pid, doesn't really give me any hint about which module is leaking? Use erlang:process_info(Pid) and sys:get_status(Pid) to figure out what those leaking processes are From max.lapshin@REDACTED Wed Jun 20 14:17:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Jun 2012 16:17:19 +0400 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> Message-ID: So, Morgan is right. If your code doesn't leak (and you see normal amount of processes via appmon), than use following code: PidList1 = processes(), timer:sleep(5000), PidList2 = processes(), Delta = PidList2 - PidList1, than inspect them with process_info(Pid) You need dictionary, current_function, etc. From msegalis@REDACTED Wed Jun 20 14:40:13 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 14:40:13 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> Message-ID: <7A39FCB9-796D-42B6-9C19-4139996BA90A@gmail.com> Hi Max, unfortunately, the number of connected users is constantly growing, the delta will not really help here. However, yes, I guess dictionary, current_function will help here? I'll try to do a counter for each type, and print the whole structure, to see which counter is abnormally high? Thanks, Morgan Le 20 juin 2012 ? 14:17, Max Lapshin a ?crit : > So, Morgan is right. > > If your code doesn't leak (and you see normal amount of processes via > appmon), than use following code: > > > PidList1 = processes(), > timer:sleep(5000), > PidList2 = processes(), > Delta = PidList2 - PidList1, > > than inspect them with process_info(Pid) > > You need dictionary, current_function, etc. From max.lapshin@REDACTED Wed Jun 20 14:42:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Jun 2012 16:42:19 +0400 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: <7A39FCB9-796D-42B6-9C19-4139996BA90A@gmail.com> References: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> <7A39FCB9-796D-42B6-9C19-4139996BA90A@gmail.com> Message-ID: On Wed, Jun 20, 2012 at 4:40 PM, Morgan Segalis wrote: > Hi Max, > > unfortunately, the number of connected users is constantly growing, the delta will not really help here. > However, yes, I guess dictionary, current_function will help here? > So remove from this delta those processes, which are listed in your supervisor tree. From msegalis@REDACTED Wed Jun 20 14:51:54 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 14:51:54 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: <83B44484-06E3-4BE2-8557-5F5B9ED32410@gmail.com> <7A39FCB9-796D-42B6-9C19-4139996BA90A@gmail.com> Message-ID: <791B32F9-1FCF-47DF-9C17-A293B3B2E9B0@gmail.com> Yeah, good idea, after I'm sure that the leak is not coming from that, that's what I'll do !! Thanks ! Morgan. Le 20 juin 2012 ? 14:42, Max Lapshin a ?crit : > On Wed, Jun 20, 2012 at 4:40 PM, Morgan Segalis wrote: >> Hi Max, >> >> unfortunately, the number of connected users is constantly growing, the delta will not really help here. >> However, yes, I guess dictionary, current_function will help here? >> > > > So remove from this delta those processes, which are listed in your > supervisor tree. From robert.virding@REDACTED Wed Jun 20 15:53:46 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 20 Jun 2012 14:53:46 +0100 (BST) Subject: [erlang-questions] Let's get rid of packages In-Reply-To: Message-ID: <39031268-0661-49bd-a90c-43057bc9595d@knuth> +1 Yes, let's get rid of them. Robert ----- Original Message ----- > Hello everyone, > I have heard (delightful) rumors that packages are being removed. If > this is true, can it be confirmed? > Otherwise I would like to provide some patches because embedded mode > is not working correctly with packages (but everything else seems to > be working properly in my experience). > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Jun 20 20:36:55 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 20 Jun 2012 13:36:55 -0500 Subject: [erlang-questions] Chicago meetup tonight Message-ID: If you in the Chicago area and have some free time tonight, this will be epic: * http://www.meetup.com/ErlangChicago/events/68878062/ Garrett Fail or success TDB From msegalis@REDACTED Wed Jun 20 20:53:00 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 20:53:00 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: Message-ID: <1B73534E-FDA1-4934-BD06-734D1D1A3107@gmail.com> To Everyone ! Thank you all for your help? I have been able to find the leak thanks to all of you? Here's what I ended up with : info() -> [{init, InitProcesses}] = ets:lookup(table_process, init), CurrentProcesses = erlang:processes(), DiffProcesses = lists:subtract(CurrentProcesses, InitProcesses), get_info(DiffProcesses, dict:new()). get_info([], Dict) -> io:fwrite("Dict: ~p~n", [Dict]); get_info([P|L], Dict) -> case erlang:process_info(P, dictionary) of {dictionary, Info} -> Initial_call = get_dict('$initial_call', Info), Ancestor = get_dict('$ancestors', Info), case dict:find({Initial_call, Ancestor}, Dict) of {ok, Nb} -> IncrNb = Nb + 1; error -> IncrNb = 1 end, Dict2 = dict:store({Initial_call, Ancestor}, IncrNb, Dict), get_info(L, Dict2); undefined -> case dict:find(undefined, Dict) of {ok, Nb} -> IncrNb = Nb + 1; error -> IncrNb = 1 end, Dict2 = dict:store(undefined, IncrNb, Dict), get_info(L, Dict2) end. With at the end of the start : ets:new(table_process, [named_table, protected, set, {keypos, 1}]), ets:insert(table_process, {init, erlang:processes()}). The ets table give me the ability to remove all Erlang's VM processes of the dictionary ! Hope it helps someone else, Morgan. Le 20 juin 2012 ? 13:45, Morgan Segalis a ?crit : > Hi everyone ! > > My Erlang App seems to have a process leak? > I would like to know if there is a technic in order to seek any process leak in Erlang ? (pinpoint the module that leaks, or get the module of the leaked process) > > I cannot seems to be able to reproduce the issue manually, I only see it on production servers. > > Any help, or hint would be greatly appreciated. > > Thank you, > > Morgan. From max.lapshin@REDACTED Wed Jun 20 21:05:11 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Jun 2012 23:05:11 +0400 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: <1B73534E-FDA1-4934-BD06-734D1D1A3107@gmail.com> References: <1B73534E-FDA1-4934-BD06-734D1D1A3107@gmail.com> Message-ID: What was leaking? From msegalis@REDACTED Wed Jun 20 21:15:20 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Wed, 20 Jun 2012 21:15:20 +0200 Subject: [erlang-questions] how to seek Leaking processes ? In-Reply-To: References: <1B73534E-FDA1-4934-BD06-734D1D1A3107@gmail.com> Message-ID: <4B7DCFDF-E217-498B-A9B9-F5B400C72DAF@gmail.com> It was leaking when the ssl_accept failed? my gen_server wasn't terminated. I found out by connecting with telnet, when connected, it added me 2 processes (gen_server + SSL), after disconnect 1 processes was still alive (gen_server). Le 20 juin 2012 ? 21:05, Max Lapshin a ?crit : > What was leaking? From carlsson.richard@REDACTED Wed Jun 20 21:27:11 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 20 Jun 2012 22:27:11 +0300 Subject: [erlang-questions] Missing SASL Crash Report In-Reply-To: <4FE0CD90.6090507@eonblast.com> References: <4FE0CD90.6090507@eonblast.com> Message-ID: <4FE2240F.80603@gmail.com> Either simply that the timing is different (when you run all of it from the command line, your code may begin to execute before sasl is fully operational), or possibly because things executed with -s on the command line are running before the system boot sequence is completed, and it could be that sasl is not operational until booting is finished. /Richard On 06/19/2012 10:05 PM, H. Diedrich wrote: > Hi, > > why would I NOT get a CRASH REPORT listed out (although there is a > crash) in cases where I start a program directly, like > > $ erl -boot start_sasl -pa ./ebin ./lib/Emysql/ebin -s run run -s init > stop -noshell > > > -module(run). > -export([run/0]). > > run() -> > > crypto:start(), > emysql:start(), > mysrv:start(), > pong = mysrv:ping(), > pong = mysrv:ping(gen_server), > > io:format("start up and pings complete.~n"), > ok. > > > But if I do that in the shell, I get a CRASH REPORT (which is what I > want to find a bug)? > > $ erl -boot start_sasl -pa ./ebin ./lib/Emysql/ebin > > crypto:start(), emysql:start(), nbtsrv:start(). > > Thanks, > Henning > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From klacke@REDACTED Wed Jun 20 22:58:00 2012 From: klacke@REDACTED (Claes Wikstrom) Date: Wed, 20 Jun 2012 22:58:00 +0200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 Message-ID: <4FE23958.80401@hyber.org> I just posted the following note on the Yaws list, all of you using Yaws for production with cookie based auth need to take action. Actually, anyone using random:uniform/1 for anything security related need to pay attention. /klacke --------------- Folks, New yaws release which contains a fix to pretty serious security hole. The relevant relnote entry is: Use crypto:rand_bytes() instead of the cryptographically weak random module. Swedish security consultant and cryptographer Kalle Zetterlund discovered a way to - given a sequence of cookies produced by yaws_session_server - predict the next session id. Thus providing a gaping security hole into yaws servers that use the yaws_session_server to maintain cookie based HTTP sessions (klacke/kallez) It's been almost 6 months since the last release, so this one also contains a long series of good fixes and improvements from a lot of good people. Thanks everyone !! Code, release, relnotes, docs etc at http://yaws.hyber.org/ Yaws team - /klacke/Steve/Christopher From nem@REDACTED Wed Jun 20 23:10:08 2012 From: nem@REDACTED (Geoff Cant) Date: Wed, 20 Jun 2012 14:10:08 -0700 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE23958.80401@hyber.org> References: <4FE23958.80401@hyber.org> Message-ID: Hi Klake, Is the problem related to predictable seeding of random (set to {A,B,C} = erlang:now() at some point) or is it a bigger break in taking a series of outputs from random:uniform and working out the internal state from that? Just trying to figure out if kallez's attack is a brute force discovery of a weak seed, or if it's a more complete break of the generator itself given an unknown seed. Cheers, -Geoff On 2012-06-20, at 13:58 , Claes Wikstrom wrote: > > I just posted the following note on the Yaws list, all of you > using Yaws for production with cookie based auth need to take action. > Actually, anyone using random:uniform/1 for anything security related > need to pay attention. > > /klacke > > --------------- > > > Folks, > > New yaws release which contains a fix to pretty serious security hole. > The relevant relnote entry is: > > Use crypto:rand_bytes() instead of the cryptographically weak random module. Swedish security consultant and cryptographer Kalle Zetterlund discovered a way to - given a sequence of cookies produced by yaws_session_server - predict the next session id. Thus providing a gaping security hole into yaws servers that use the yaws_session_server to maintain cookie based HTTP sessions (klacke/kallez) > > > It's been almost 6 months since the last release, so this one also contains > a long series of good fixes and improvements from a lot of good people. > > Thanks everyone !! > > > Code, release, relnotes, docs etc at http://yaws.hyber.org/ > > Yaws team - > > /klacke/Steve/Christopher > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Geoff Cant From klacke@REDACTED Wed Jun 20 23:17:52 2012 From: klacke@REDACTED (Claes Wikstrom) Date: Wed, 20 Jun 2012 23:17:52 +0200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: References: <4FE23958.80401@hyber.org> Message-ID: <4FE23E00.8040401@hyber.org> On 06/20/2012 11:10 PM, Geoff Cant wrote: > Hi Klake, > > Is the problem related to predictable seeding of random (set to {A,B,C} = > erlang:now() at some point) or is it a bigger break in taking a series of > outputs from random:uniform and working out the internal state from that? > Just trying to figure out if kallez's attack is a brute force discovery of a > weak seed, or if it's a more complete break of the generator itself given an > unknown seed. > > Cheers, It's not, Yaws was using the seed as in {X,Y,Z} = seed(), ... seed() -> case (catch list_to_binary( os:cmd("dd if=/dev/urandom ibs=12 count=1 2>/dev/null"))) of <> -> {X, Y, Z}; _ -> now() end. The problem is much deeper, it's the random algorithm itself. It's said that it's cryptographically weak - now I've seen how weak. Very weak. /klacke From nem@REDACTED Wed Jun 20 23:37:44 2012 From: nem@REDACTED (Geoff Cant) Date: Wed, 20 Jun 2012 14:37:44 -0700 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE23E00.8040401@hyber.org> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> Message-ID: <3857FDFA-F93C-4D94-AD41-2E52641C42EA@erlang.geek.nz> On 2012-06-20, at 14:17 , Claes Wikstrom wrote: > On 06/20/2012 11:10 PM, Geoff Cant wrote: >> Hi Klake, >> >> Is the problem related to predictable seeding of random (set to {A,B,C} = >> erlang:now() at some point) or is it a bigger break in taking a series of >> outputs from random:uniform and working out the internal state from that? >> Just trying to figure out if kallez's attack is a brute force discovery of a >> weak seed, or if it's a more complete break of the generator itself given an >> unknown seed. >> >> Cheers, > > > It's not, Yaws was using the seed as in > > > {X,Y,Z} = seed(), > > ... > > > seed() -> > case (catch list_to_binary( > os:cmd("dd if=/dev/urandom ibs=12 count=1 2>/dev/null"))) of > <> -> > {X, Y, Z}; > _ -> > now() > end. > > > The problem is much deeper, it's the random algorithm itself. It's said that > it's cryptographically weak - now I've seen how weak. Very weak. That's pretty neat indeed then (as an attack, not so great for anyone using random:uniform for any crypto-ish purpose). I'd love to look at a description of the break if one becomes available. Cheers, -- Geoff Cant From clist@REDACTED Thu Jun 21 00:30:08 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Thu, 21 Jun 2012 00:30:08 +0200 Subject: [erlang-questions] simple_one_by_one trouble and rare start_link/2 gen_fsm Message-ID: <201206210030.08517.clist@uah.es> Hi, Please see example attached While calling a simple_one_by_one supervisor to start a gen fsm I found it causes calling a start_link/2 function on the callback module of the gen_fsm. I was not able to locate this start_link/2 on the docs and i dont know how Args are suppoused to be joined (those from the childspec on the supervisor and those from the start_child call. Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [QuickCheck] [PropEr] [Erl0MQ] Eshell V5.9.1 (abort with ^G) 1> test:main(). Starting simple_one_by_one group supervisor... [GROUP SUP] Ready to manage MuC wuth opts: [{option1,40}]'s Adding a child dynamically... start_link called with 2 arguments: [{option1,40}] and [{dynamic_option,43}] this is not documented!! init fsm with args: [] {ok,<0.34.0>} Im alive! Im alive! Im alive! Im alive! Im alive! Im alive! FSM terminated (shutdown) after a few iterations on state idle... 2> am I wrong or this is not documented (R15B1)? /Angel -------------- next part -------------- A non-text attachment was scrubbed... Name: group_supervisor.erl Type: text/x-erlang Size: 465 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sim_group_fsm.erl Type: text/x-erlang Size: 2336 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.erl Type: text/x-erlang Size: 375 bytes Desc: not available URL: From pablo.platt@REDACTED Thu Jun 21 00:45:00 2012 From: pablo.platt@REDACTED (Pablo Platt) Date: Wed, 20 Jun 2012 15:45:00 -0700 (PDT) Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <3857FDFA-F93C-4D94-AD41-2E52641C42EA@erlang.geek.nz> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <3857FDFA-F93C-4D94-AD41-2E52641C42EA@erlang.geek.nz> Message-ID: <1340232300.92382.YahooMailNeo@web112603.mail.gq1.yahoo.com> What are the alternatives? ________________________________ From: Geoff Cant To: Claes Wikstrom Cc: erlang-questions Sent: Thursday, June 21, 2012 12:37 AM Subject: Re: [erlang-questions] Yaws security alert - Yaws 1.93 On 2012-06-20, at 14:17 , Claes Wikstrom wrote: > On 06/20/2012 11:10 PM, Geoff Cant wrote: >> Hi Klake, >> >> Is the problem related to predictable seeding of random (set to {A,B,C} = >> erlang:now() at some point) or is it a bigger break in taking a series of >> outputs from random:uniform and working out the internal state from that? >> Just trying to figure out if kallez's attack is a brute force discovery of a >> weak seed, or if it's a more complete break of the generator itself given an >> unknown seed. >> >> Cheers, > > > It's not, Yaws was using the seed as in > > >? ? {X,Y,Z} = seed(), > > ... > > > seed() -> >? ? case (catch list_to_binary( >? ? ? ? ? ? ? ? ? os:cmd("dd if=/dev/urandom ibs=12 count=1 2>/dev/null"))) of >? ? ? ? <> -> >? ? ? ? ? ? {X, Y, Z}; >? ? ? ? _ -> >? ? ? ? ? ? now() >? ? end. > > > The problem is much deeper, it's the random algorithm itself. It's said that > it's cryptographically weak - now I've seen how weak. Very weak. That's pretty neat indeed then (as an attack, not so great for anyone using random:uniform for any crypto-ish purpose). I'd love to look at a description of the break if one becomes available. Cheers, -- Geoff Cant _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jun 21 02:00:11 2012 From: bob@REDACTED (Bob Ippolito) Date: Wed, 20 Jun 2012 20:00:11 -0400 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <1340232300.92382.YahooMailNeo@web112603.mail.gq1.yahoo.com> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <3857FDFA-F93C-4D94-AD41-2E52641C42EA@erlang.geek.nz> <1340232300.92382.YahooMailNeo@web112603.mail.gq1.yahoo.com> Message-ID: The random module is *very* weak, it has less than 48 bits of state (Wichmann-Hill 1982). It doesnt really generate results appropriate for double precision float, and it fails modern test suites for PRNGs, so it's basically unsuitable for most modern applications. Also, I haven't looked at Yaws' implementation but the random module only ensures that you have a good seed if you are using the process dictionary version of the API, otherwise you have to ensure that each component is non-zero and not an integer multiple of the prime for that component yourself. The best alternative is what this version appears to use: the crypto module. If you need something faster that doesn't have to be safe for cryptographic purposes you'll have to look outside of OTP. On Thursday, June 21, 2012, Pablo Platt wrote: > What are the alternatives? > > ------------------------------ > *From:* Geoff Cant 'nem@REDACTED');>> > *To:* Claes Wikstrom 'klacke@REDACTED');>> > *Cc:* erlang-questions 'cvml', 'erlang-questions@REDACTED');>> > *Sent:* Thursday, June 21, 2012 12:37 AM > *Subject:* Re: [erlang-questions] Yaws security alert - Yaws 1.93 > > > On 2012-06-20, at 14:17 , Claes Wikstrom wrote: > > > On 06/20/2012 11:10 PM, Geoff Cant wrote: > >> Hi Klake, > >> > >> Is the problem related to predictable seeding of random (set to {A,B,C} > = > >> erlang:now() at some point) or is it a bigger break in taking a series > of > >> outputs from random:uniform and working out the internal state from > that? > >> Just trying to figure out if kallez's attack is a brute force discovery > of a > >> weak seed, or if it's a more complete break of the generator itself > given an > >> unknown seed. > >> > >> Cheers, > > > > > > It's not, Yaws was using the seed as in > > > > > > {X,Y,Z} = seed(), > > > > ... > > > > > > seed() -> > > case (catch list_to_binary( > > os:cmd("dd if=/dev/urandom ibs=12 count=1 > 2>/dev/null"))) of > > <> -> > > {X, Y, Z}; > > _ -> > > now() > > end. > > > > > > The problem is much deeper, it's the random algorithm itself. It's said > that > > it's cryptographically weak - now I've seen how weak. Very weak. > > > That's pretty neat indeed then (as an attack, not so great for anyone > using random:uniform for any crypto-ish purpose). I'd love to look at a > description of the break if one becomes available. > > Cheers, > -- > Geoff Cant > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED 'erlang-questions@REDACTED');> > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jun 21 02:45:57 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 21 Jun 2012 12:45:57 +1200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE23E00.8040401@hyber.org> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> Message-ID: <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> On 21/06/2012, at 9:17 AM, Claes Wikstrom wrote: > The problem is much deeper, it's the random algorithm itself. It's said that > it's cryptographically weak - now I've seen how weak. Very weak. The algorithm is AS183, the Wichmann-Hill 3-cycle generator. It is antique, designed to cope with machines with limited arithmetic (like the Xerox D-machines XQP ran on), have a tolerably long period (in the days when 1MHz was fast), and serve the needs of simulations (small ones, by today's standards). It was *never* intended to be suitable for cryptography. Even the modern 4-cycle algorithm from the same authors has not the faintest claim to suitability for cryptography. From vances@REDACTED Thu Jun 21 06:22:54 2012 From: vances@REDACTED (Vance Shipley) Date: Thu, 21 Jun 2012 09:52:54 +0530 Subject: [erlang-questions] simple_one_by_one trouble and rare start_link/2 gen_fsm In-Reply-To: <201206210030.08517.clist@uah.es> References: <201206210030.08517.clist@uah.es> Message-ID: <20120621042238.GA15583@aluminum.wavenet.lk> On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez Miguel wrote: } While calling a simple_one_by_one supervisor to start a gen fsm I found } it causes calling a start_link/2 function on the callback module of the } gen_fsm. You are specifying that function to be called to start the worker yourself. In your module group_supervisor.erl you provide a child_spec() in the return value of init/1: {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, worker, [sim_group]} The form of which is: {Id, StartFunc, Restart, Shutdown, Type, Modules} The second argument, StartFunc, has the form: {Module, Function, Arguments} You provided: {sim_group_fsm, start_link, [Opts]} -- -Vance From mjtruog@REDACTED Thu Jun 21 09:06:08 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 21 Jun 2012 00:06:08 -0700 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <3857FDFA-F93C-4D94-AD41-2E52641C42EA@erlang.geek.nz> <1340232300.92382.YahooMailNeo@web112603.mail.gq1.yahoo.com> Message-ID: <4FE2C7E0.8070204@gmail.com> On 06/20/2012 05:00 PM, Bob Ippolito wrote: > The random module is *very* weak, it has less than 48 bits of state (Wichmann-Hill 1982). It doesnt really generate results appropriate for double precision float, and it fails modern test suites for PRNGs, so it's basically unsuitable for most modern applications. Also, I haven't looked at Yaws' implementation but the random module only ensures that you have a good seed if you are using the process dictionary version of the API, otherwise you have to ensure that each component is non-zero and not an integer multiple of the prime for that component yourself. > > The best alternative is what this version appears to use: the crypto module. If you need something faster that doesn't have to be safe for cryptographic purposes you'll have to look outside of OTP. If you are interested in an alternative to the random module which does not need to be safe for cryptographic purposes, there is an implementation of a newer algorithm done by the same authors (Wichmann-Hill 2006) which has an implementation here https://github.com/jj1bdx/sfmt-erlang/blob/master/src/random_wh06_int.erl . I believe this implementation is faithful to the original algorithm and avoids precision problems by leveraging Erlang's big integers support. However, I haven't gotten to providing tests for the algorithm yet, because I haven't needed it yet. A simple application, is a quicker way to do a v4 UUID (i.e., quicker than crypto), where you are not forced to call the random module multiple times (since random only provides 45 bits of pseudo-randomness, but the newer 2006 algorithm provides 124 bits of pseudo-randomness). So, if anyone is interested, that is a place to look if you need more pseudo-randomness for non-cryptographic purposes. - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From clist@REDACTED Thu Jun 21 09:30:15 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Thu, 21 Jun 2012 09:30:15 +0200 Subject: [erlang-questions] simple_one_by_one trouble and rare start_link/2 gen_fsm In-Reply-To: <20120621042238.GA15583@aluminum.wavenet.lk> References: <201206210030.08517.clist@uah.es> <20120621042238.GA15583@aluminum.wavenet.lk> Message-ID: <201206210930.15698.clist@uah.es> That's right... But start_link it is expected to be or arity 3 and Opts is a property list. but then on test:main/0 when i do supervisor:start_child(SupPid,Opts2) being Opts2 another property list, then suppervisor should call: apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 which in turn calls gen_fsm:start_link, thus providing a Pid in the end... but what it really does is apply(sim_group_fsm,start_link,Opts ++ Opts2,[ ]) and then crashes unless i declare a start_link/2 on my gen_fsm... { {badmatch, {error, {'EXIT', {undef, [ {sim_group_fsm,start_link,[[{option1,40}],[{option29,100}]],[]}, {supervisor,do_start_child_i,3,[ {file,"supervisor.erl"}, {line,319} ]}...... i dont see why i finish having a apply/4 call in supervisor: 319 instead of an apply/3... I want the sup to propagate one property list to children specs while i will provide another during children spawns providing that sup combines two in any way children will manage to demunge ...both property lists So cant just see what im doing wrong.... /angel PS: i changed a bit the test code to force sup crash On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez Miguel wrote: > } While calling a simple_one_by_one supervisor to start a gen fsm I found > } it causes calling a start_link/2 function on the callback module of the > } gen_fsm. > > You are specifying that function to be called to start the worker yourself. > In your module group_supervisor.erl you provide a child_spec() in the > return value of init/1: > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, worker, > [sim_group]} > > > The form of which is: > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > The second argument, StartFunc, has the form: > > {Module, Function, Arguments} > > You provided: > > {sim_group_fsm, start_link, [Opts]} -------------- next part -------------- A non-text attachment was scrubbed... Name: group_supervisor.erl Type: text/x-erlang Size: 528 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sim_group_fsm.erl Type: text/x-erlang Size: 2329 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.erl Type: text/x-erlang Size: 576 bytes Desc: not available URL: From klacke@REDACTED Thu Jun 21 10:03:20 2012 From: klacke@REDACTED (Claes Wikstrom) Date: Thu, 21 Jun 2012 10:03:20 +0200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> Message-ID: <4FE2D548.80909@hyber.org> On 06/21/2012 02:45 AM, Richard O'Keefe wrote: > > On 21/06/2012, at 9:17 AM, Claes Wikstrom wrote: >> The problem is much deeper, it's the random algorithm itself. It's said that >> it's cryptographically weak - now I've seen how weak. Very weak. > > The algorithm is AS183, the Wichmann-Hill 3-cycle generator. > It is antique, designed to cope with machines with limited arithmetic > (like the Xerox D-machines XQP ran on), have a tolerably long period > (in the days when 1MHz was fast), and serve the needs of simulations > (small ones, by today's standards). > > It was *never* intended to be suitable for cryptography. Indeed, that being said, I think there is quite a few Erlang applications out there that use the OTP random module, some, probably quite a few, of those applications use the random module in what could be considered a security related setting. It could be anything, the original author needed a random number, picked the random module, and now years later, it turns out that these random numbers are security related. Not good, a good solution would be to replace the current random module with a backwards compat implementation that use a better algorithm. /klacke From zerthurd@REDACTED Thu Jun 21 12:15:34 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 21 Jun 2012 17:15:34 +0700 Subject: [erlang-questions] Hexadecimal representation of binaries in shell Message-ID: Hello When I work with binary I cannot convert decimal representation of it to bitwise in the mind. It should be easier when binaries will displayed in hexadecimal representation. For example: <<45,3,243,5>> will displayed like <<16#2D, 16#03, 16#F3, 16#05>> Is it possible to enable this behaviour for whole pretty printing in shell? -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Thu Jun 21 12:47:46 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 21 Jun 2012 12:47:46 +0200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE2D548.80909@hyber.org> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> <4FE2D548.80909@hyber.org> Message-ID: On Thu, Jun 21, 2012 at 10:03 AM, Claes Wikstrom wrote: > On 06/21/2012 02:45 AM, Richard O'Keefe wrote: > > > > > > On 21/06/2012, at 9:17 AM, Claes Wikstrom wrote: > > > > > > The problem is much deeper, it's the random algorithm itself. > > > It's said that it's cryptographically weak - now I've seen how > > > weak. Very weak. > > > > > > The algorithm is AS183, the Wichmann-Hill 3-cycle generator. It is > > antique, designed to cope with machines with limited arithmetic > > (like the Xerox D-machines XQP ran on), have a tolerably long > > period (in the days when 1MHz was fast), and serve the needs of > > simulations (small ones, by today's standards). > > > > It was *never* intended to be suitable for cryptography. > > Indeed, that being said, I think there is quite a few Erlang > applications out there that use the OTP random module, some, > probably quite a few, of those applications use the random module in > what could be considered a security related setting. It could be > anything, the original author needed a random number, picked the > random module, and now years later, it turns out that ?these random > numbers are security related. > > Not good, a good solution would be to replace the current random > module with a backwards compat implementation that use a better > algorithm. It should probably be replaced with Kenji's sfmt-erlang or an implementation of (C)MWC. https://groups.google.com/group/comp.soft-sys.math.mathematica/msg/95a94c3b2aa5f077 From mononcqc@REDACTED Thu Jun 21 13:32:20 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 21 Jun 2012 07:32:20 -0400 Subject: [erlang-questions] Hexadecimal representation of binaries in shell In-Reply-To: References: Message-ID: <4FE30644.4010207@ferd.ca> There's this binary pretty printer library that could be useful to you: https://github.com/jtendo/binpp On Thu Jun 21 06:15:34 2012, Maxim Treskin wrote: > Hello > > When I work with binary I cannot convert decimal representation of it > to bitwise in the mind. It should be easier when binaries will > displayed in hexadecimal representation. > For example: > > <<45,3,243,5>> will displayed like <<16#2D, 16#03, 16#F3, 16#05>> > > Is it possible to enable this behaviour for whole pretty printing in > shell? > > -- > Max Treskin > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zerthurd@REDACTED Thu Jun 21 13:48:02 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 21 Jun 2012 18:48:02 +0700 Subject: [erlang-questions] Hexadecimal representation of binaries in shell In-Reply-To: <4FE30644.4010207@ferd.ca> References: <4FE30644.4010207@ferd.ca> Message-ID: Thank you, Fred! It is cool application. On 21 June 2012 18:32, Fred Hebert wrote: > There's this binary pretty printer library that could be useful to you: > https://github.com/jtendo/**binpp > > > On Thu Jun 21 06:15:34 2012, Maxim Treskin wrote: > >> Hello >> >> When I work with binary I cannot convert decimal representation of it >> to bitwise in the mind. It should be easier when binaries will >> displayed in hexadecimal representation. >> For example: >> >> <<45,3,243,5>> will displayed like <<16#2D, 16#03, 16#F3, 16#05>> >> >> Is it possible to enable this behaviour for whole pretty printing in >> shell? >> >> -- >> Max Treskin >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Thu Jun 21 15:47:08 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Thu, 21 Jun 2012 15:47:08 +0200 Subject: [erlang-questions] EUnit so freakishly silent. Talk to me. Maybe? Message-ID: This is all I am getting out of EUnit on one of a series of test exeuctions: undefined How does one decipher this? I understand that something was called with some arguments that do not match some function in the thread of execution. But where are my stack traces that led to this? I am actually fed up with EUnit enough that I am on a verge of dumping it. Can anyone suggest a better tool for unit test (not integration, not functional - I just want my unit testing tool that I do not have to strangle for information)? Thanks, Alex. From vances@REDACTED Thu Jun 21 16:20:37 2012 From: vances@REDACTED (Vance Shipley) Date: Thu, 21 Jun 2012 19:50:37 +0530 Subject: [erlang-questions] JavaScript compiler for Erlang Message-ID: <20120621142036.GE16243@aluminum.wavenet.lk> There was a thread by this name back in 2007. Has there been any progress on a beam targeted Javascipt compiler? -- -Vance From ingo.jaeckel@REDACTED Thu Jun 21 17:12:26 2012 From: ingo.jaeckel@REDACTED (Ingo Jaeckel) Date: Thu, 21 Jun 2012 11:12:26 -0400 Subject: [erlang-questions] JavaScript compiler for Erlang In-Reply-To: <20120621142036.GE16243@aluminum.wavenet.lk> References: <20120621142036.GE16243@aluminum.wavenet.lk> Message-ID: Just out of curiosity: Do you want to compile JavaScript code to Erlang or Erlang to JavaScript? In either case - Why? What is the application behind this? What about https://github.com/beamjs/erlv8/ and http://beamjs.org/? Ingo On Thu, Jun 21, 2012 at 10:20 AM, Vance Shipley wrote: > There was a thread by this name back in 2007. ?Has there been any > progress on a beam targeted Javascipt compiler? > > -- > ? ? ? ?-Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anthonym@REDACTED Thu Jun 21 19:27:22 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 21 Jun 2012 10:27:22 -0700 Subject: [erlang-questions] Dialyzer and ets:fun2ms In-Reply-To: <38176269-3A72-48FB-96F4-47F8AC19EB76@erlang.geek.nz> References: <38176269-3A72-48FB-96F4-47F8AC19EB76@erlang.geek.nz> Message-ID: <20120621172722.GA22846@alumni.caltech.edu> I came across this same issue yesterday, anyone from the dialyzer team want to comment on a better way to deal with records in matchspecs? Thanks, -Anthony On Tue, May 22, 2012 at 01:39:10PM -0700, Geoff Cant wrote: > Hi all, I have hit this problem with my code and dialyzer a few times now and wonder what everyone else does to avoid it. > > I have record definitions with type specifications as follows: > > > -module(rec_example). > > > > -export([find_by_colour/1]). > > > > -include_lib("stdlib/include/ms_transform.hrl"). > > > > -record(myrecord, {field = 0 :: integer(), > > roses = red :: 'red' | 'blue'}). > > > > I then add some functions that use ets matchspecs or a fun2ms transform and have to fill in fields with dummy values as required by the API (e.g. '_' and '$1'): > > > find_by_colour(Colour) when is_atom(Colour) -> > > ets:lookup(myrecords, ets:fun2ms(fun (#myrecord{roses=C}) > > when C =:= Colour -> > > object() > > end)). > > > Dialyzer complains that this function can't succeed: > > > rec_example.erl:10: Function find_by_colour/1 has no local return > > rec_example.erl:11: Record construction #myrecord{field::'_',roses::'$1'} violates the declared type of field field::integer() and roses::'blue' | 'red' > > > > To me this seems like reasonable code to want to write, but dialyzer hates it. The two (terrible) solutions I've found to avoid the dialyzer warnings are to add the dummy values to the record type spec: > > > -record(myrecord, {field = 0 :: integer() | '_', > > roses = red :: 'red' | 'blue' | '$1'}). > > > Or to hide the record construction from dialyzer by constructing the matchspec using functions. First run fun2ms manually to get: > > > [{#myrecord{field = '_',roses = '$1'}, > > [{'=:=','$1',Colour}], > > ['$_']}] > > > Which still complains, so you then do something far worse: > > > [{list_to_tuple([myrecord, '_', '$1']), > > [{'=:=','$1',Colour}], > > ['$_']}] > > > Clearly none of these options are good - what's a better way to have record type specs and matchspecs and no dialyzer warnings? Or is there something I'm missing that replaces my use of matchspecs and avoids the problem? > > > Cheers, > -- > Geoff Cant > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From freeakk@REDACTED Thu Jun 21 19:28:12 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 21 Jun 2012 21:28:12 +0400 Subject: [erlang-questions] EUnit so freakishly silent. Talk to me. Maybe? In-Reply-To: References: Message-ID: This bug is because eunit cannot print the new version of stacktrace (with line and file positions). http://stackoverflow.com/questions/10509238/not-output-exception-stack-trace-in-eunit Here it was fixed: https://github.com/richcarl/eunit From attila.r.nohl@REDACTED Thu Jun 21 19:51:24 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 21 Jun 2012 19:51:24 +0200 Subject: [erlang-questions] common_test vs supervised process Message-ID: Hello! I try to test our code with common_test (from the OTP master branch). This code starts a supervisor, which starts a process. For some reason the test fails with: *** CT Error Notification 2012-06-21 19:38:59.236 *** Error detected: shutdown even though the started process doesn't crash (or at least I don't see it). I'm probably making something really stupid, do you have any idea what? Here's the important part of the code. The testcase (the init_* functions are empty): start_stop_test(_Config) -> {ok, Pid} = inets:start(ftpd, [{port, 2021}]), inets:stop(ftpd, Pid). The ftpd module: start_service(Config) -> ftpd_sup:start_link(Config). stop_service(Pid) when is_pid(Pid) -> case ftpd_sup:stop(Pid) of true -> ok; R -> { error, R } end. The ftpd_sup module: start_link(Args) -> supervisor:start_link(ftpd_sup, Args). stop(Pid) -> exit(Pid,shutdown), true. init(Args) -> NewArgs = [ {sup_pid, self()} | Args], {ok, {{one_for_one, 10, 60}, [{ftpd_listener, {ftpd_listener, start_link, [NewArgs]}, permanent, brutal_kill, worker, [ftpd_listener]}]}}. And the ftpd_listener: start_link(Args) -> proc_lib:start_link(?MODULE, listener_init, [self(), Args]). listener_init(Parent, Args) -> proc_lib:init_ack(Parent, {ok, self()}), Port = proplists:get_value(port,Args), SupPid = proplists:get_value(sup_pid,Args), {ok, LSock} = gen_tcp:listen(Port, [binary, {packet, 0}, {active, false}]), loop(LSock, SupPid). loop(LSock, SupPid) -> case gen_tcp:accept(LSock) of {ok, Sock} -> loop(LSock, SupPid); {_, Res} -> ok end. From kostis@REDACTED Thu Jun 21 20:18:20 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 21 Jun 2012 20:18:20 +0200 Subject: [erlang-questions] Dialyzer and ets:fun2ms In-Reply-To: <20120621172722.GA22846@alumni.caltech.edu> References: <38176269-3A72-48FB-96F4-47F8AC19EB76@erlang.geek.nz> <20120621172722.GA22846@alumni.caltech.edu> Message-ID: <4FE3656C.6010609@cs.ntua.gr> On 06/21/2012 07:27 PM, Anthony Molinaro wrote: > I came across this same issue yesterday, anyone from the dialyzer team want > to comment on a better way to deal with records in matchspecs? Currently, there is really good way to deal with this problem.... (However note that the problem appears *only* if you type records.) In my opinion, whoever added match specs in the language clearly was not thinking straight. They overloaded some random terms, namely the atoms '_', '$1', '$2', ... which, although admittedly quite uncommon, are perfectly valid Erlang atoms that applications can freely use if they want to to mean something completely different than they do in matchspecs. Sooner or later, this was bound to surface as a problem... Nowhere else in the language is there specified that these atoms cannot be used by applications as regular atoms. Consequently, dialyzer does not (and will not!) treat them specially. In my opinion, there should be a DSL for matchspecs that does not clash with the rest of the Erlang language. In the meantime, the best "solution" I see to this problem is to define the following type: -type matchspec_atom() :: '_' | '$1' | ... | '$42'. and use it in whatever record field names are used in matchspecs in your code. I.e.: -record(myrecord, {field = 0 :: integer() | matchspec_atom(), roses = red :: 'red' | 'blue' | matchspec_atom()}). Hope this helps, Kostis > On Tue, May 22, 2012 at 01:39:10PM -0700, Geoff Cant wrote: >> Hi all, I have hit this problem with my code and dialyzer a few times now and wonder what everyone else does to avoid it. >> >> I have record definitions with type specifications as follows: >> >>> -module(rec_example). >>> >>> -export([find_by_colour/1]). >>> >>> -include_lib("stdlib/include/ms_transform.hrl"). >>> >>> -record(myrecord, {field = 0 :: integer(), >>> roses = red :: 'red' | 'blue'}). >>> >> >> I then add some functions that use ets matchspecs or a fun2ms transform and have to fill in fields with dummy values as required by the API (e.g. '_' and '$1'): >> >>> find_by_colour(Colour) when is_atom(Colour) -> >>> ets:lookup(myrecords, ets:fun2ms(fun (#myrecord{roses=C}) >>> when C =:= Colour -> >>> object() >>> end)). >> >> >> Dialyzer complains that this function can't succeed: >> >>> rec_example.erl:10: Function find_by_colour/1 has no local return >>> rec_example.erl:11: Record construction #myrecord{field::'_',roses::'$1'} violates the declared type of field field::integer() and roses::'blue' | 'red' >> >> >> >> To me this seems like reasonable code to want to write, but dialyzer hates it. The two (terrible) solutions I've found to avoid the dialyzer warnings are to add the dummy values to the record type spec: >> >>> -record(myrecord, {field = 0 :: integer() | '_', >>> roses = red :: 'red' | 'blue' | '$1'}). >> >> >> Or to hide the record construction from dialyzer by constructing the matchspec using functions. First run fun2ms manually to get: >> >>> [{#myrecord{field = '_',roses = '$1'}, >>> [{'=:=','$1',Colour}], >>> ['$_']}] >> >> >> Which still complains, so you then do something far worse: >> >>> [{list_to_tuple([myrecord, '_', '$1']), >>> [{'=:=','$1',Colour}], >>> ['$_']}] >> >> >> Clearly none of these options are good - what's a better way to have record type specs and matchspecs and no dialyzer warnings? Or is there something I'm missing that replaces my use of matchspecs and avoids the problem? >> >> >> Cheers, >> -- >> Geoff Cant >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From kostis@REDACTED Thu Jun 21 20:28:58 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 21 Jun 2012 20:28:58 +0200 Subject: [erlang-questions] Dialyzer and ets:fun2ms In-Reply-To: <4FE3656C.6010609@cs.ntua.gr> References: <38176269-3A72-48FB-96F4-47F8AC19EB76@erlang.geek.nz> <20120621172722.GA22846@alumni.caltech.edu> <4FE3656C.6010609@cs.ntua.gr> Message-ID: <4FE367EA.6070507@cs.ntua.gr> On 06/21/2012 08:18 PM, Kostis Sagonas wrote: > On 06/21/2012 07:27 PM, Anthony Molinaro wrote: >> I came across this same issue yesterday, anyone from the dialyzer team >> want >> to comment on a better way to deal with records in matchspecs? > > Currently, there is really good way to deal with this problem.... I obviously meant "Currently, there is really NO good way to ..." in the above. Kostis From erisa85d@REDACTED Thu Jun 21 20:37:44 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Thu, 21 Jun 2012 20:37:44 +0200 Subject: [erlang-questions] SMP for IO-bound applications Message-ID: Hi, As part of my studies, I have recently been doing some performance evaluations on Erlang SMP improvements for IO-bound applications. The applications I considered for the evaluation were Emysql driver ( https://github.com/Eonblast/Emysql) and Mnesia. I have created an Erlang module for performing reads/writes from/to Mysql DB through Emysql driver, and another Erlang module for communicating with Mnesia. Both the modules have two methods write(nr_records) which writes nr_records records to the database (MySQL, or Mnesia), and read(nr_records) which reads nr_records random records from the db. (both the reads and writes are consecutive, not in chunks) I have a load generator tool which simulates simultaneous requests to both the modules and gathers performance statistics. I generate as much load as needed to stress the applications (near the saturation point) The metrics I uses are the throughput (no. of sessions/sec), and the session duration. A session is a call to write(nr_records) or read(nr_records) function depending on the test case (read or write). So if I call write(500), I measure the duration of a session that does 500 hundred inserts into the database. The tests I run have these parameters: A)Type of operations: 1- Reads 2-Writes I have just one table with 20 fields, and the reads and writes are just select and insert operations in that table B) I/O applications: 1- Emysql 2-Mnesia C) SMP parameters: 1- SMP enabled, no. of schedulers = no. of cpu cores 2-SMP enabled, no. of schedulers = 1 3- SMP disabled D) Erlang OTP versions: 1- Erlang R12B (The SMP has only one run-queue and multiple schedulers) 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) The tests included all possible combinations from A,B,C,D. They were run in Solaris 10 x86 (a 16 cores machine). In general these were the results I got: *Emysql driver:* 1- There is a a big difference in performance between SMP enabled and disabled in both the Erlang versions (R12 and R15) for both read and write tests. So I can say that SMP rocks! However, you have to have enough load to achieve that (for low load I could not see any difference, sometimes it was even better to disable SMP) 2- I was expecting a much better performance for SMP enabled (no. of cores = no. of schedulers) in R15 than in R12, since the schedulers' logic has changed a lot from R12 to R15. But the results were more or less the same. I was thinking since I had a multithreaded db like MySql, a multiple db-connections driver as Emysql, and SMP with multiple schedulers with their own run queues, I could get better results than in R12 where there is only one run-queue and multiple schedulers (more lock contention). Is it maybe because the processes are just doing IO and nothing CPU-intensive? 3- I realized that CPU usage in R15 SMP enabled is twice higher than the CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, and I can see that the beam process is using up to 75-80% of the CPU, the rest is the mysql daemon process. Is it because in R15 the scheduling algorithm has become more complex? *Mnesia* * * I have only one table (as I said before) created with the attribute disc_copies, and it is stored only in one node. The whole table fits in RAM, and I tried to keep it simple by controlling the size of the table during my tests, in order not exceed RAM capacity 4- For the write tests in Mnesia I see the same behavior as in Mysql. However, since the data are all loaded in RAM, the session duration is shorter, and the gain in performance between SMP enabled and disabled is not as huge as in Mysql. 5- For the read tests, I have two observations. First, the dirty reads are 10 times faster than the reads that use transactions. Second, I have better results for SMP disabled (twice faster, though in both cases the session duration is in order of millisecods, 70 msec vs 30 msec). 6- Same as in Mysql, no difference in performance between R12 and R15 with SMP enabled, for both read and write tests. If you have read so far, and have a comment on my results, you are welcome. Especially about the no difference btw R12 and R15 and the high CPU usage in R15 when the applications are IO-bound. Thanks, Erisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From erisa85d@REDACTED Thu Jun 21 21:28:13 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Thu, 21 Jun 2012 21:28:13 +0200 Subject: [erlang-questions] SMP for IO-bound applications In-Reply-To: References: Message-ID: One additonal comment I forgot to mention: For all Mnesia tests, the CPU usage is low in general. Only for the transactional read test in R15 with SMP enabled i get a CPU usage of 70%. >From the load generator tool, i call the read(N) function that is showed below: exec_read(0) -> done_reading; exec_read(N) -> Id=random:uniform(7000000), %%io:format("~p~n",[Id]), %%[_Row] = mnesia:dirty_read({subscriber,Id}), Fun = fun() -> mnesia:read({subscriber,Id}) end, {atomic,[Row]}=mnesia:transaction(Fun), %io:format("~p~n",[Row]), exec_read(N-1). read(N) -> {A1,A2,A3} = now(), random:seed(A1, A2, A3), exec_read(N). I am using mpstat while my tests are running and beside the CPU usage, another difference of this test from the other Mnesia tests is that the syscl (system calls) column of mpstat has a value which is 20 times higher than the other test cases. On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi wrote: > Hi, > As part of my studies, I have recently been doing some performance > evaluations on Erlang SMP improvements for IO-bound applications. > > The applications I considered for the evaluation were Emysql driver ( > https://github.com/Eonblast/Emysql) and Mnesia. > I have created an Erlang module for performing reads/writes from/to Mysql > DB through Emysql driver, and another Erlang module for communicating with > Mnesia. > Both the modules have two methods write(nr_records) which writes > nr_records records to the database (MySQL, or Mnesia), and read(nr_records) > which reads nr_records random records from the db. (both the reads and > writes are consecutive, not in chunks) > I have a load generator tool which simulates simultaneous requests to both > the modules and gathers performance statistics. I generate as much load as > needed to stress the applications (near the saturation point) > The metrics I uses are the throughput (no. of sessions/sec), and the > session duration. A session is a call to write(nr_records) > or read(nr_records) function depending on the test case (read or write). So > if I call write(500), I measure the duration of a session that does 500 > hundred inserts into the database. > > The tests I run have these parameters: > > A)Type of operations: 1- Reads 2-Writes > I have just one table with 20 fields, and the reads and writes are just > select and insert operations in that table > > B) I/O applications: 1- Emysql 2-Mnesia > > C) SMP parameters: > 1- SMP enabled, no. of schedulers = no. of cpu cores > 2-SMP enabled, no. of schedulers = 1 > 3- SMP disabled > > D) Erlang OTP versions: > 1- Erlang R12B (The SMP has only one run-queue and multiple schedulers) > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) > > The tests included all possible combinations from A,B,C,D. They were run > in Solaris 10 x86 (a 16 cores machine). > In general these were the results I got: > > *Emysql driver:* > 1- There is a a big difference in performance between SMP enabled and > disabled in both the Erlang versions (R12 and R15) for both read and write > tests. So I can say that SMP rocks! However, you have to have enough load > to achieve that (for low load I could not see any difference, sometimes it > was even better to disable SMP) > > 2- I was expecting a much better performance for SMP enabled (no. of cores > = no. of schedulers) in R15 than in R12, since the schedulers' logic has > changed a lot from R12 to R15. But the results were more or less the same. > I was thinking since I had a multithreaded db like MySql, a multiple > db-connections driver as Emysql, and SMP with multiple schedulers with > their own run queues, I could get better results than in R12 where there is > only one run-queue and multiple schedulers (more lock contention). > Is it maybe because the processes are just doing IO and nothing > CPU-intensive? > > 3- I realized that CPU usage in R15 SMP enabled is twice higher than the > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, and I > can see that the beam process is using up to 75-80% of the CPU, the rest is > the mysql daemon process. Is it because in R15 the scheduling algorithm has > become more complex? > > *Mnesia* > * > * > I have only one table (as I said before) created with the attribute > disc_copies, and it is stored only in one node. The whole table fits in > RAM, and I tried to keep it simple by controlling the size of the table > during my tests, in order not exceed RAM capacity > > 4- For the write tests in Mnesia I see the same behavior as in Mysql. > However, since the data are all loaded in RAM, the session duration is > shorter, and the gain in performance between SMP enabled and disabled is > not as huge as in Mysql. > > 5- For the read tests, I have two observations. First, the dirty reads are > 10 times faster than the reads that use transactions. Second, I have better > results for SMP disabled (twice faster, though in both cases the session > duration is in order of millisecods, 70 msec vs 30 msec). > > 6- Same as in Mysql, no difference in performance between R12 and R15 with > SMP enabled, for both read and write tests. > > If you have read so far, and have a comment on my results, you are > welcome. Especially about the no difference btw R12 and R15 and the high > CPU usage in R15 when the applications are IO-bound. > > Thanks, > Erisa > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jun 22 02:21:21 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jun 2012 12:21:21 +1200 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE2D548.80909@hyber.org> References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> <4FE2D548.80909@hyber.org> Message-ID: <729C3238-B51B-4EBB-8E7E-014BA8F71FB9@cs.otago.ac.nz> On 21/06/2012, at 8:03 PM, Claes Wikstrom wrote: > Indeed, that being said, I think there is quite a few Erlang applications > out there that use the OTP random module, some, probably quite a few, of > those applications use the random module in what could be considered > a security related setting. It could be anything, the original author > needed a random number, picked the random module, and now years later, it turns out that these random numbers are security related. > > Not good, a good solution would be to replace the current random module with > a backwards compat implementation that use a better algorithm. Was it the Erlang thread or the SWI Prolog thread where we had a lengthy discussion about this not so long ago? I remember writing a version of the 4-cycle algorithm for one of those groups. Java makes a clear distinction between java.util.Random and java.Security.SecureRandom, and for good reason. A pseudo-random number generator to be used for generating test cases and doing simulations has *different* quality goals from a prng to be used for security applications. There are plenty of better algorithms for the first purpose; the Web is full of them. But they *still* are not going to be trustworthy for cryptographic purposes, and it is *still* going to be a mistake for anyone to use them so. We need a secure_random module, and I leave designing that to people who know something about the area. From jinni.park@REDACTED Fri Jun 22 02:40:22 2012 From: jinni.park@REDACTED (Park, Sungjin) Date: Fri, 22 Jun 2012 09:40:22 +0900 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <4FE23958.80401@hyber.org> References: <4FE23958.80401@hyber.org> Message-ID: Thanks for invaluable info. Anyways, what would be an alternative for random:uniform/1? Or is there any patch for the problem? /Sungjin On Thu, Jun 21, 2012 at 5:58 AM, Claes Wikstrom wrote: > > I just posted the following note on the Yaws list, all of you > using Yaws for production with cookie based auth need to take action. > Actually, anyone using random:uniform/1 for anything security related > need to pay attention. > > /klacke > > --------------- > > > Folks, > > New yaws release which contains a fix to pretty serious security hole. > The relevant relnote entry is: > > Use crypto:rand_bytes() instead of the cryptographically weak random > module. Swedish security consultant and cryptographer Kalle Zetterlund > discovered a way to - given a sequence of cookies produced by > yaws_session_server - predict the next session id. Thus providing a gaping > security hole into yaws servers that use the yaws_session_server to > maintain cookie based HTTP sessions (klacke/kallez) > > > It's been almost 6 months since the last release, so this one also contains > a long series of good fixes and improvements from a lot of good people. > > Thanks everyone !! > > > Code, release, relnotes, docs etc at http://yaws.hyber.org/ > > Yaws team - > > /klacke/Steve/Christopher > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Fri Jun 22 08:09:28 2012 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Fri, 22 Jun 2012 09:09:28 +0300 Subject: [erlang-questions] EUnit so freakishly silent. Talk to me. Maybe? In-Reply-To: References: Message-ID: <4FE40C18.30806@gmail.com> On 2012.06.21 16:47, Alex Shneyderman wrote: > This is all I am getting out of EUnit on one of a series of test > exeuctions: > > undefined I've noticed that this is the return value if your test times out, although it usually prints *timed out* as well, as far as I remember. > How does one decipher this? I understand that something was called > with some arguments that do not match some function in the thread of > execution. But where are my stack traces that led to this? Not necessarily, that would usually be badarg or function_clause or *undef*[1]. > I am actually fed up with EUnit enough that I am on a verge of > dumping it. Can anyone suggest a better tool for unit test (not > integration, not functional - I just want my unit testing tool that I > do not have to strangle for information)? EUnit is actually really good, don't give up on it :-) As Michael suggested you can either build Erlang with the patched EUnit version, or you can try using CT, but I don't think you'll find the transition very pleasant. Ignas [1]: http://www.erlang.org/doc/reference_manual/errors.html#id81672 From be.dmitry@REDACTED Fri Jun 22 10:22:37 2012 From: be.dmitry@REDACTED (=?utf-8?B?YmUuZG1pdHJ5QGdtYWlsLmNvbQ==?=) Date: Fri, 22 Jun 2012 12:22:37 +0400 Subject: [erlang-questions] =?utf-8?q?SMP_for_IO-bound_applications?= Message-ID: <4fe42c05.7067980a.1893.6fb7@mx.google.com> I think that much of CPU usage is created by your MySQL driver. You should have a look at driver details. There may be copying of the query you send to MySQL between your benchmarking process and the real process that owns the socket. The same with result - parsing response and sending result back to benchmarking process. There may be a bottleneck in choosing MySQL connection from connection pool if it is made through supervisor. Try to use etop to see what functions are at the bottlenecks. I mean your io-intensive test may be just the test showing driver bottlenecks and not the problems in smp. ----- Reply message ----- From: "Erisa Dervishi" To: Subject: [erlang-questions] SMP for IO-bound applications Date: Thu, Jun 21, 2012 23:28 One additonal comment I forgot to mention: For all Mnesia tests, the CPU usage is low in general. Only for the transactional read test in R15 with SMP enabled i get a CPU usage of 70%. From the load generator tool, i call the read(N) function that is showed below: exec_read(0) -> done_reading; exec_read(N) -> Id=random:uniform(7000000), %%io:format("~p~n",[Id]), %%[_Row] = mnesia:dirty_read({subscriber,Id}), Fun = fun() -> mnesia:read({subscriber,Id}) end, {atomic,[Row]}=mnesia:transaction(Fun), %io:format("~p~n",[Row]), exec_read(N-1). read(N) -> {A1,A2,A3} = now(), random:seed(A1, A2, A3), exec_read(N). I am using mpstat while my tests are running and beside the CPU usage, another difference of this test from the other Mnesia tests is that the syscl (system calls) column of mpstat has a value which is 20 times higher than the other test cases. On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi wrote: > Hi, > As part of my studies, I have recently been doing some performance > evaluations on Erlang SMP improvements for IO-bound applications. > > The applications I considered for the evaluation were Emysql driver ( > https://github.com/Eonblast/Emysql) and Mnesia. > I have created an Erlang module for performing reads/writes from/to Mysql > DB through Emysql driver, and another Erlang module for communicating with > Mnesia. > Both the modules have two methods write(nr_records) which writes > nr_records records to the database (MySQL, or Mnesia), and read(nr_records) > which reads nr_records random records from the db. (both the reads and > writes are consecutive, not in chunks) > I have a load generator tool which simulates simultaneous requests to both > the modules and gathers performance statistics. I generate as much load as > needed to stress the applications (near the saturation point) > The metrics I uses are the throughput (no. of sessions/sec), and the > session duration. A session is a call to write(nr_records) > or read(nr_records) function depending on the test case (read or write). So > if I call write(500), I measure the duration of a session that does 500 > hundred inserts into the database. > > The tests I run have these parameters: > > A)Type of operations: 1- Reads 2-Writes > I have just one table with 20 fields, and the reads and writes are just > select and insert operations in that table > > B) I/O applications: 1- Emysql 2-Mnesia > > C) SMP parameters: > 1- SMP enabled, no. of schedulers = no. of cpu cores > 2-SMP enabled, no. of schedulers = 1 > 3- SMP disabled > > D) Erlang OTP versions: > 1- Erlang R12B (The SMP has only one run-queue and multiple schedulers) > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) > > The tests included all possible combinations from A,B,C,D. They were run > in Solaris 10 x86 (a 16 cores machine). > In general these were the results I got: > > *Emysql driver:* > 1- There is a a big difference in performance between SMP enabled and > disabled in both the Erlang versions (R12 and R15) for both read and write > tests. So I can say that SMP rocks! However, you have to have enough load > to achieve that (for low load I could not see any difference, sometimes it > was even better to disable SMP) > > 2- I was expecting a much better performance for SMP enabled (no. of cores > = no. of schedulers) in R15 than in R12, since the schedulers' logic has > changed a lot from R12 to R15. But the results were more or less the same. > I was thinking since I had a multithreaded db like MySql, a multiple > db-connections driver as Emysql, and SMP with multiple schedulers with > their own run queues, I could get better results than in R12 where there is > only one run-queue and multiple schedulers (more lock contention). > Is it maybe because the processes are just doing IO and nothing > CPU-intensive? > > 3- I realized that CPU usage in R15 SMP enabled is twice higher than the > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, and I > can see that the beam process is using up to 75-80% of the CPU, the rest is > the mysql daemon process. Is it because in R15 the scheduling algorithm has > become more complex? > > *Mnesia* > * > * > I have only one table (as I said before) created with the attribute > disc_copies, and it is stored only in one node. The whole table fits in > RAM, and I tried to keep it simple by controlling the size of the table > during my tests, in order not exceed RAM capacity > > 4- For the write tests in Mnesia I see the same behavior as in Mysql. > However, since the data are all loaded in RAM, the session duration is > shorter, and the gain in performance between SMP enabled and disabled is > not as huge as in Mysql. > > 5- For the read tests, I have two observations. First, the dirty reads are > 10 times faster than the reads that use transactions. Second, I have better > results for SMP disabled (twice faster, though in both cases the session > duration is in order of millisecods, 70 msec vs 30 msec). > > 6- Same as in Mysql, no difference in performance between R12 and R15 with > SMP enabled, for both read and write tests. > > If you have read so far, and have a comment on my results, you are > welcome. Especially about the no difference btw R12 and R15 and the high > CPU usage in R15 when the applications are IO-bound. > > Thanks, > Erisa > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clist@REDACTED Fri Jun 22 11:04:38 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Fri, 22 Jun 2012 11:04:38 +0200 Subject: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 In-Reply-To: <201206210930.15698.clist@uah.es> References: <201206210030.08517.clist@uah.es> <20120621042238.GA15583@aluminum.wavenet.lk> <201206210930.15698.clist@uah.es> Message-ID: <201206221104.38808.clist@uah.es> i did'nt find any example of mixing args from the supervirsor childspec and start_child(...) so im going to make my own digging in this issue: Let see How the simple_one_by_one works... handle_call({start_child, EArgs}, _From, State) when ?is_simple(State) -> Child = hd(State#state.children), #child{mfargs = {M, F, A}} = Child, Args = A ++ EArgs, case do_start_child_i(M, F, Args) of <------>{ok, undefined} when Child#child.restart_type =:= temporary -> <------> {reply, {ok, undefined}, State}; <------>{ok, Pid} -> <------> NState = save_dynamic_child(Child#child.restart_type, Pid, Args, State), <------> {reply, {ok, Pid}, NState}; <------>{ok, Pid, Extra} -> <------> NState = save_dynamic_child(Child#child.restart_type, Pid, Args, State), <------> {reply, {ok, Pid, Extra}, NState}; <------>What -> <------> {reply, What, State} end; Shouldnt be do_start_child_i(M,F,EArgs) of the form : do_start_child(M,F,[EArgs]) ?? or the inner call apply(M,F,[A])? Provided the docs state that your unique exported funcion must be of arity one you should expect that code will take measures in order to enforce arity one, enclosing whatever you pass it in a List on the apply phase. The semanctics of apply use a list to KNOW how many args the target call has, while the semantics of sup childspec + sup start_child use a list ++ op to know how many args will be passed to your callback of arity ONE. So for the apply part all your functions are arity ONE Without the ability to join Args from the ChildSpec and from the start child call i would concur that you should enclose in a list whatever you want to be as a sole argument and this code be ok, but as soon as you can put two or more args one on the ChildSpec and one on the call Sup code must enforce that only one arg APPLY call is made so you end calling start_link/1 whatever you pass on the args... Even the DOCS state "[ term() ]" as the MFA on the ChildSpec My stdlib-1.18.1.pdf says (Pag 369) "start_child(SupRef, ChildSpec) -> startchild_ret() Types: SupRef = sup_ref() ChildSpec = child_spec() | (List :: [term()]) ... ... ...If the case of a simple_one_for_one supervisor, the child specification defined in Module:init/1 will be used and ChildSpec should instead be an arbitrary list of terms List. The child process will then be started by appending List to the existing start function arguments, i.e. by calling apply(M, F, A++List) where {M,F,A} is the start function defined in the child specification." Here IMHO lies the error, as A is a list we have [any()...] ++ [any()...] is a list [any()...,any()...] and thats get us an apply call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} and you try provide a [..] in your start_child it will fail unless both argslists are void. So a combination of: init(Opts) -> io:format("[GROUP SUP] Ready to manage MuC with opts: ~p\'s\n",[Opts]), {ok, { {simple_one_for_one, 1, 60}, [ {mucfsm, {sim_group_fsm, start_link, [arg1,arg2]}, transient, 60, worker, [sim_group]} ] } }. and a client that makes a call like: ... {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), ... Gets you a final apply call {M,F,A} where A is in the form [arg1,arg2,arg3,arg4] not [[arg1,arg2,argf3,arg4]] at is should be when you intent to call a start_link/1.... Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [QuickCheck] [PropEr] [Erl0MQ] Starting simple_one_by_one group supervisor... [GROUP SUP] staring with argument: [{option1,40}] Eshell V5.9.1 (abort with ^G) 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s Adding a child dynamically... {"init terminating in do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.erl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{line,344}]}, {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]}]}} /Angel On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel escribi?: > That's right... > > But start_link it is expected to be or arity 3 and Opts is a property list. > > but then on test:main/0 when i do supervisor:start_child(SupPid,Opts2) > > being Opts2 another property list, then suppervisor should call: > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 which in turn > calls gen_fsm:start_link, thus providing a Pid in the end... > > but what it really does is apply(sim_group_fsm,start_link,Opts ++ Opts2,[ > ]) and then crashes unless i declare a start_link/2 on my gen_fsm... > > { > {badmatch, > {error, > {'EXIT', > {undef, > [ > {sim_group_fsm,start_link,[[{option1,40}],[{option29,100}]],[]}, > {supervisor,do_start_child_i,3,[ > {file,"supervisor.erl"}, > {line,319} > ]}...... > > i dont see why i finish having a apply/4 call in supervisor: 319 instead of > an apply/3... > > I want the sup to propagate one property list to children specs while i > will provide another during children spawns providing that sup combines > two in any way children will manage to demunge ...both property lists > > So cant just see what im doing wrong.... > > /angel > > > PS: i changed a bit the test code to force sup crash > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez Miguel wrote: > > } While calling a simple_one_by_one supervisor to start a gen fsm I > > found } it causes calling a start_link/2 function on the callback > > module of the } gen_fsm. > > > > You are specifying that function to be called to start the worker > > yourself. In your module group_supervisor.erl you provide a child_spec() > > in the > > > > return value of init/1: > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, worker, > > > > [sim_group]} > > > > The form of which is: > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > The second argument, StartFunc, has the form: > > {Module, Function, Arguments} > > > > You provided: > > {sim_group_fsm, start_link, [Opts]} From mjollnir.ray@REDACTED Fri Jun 22 13:42:09 2012 From: mjollnir.ray@REDACTED (Wu Ray) Date: Fri, 22 Jun 2012 19:42:09 +0800 Subject: [erlang-questions] R15B01 dtrace support on FreeBSD 9.0 broken Message-ID: Hi, guys I cannot configure with dtrace on my FreeBSD 9.0(x86_64), it was broken when configure ./configure --with-dynamic-trace=dtrace ....... dtrace: failed to compile script emulator/beam/erlang_dtrace.d: "/usr/lib/dtrace/psinfo.d", line 37: syntax error near "uid_t" configure: error: Could not precompile erlang_dtrace.d: dtrace -h failed configure: error: /bin/sh '/usr/home/wulei/otp_src_R15B01/erts/configure' failed for erts the dtrace version: $ dtrace -V dtrace: Sun D 1.6.3 I read the README.dtrace.md, at "Supported platforms", it said: * FreeBSD 9.0, though please see the "FreeBSD 9.0 Release Notes" section below! BUT there is no this section below, does I do something special on FreeBSD. Best Regards From erisa85d@REDACTED Fri Jun 22 13:51:52 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Fri, 22 Jun 2012 13:51:52 +0200 Subject: [erlang-questions] SMP for IO-bound applications In-Reply-To: <4fe42c05.7067980a.1893.6fb7@mx.google.com> References: <4fe42c05.7067980a.1893.6fb7@mx.google.com> Message-ID: Hey, thank you for your reply, but I use the same mysql driver even for my evaluations in Erlang OTP R12 and do not get the same CPU usage (for R12 is up to 45%). And the tests are exactly the same (same load). I have also checked the message queue of the process that supervises the connections, and it looks like there isn't a bottleneck in there (the queue length is most of the time 0 and up to 4 messages at most) And even for Mnesia the CPU usage (though low), doubles from R12 to R15 with smp enabled from 6% to 12% for write tests On Fri, Jun 22, 2012 at 10:22 AM, be.dmitry@REDACTED wrote: > I think that much of CPU usage is created by your MySQL driver. > You should have a look at driver details. There may be copying of the > query you send to MySQL between your benchmarking process and the real > process that owns the socket. The same with result - parsing response and > sending result back to benchmarking process. There may be a bottleneck in > choosing MySQL connection from connection pool if it is made through > supervisor. > Try to use etop to see what functions are at the bottlenecks. > I mean your io-intensive test may be just the test showing driver > bottlenecks and not the problems in smp. > > > > > ----- Reply message ----- > From: "Erisa Dervishi" > To: > Subject: [erlang-questions] SMP for IO-bound applications > Date: Thu, Jun 21, 2012 23:28 > > > One additonal comment I forgot to mention: > > For all Mnesia tests, the CPU usage is low in general. Only for the > transactional read test in R15 with SMP enabled i get a CPU usage of 70%. > From the load generator tool, i call the read(N) function that is showed > below: > > exec_read(0) -> done_reading; > > exec_read(N) -> > > Id=random:uniform(7000000), > > %%io:format("~p~n",[Id]), > > %%[_Row] = mnesia:dirty_read({subscriber,Id}), > > Fun = fun() -> > > mnesia:read({subscriber,Id}) > > end, > > {atomic,[Row]}=mnesia:transaction(Fun), > > %io:format("~p~n",[Row]), > > exec_read(N-1). > > read(N) -> > > {A1,A2,A3} = now(), > > random:seed(A1, A2, A3), > > exec_read(N). > > > I am using mpstat while my tests are running and beside the CPU usage, > another difference of this test from the other Mnesia tests is that the > syscl (system calls) column of mpstat has a value which is 20 times higher > than the other test cases. > > > > On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi > wrote: > > > Hi, > > As part of my studies, I have recently been doing some performance > > evaluations on Erlang SMP improvements for IO-bound applications. > > > > The applications I considered for the evaluation were Emysql driver ( > > https://github.com/Eonblast/Emysql) and Mnesia. > > I have created an Erlang module for performing reads/writes from/to Mysql > > DB through Emysql driver, and another Erlang module for communicating > with > > Mnesia. > > Both the modules have two methods write(nr_records) which writes > > nr_records records to the database (MySQL, or Mnesia), and > read(nr_records) > > which reads nr_records random records from the db. (both the reads and > > writes are consecutive, not in chunks) > > I have a load generator tool which simulates simultaneous requests to > both > > the modules and gathers performance statistics. I generate as much load > as > > needed to stress the applications (near the saturation point) > > The metrics I uses are the throughput (no. of sessions/sec), and the > > session duration. A session is a call to write(nr_records) > > or read(nr_records) function depending on the test case (read or write). > So > > if I call write(500), I measure the duration of a session that does 500 > > hundred inserts into the database. > > > > The tests I run have these parameters: > > > > A)Type of operations: 1- Reads 2-Writes > > I have just one table with 20 fields, and the reads and writes are just > > select and insert operations in that table > > > > B) I/O applications: 1- Emysql 2-Mnesia > > > > C) SMP parameters: > > 1- SMP enabled, no. of schedulers = no. of cpu cores > > 2-SMP enabled, no. of schedulers = 1 > > 3- SMP disabled > > > > D) Erlang OTP versions: > > 1- Erlang R12B (The SMP has only one run-queue and multiple schedulers) > > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) > > > > The tests included all possible combinations from A,B,C,D. They were run > > in Solaris 10 x86 (a 16 cores machine). > > In general these were the results I got: > > > > *Emysql driver:* > > > 1- There is a a big difference in performance between SMP enabled and > > disabled in both the Erlang versions (R12 and R15) for both read and > write > > tests. So I can say that SMP rocks! However, you have to have enough > load > > to achieve that (for low load I could not see any difference, sometimes > it > > was even better to disable SMP) > > > > 2- I was expecting a much better performance for SMP enabled (no. of > cores > > = no. of schedulers) in R15 than in R12, since the schedulers' logic has > > changed a lot from R12 to R15. But the results were more or less the > same. > > I was thinking since I had a multithreaded db like MySql, a multiple > > db-connections driver as Emysql, and SMP with multiple schedulers with > > their own run queues, I could get better results than in R12 where there > is > > only one run-queue and multiple schedulers (more lock contention). > > Is it maybe because the processes are just doing IO and nothing > > CPU-intensive? > > > > 3- I realized that CPU usage in R15 SMP enabled is twice higher than the > > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, and I > > can see that the beam process is using up to 75-80% of the CPU, the rest > is > > the mysql daemon process. Is it because in R15 the scheduling algorithm > has > > become more complex? > > > > *Mnesia* > > * > > > * > > I have only one table (as I said before) created with the attribute > > disc_copies, and it is stored only in one node. The whole table fits in > > RAM, and I tried to keep it simple by controlling the size of the table > > during my tests, in order not exceed RAM capacity > > > > 4- For the write tests in Mnesia I see the same behavior as in Mysql. > > However, since the data are all loaded in RAM, the session duration is > > shorter, and the gain in performance between SMP enabled and disabled is > > not as huge as in Mysql. > > > > 5- For the read tests, I have two observations. First, the dirty reads > are > > 10 times faster than the reads that use transactions. Second, I have > better > > results for SMP disabled (twice faster, though in both cases the session > > duration is in order of millisecods, 70 msec vs 30 msec). > > > > 6- Same as in Mysql, no difference in performance between R12 and R15 > with > > SMP enabled, for both read and write tests. > > > > If you have read so far, and have a comment on my results, you are > > welcome. Especially about the no difference btw R12 and R15 and the high > > CPU usage in R15 when the applications are IO-bound. > > > > Thanks, > > Erisa > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahrdad1@REDACTED Fri Jun 22 16:50:37 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Fri, 22 Jun 2012 10:50:37 -0400 Subject: [erlang-questions] Erlang is touch of genius Message-ID: Hi Erlangers I don't know if I am the only one who feels this way but every single time I implement a complex task in Erlang, I imagine amount of effort (not to mention frustration) I had to dedicate to accomplish the same task in Java/J2ee (I am senior Java/J2ee developer after all). And this brings to my mind Einstein's famous quote: "Any intelligent fool can make things bigger and more complex... It takes a touch of genius --- and a lot of courage to move in the opposite direction." Clearly Erlang is touch of genius. -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Fri Jun 22 17:07:08 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Fri, 22 Jun 2012 16:07:08 +0100 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: References: Message-ID: Yes, I'd agree... I'm happy to be drinking the Erlang kool-aid, too. On 22 June 2012 15:50, Shahrdad Shadab wrote: > Hi Erlangers > > I don't know if I am the only one who feels this way but every single time I > implement a complex task in Erlang, > I imagine amount of effort (not to mention frustration) I had to dedicate to > accomplish the same task in Java/J2ee > (I am senior Java/J2ee developer after all). And this brings to my mind > Einstein's famous quote: > > "Any intelligent fool can make things bigger and more complex... > It takes a touch of genius --- and a lot of courage to move in the opposite > direction." > > Clearly Erlang is touch of genius. > > > -- > Software Architect & Computer Scientist > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From vances@REDACTED Fri Jun 22 19:15:21 2012 From: vances@REDACTED (Vance Shipley) Date: Fri, 22 Jun 2012 22:45:21 +0530 Subject: [erlang-questions] JavaScript compiler for Erlang In-Reply-To: References: <20120621142036.GE16243@aluminum.wavenet.lk> Message-ID: <20120622171508.GA332@aluminum.motivity.ca> On Thu, Jun 21, 2012 at 11:12:26AM -0400, Ingo Jaeckel wrote: } Just out of curiosity: Do you want to compile JavaScript code to } Erlang or Erlang to JavaScript? I want to run Javascript in the beam emulator. } In either case - Why? What is the application behind this? At the moment I am working with VoiceXML which incorporates ECMAScript. Although they call a VoiceXML interpreter a "voice browser" if you consider where it is typically used in the network it is a server side application. } What about https://github.com/beamjs/erlv8/ and http://beamjs.org/? Yes, there are a couple bindings to external engines. Erlyjs was taking the approach I would prefer, interpreting the ECMAscript on the beam machine. -- -Vance From wojtek@REDACTED Fri Jun 22 19:42:59 2012 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Fri, 22 Jun 2012 19:42:59 +0200 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: References: Message-ID: <4FE4AEA3.3020404@power.com.pl> On 2012-06-22 17:07, Paul Barry wrote: > Yes, I'd agree... I'm happy to be drinking the Erlang kool-aid, too. > Yes, it is amazing how all the pieces of the puzzle fit together: processes, messaging (send semantics, selective receive), locality in garbage collection, functional programming, strong dynamic typing, pattern matching, atoms, recursion, hot code update, distribution, .... And there are more jigsaw pieces coming: native implemented processes, hasmaps (which are conceptually funcions, by the way). While java is collapsing under its own weight, there is no horizon of erlang improvement even remotely visible. Then comes the OTP, where you realize that many problems you are not yet even aware of, have already been solved. --Regards, Wojtek Narczynski From ingo.jaeckel@REDACTED Fri Jun 22 20:18:02 2012 From: ingo.jaeckel@REDACTED (Ingo Jaeckel) Date: Fri, 22 Jun 2012 14:18:02 -0400 Subject: [erlang-questions] JavaScript compiler for Erlang In-Reply-To: <20120622171508.GA332@aluminum.motivity.ca> References: <20120621142036.GE16243@aluminum.wavenet.lk> <20120622171508.GA332@aluminum.motivity.ca> Message-ID: > I want to run Javascript in the beam emulator. Sounds a bit like running node.js on beam :) Cool stuff! Thanks for sharing! Ingo On Fri, Jun 22, 2012 at 1:15 PM, Vance Shipley wrote: > On Thu, Jun 21, 2012 at 11:12:26AM -0400, Ingo Jaeckel wrote: > } ?Just out of curiosity: Do you want to compile JavaScript code to > } ?Erlang or Erlang to JavaScript? > > I want to run Javascript in the beam emulator. > > } ?In either case - Why? What is the application behind this? > > At the moment I am working with VoiceXML which incorporates ECMAScript. > Although they call a VoiceXML interpreter a "voice browser" if you > consider where it is typically used in the network it is a server side > application. > > } ?What about https://github.com/beamjs/erlv8/ and http://beamjs.org/? > > Yes, there are a couple bindings to external engines. ?Erlyjs was > taking the approach I would prefer, interpreting the ECMAscript on > the beam machine. > > -- > ? ? ? ?-Vance From ingo.jaeckel@REDACTED Fri Jun 22 21:11:21 2012 From: ingo.jaeckel@REDACTED (Ingo Jaeckel) Date: Fri, 22 Jun 2012 15:11:21 -0400 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: <4FE4AEA3.3020404@power.com.pl> References: <4FE4AEA3.3020404@power.com.pl> Message-ID: What about tool support e.g. refactoring, (remote) debugging, deployment, monitoring, profiling, code analysis tools, etc. There are so many tools supporting the development process for more popular languages like Java and C#. Do you think that (due to the language itself?) those are not needed for Erlang? Compared to the free/commercial tools available for Java, the Erlang tool chain looks not mature enough to me. I *love* the language itself and achieving nine 9s is great, too [1]. But don't you need so much more than great language concepts on a day-to-day basis when you are choosing the language for your next super-cool project? Ingo [1] http://ll2.ai.mit.edu/talks/armstrong.pdf On Fri, Jun 22, 2012 at 1:42 PM, Wojtek Narczy?ski wrote: > On 2012-06-22 17:07, Paul Barry wrote: >> >> Yes, I'd agree... I'm happy to be drinking the Erlang kool-aid, too. >> > Yes, it is amazing how all the pieces of the puzzle fit together: processes, > messaging (send semantics, selective receive), locality in garbage > collection, functional programming, strong dynamic typing, pattern matching, > atoms, recursion, hot code update, distribution, .... And there are more > jigsaw pieces coming: native implemented processes, hasmaps (which are > conceptually funcions, by the way). While java is collapsing under its own > weight, there is no horizon of erlang improvement even remotely visible. > > Then comes the OTP, where you realize that many problems you are not yet > even aware of, have already been solved. > > --Regards, > Wojtek Narczynski > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rapsey@REDACTED Fri Jun 22 21:31:05 2012 From: rapsey@REDACTED (Rapsey) Date: Fri, 22 Jun 2012 21:31:05 +0200 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: References: <4FE4AEA3.3020404@power.com.pl> Message-ID: I do think the language itself makes them much less necessary. I've been working in erlang for 5 years or so full time. Never used a debugger and never had a need to. Never used and IDE and also never needed to. I write my code in a plain text editor (sublime text 2) and have erlang running in an external terminal. I have an erlang process running that scans my src folders every few seconds and checks if anything changed and compiles any source file that has. I don't think I could be remotely as productive using any other language. Sergej On Fri, Jun 22, 2012 at 9:11 PM, Ingo Jaeckel wrote: > What about tool support e.g. refactoring, (remote) debugging, > deployment, monitoring, profiling, code analysis tools, etc. There are > so many tools supporting the development process for more popular > languages like Java and C#. Do you think that (due to the language > itself?) those are not needed for Erlang? Compared to the > free/commercial tools available for Java, the Erlang tool chain looks > not mature enough to me. > > I *love* the language itself and achieving nine 9s is great, too [1]. > But don't you need so much more than great language concepts on a > day-to-day basis when you are choosing the language for your next > super-cool project? > > Ingo > > [1] http://ll2.ai.mit.edu/talks/armstrong.pdf > > On Fri, Jun 22, 2012 at 1:42 PM, Wojtek Narczy?ski > wrote: > > On 2012-06-22 17:07, Paul Barry wrote: > >> > >> Yes, I'd agree... I'm happy to be drinking the Erlang kool-aid, too. > >> > > Yes, it is amazing how all the pieces of the puzzle fit together: > processes, > > messaging (send semantics, selective receive), locality in garbage > > collection, functional programming, strong dynamic typing, pattern > matching, > > atoms, recursion, hot code update, distribution, .... And there are more > > jigsaw pieces coming: native implemented processes, hasmaps (which are > > conceptually funcions, by the way). While java is collapsing under its > own > > weight, there is no horizon of erlang improvement even remotely visible. > > > > Then comes the OTP, where you realize that many problems you are not yet > > even aware of, have already been solved. > > > > --Regards, > > Wojtek Narczynski > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jun 22 21:36:11 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 22 Jun 2012 21:36:11 +0200 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: References: <4FE4AEA3.3020404@power.com.pl> Message-ID: <4FE4C92B.3010906@ninenines.eu> On 06/22/2012 09:11 PM, Ingo Jaeckel wrote: > What about tool support e.g. refactoring, (remote) debugging, > deployment, monitoring, profiling, code analysis tools, etc. There are > so many tools supporting the development process for more popular > languages like Java and C#. Do you think that (due to the language > itself?) those are not needed for Erlang? Compared to the > free/commercial tools available for Java, the Erlang tool chain looks > not mature enough to me. > > I *love* the language itself and achieving nine 9s is great, too [1]. > But don't you need so much more than great language concepts on a > day-to-day basis when you are choosing the language for your next > super-cool project? Refactoring: third party projects like Wrangler, Refactorerl - I've never had the need for this myself though (Remote) debugging: remote shells, dbg, sys, redbug, too many options there Deployment: releases are half what we need, the other half is generally custom built Monitoring: snmp Profiling: eprof, fprof, cprof, and so on Code analysis: dialyzer, typer, xref, and more Erlang also has awesome tools like proper and quickcheck for writing solid test suites. Erlang/OTP comes with the most impressive toolchain I've ever seen, you just need to take the time to learn the many tools. Plus it's very easy to build upon existing tools if you have specific needs. I'm probably missing many tools, more experienced Erlang developers will surely give you more. Erlang/OTP has many gems to find for those who seek them. -- Lo?c Hoguin Erlang Cowboy Nine Nines From matthew.hillsborough@REDACTED Fri Jun 22 22:06:09 2012 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Fri, 22 Jun 2012 16:06:09 -0400 Subject: [erlang-questions] rebar release not starting with an automated start script In-Reply-To: References: Message-ID: Just a heads up, I finally fixed this issue. In order to get this working, I had to stop/start my server with the following if I want to do it headless, without an interactive shell: "script -c 'TERM=xterm-256color ./bin/myapp start' /dev/null" Simply doing "service myapp start" would cause insane behavior. I'm guessing one of my modules, maybe lager? is requiring TERM to be defined. I'm just curious how come I'm googling and no one else had this issue before. That worries me more than anything. Surely someone else ran into a similar case where headless starting of their erlang app was not working properly? On Sun, Jun 17, 2012 at 9:13 AM, Kunthar wrote: > What is your port number? > May be your rebar user have no rights to run on this port? > > > On Sun, Jun 17, 2012 at 4:34 AM, Matthew Hillsborough > wrote: >> Hi all, >> >> I am using rebar in order to make releases of my Erlang OTP >> application. The application runs rock solid thus far. I'm having one >> complication where I am trying to script an automated start of the >> Erlang application after a reboot. I trimmed down the script to the >> ultimate basics and am running something like this, completely >> headless (via Jenkins and/or RightScale). >> >> ========== >> #!/bin/bash -ex >> >> chmod u+x /home/erlang/erlapp/bin/erlapp >> /home/erlang/erlapp/bin/erlapp start >> >> ========== >> >> >> The odd part is that it does listen on the TCP port I have it built to >> listen and accept connections on. However, no connections make it >> through and I cannot attach to the console using >> ``/home/erlang/erlapp/bin/erlapp attach``. >> >> If I manually execute the above in login shell, everything works just >> great. Server starts, I can attach to the console. Server accepts >> connections from clients, etc. >> >> Any idea why I can't script start with a rebar release? >> >> Thanks all. >> >> Matthew >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > BR, > \|/ Kunthar From gumm@REDACTED Fri Jun 22 23:07:21 2012 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 22 Jun 2012 16:07:21 -0500 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: <4FE4C92B.3010906@ninenines.eu> References: <4FE4AEA3.3020404@power.com.pl> <4FE4C92B.3010906@ninenines.eu> Message-ID: Great list, Loic! Two more to add: Monitoring: Mazen Harake's entop Auto compiling: Rusty Klophaus's sync -Jesse -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Jun 22, 2012 2:36 PM, "Lo?c Hoguin" wrote: > On 06/22/2012 09:11 PM, Ingo Jaeckel wrote: > >> What about tool support e.g. refactoring, (remote) debugging, >> deployment, monitoring, profiling, code analysis tools, etc. There are >> so many tools supporting the development process for more popular >> languages like Java and C#. Do you think that (due to the language >> itself?) those are not needed for Erlang? Compared to the >> free/commercial tools available for Java, the Erlang tool chain looks >> not mature enough to me. >> >> I *love* the language itself and achieving nine 9s is great, too [1]. >> But don't you need so much more than great language concepts on a >> day-to-day basis when you are choosing the language for your next >> super-cool project? >> > > Refactoring: third party projects like Wrangler, Refactorerl - I've never > had the need for this myself though > (Remote) debugging: remote shells, dbg, sys, redbug, too many options there > Deployment: releases are half what we need, the other half is generally > custom built > Monitoring: snmp > Profiling: eprof, fprof, cprof, and so on > Code analysis: dialyzer, typer, xref, and more > > Erlang also has awesome tools like proper and quickcheck for writing solid > test suites. > > Erlang/OTP comes with the most impressive toolchain I've ever seen, you > just need to take the time to learn the many tools. Plus it's very easy to > build upon existing tools if you have specific needs. > > I'm probably missing many tools, more experienced Erlang developers will > surely give you more. Erlang/OTP has many gems to find for those who seek > them. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Jun 22 23:19:28 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Jun 2012 14:19:28 -0700 Subject: [erlang-questions] gproc Help Message-ID: I'm trying to use gproc as a very basic pub/sub. Here is example code I have (very basic): -module(test). -compile([export_all]). new() -> Pid = spawn_link(?MODULE, apply, []), gproc_ps:subscribe(l, test), Pid. apply() -> receive {gproc_ps_event, test, Msg} -> io:put_chars(Msg), apply() end. publish() -> gproc_ps:publish(l, test, "My test message"). In the shell I then do: Pid = test:new(). test:publish(). My expectation was that I would get an output of "My test message", but all I get is {gproc_ps_event,test,"My test message"}. When I do a flush(), I do get the Shell got {gproc_ps_event,test,"My test message"}. What am I doing wrong? I obviously don't understand something, can someone explain how to get this simple example to work. I know I can do Pid ! {....} in this example, but I'm trying to play with gproc. Thanks for any help! Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Jun 22 23:26:57 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 22 Jun 2012 17:26:57 -0400 Subject: [erlang-questions] gproc Help In-Reply-To: References: Message-ID: On Fri, Jun 22, 2012 at 5:19 PM, Andrew Berman wrote: > I'm trying to use gproc as a very basic pub/sub. Here is example code I > have (very basic): > > -module(test). > > -compile([export_all]). > > new() -> > Pid = spawn_link(?MODULE, apply, []), > gproc_ps:subscribe(l, test), > Pid. > > apply() -> > receive > {gproc_ps_event, test, Msg} -> io:put_chars(Msg), > apply() > end. > > publish() -> > gproc_ps:publish(l, test, "My test message"). > > In the shell I then do: > > Pid = test:new(). > test:publish(). > > My expectation was that I would get an output of "My test message", but > all I get is {gproc_ps_event,test,"My test message"}. When I do a flush(), > I do get the Shell got {gproc_ps_event,test,"My test message"}. What am I > doing wrong? I obviously don't understand something, can someone explain > how to get this simple example to work. I know I can do Pid ! {....} in > this example, but I'm trying to play with gproc. > > Thanks for any help! > You're subscribing to 'test' from the shell process, not from Pid, so that is why the shell is receiving the message. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Jun 22 23:33:27 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Jun 2012 14:33:27 -0700 Subject: [erlang-questions] gproc Help In-Reply-To: References: Message-ID: Yes, but how do I have my new process subscribe to it? Thanks, Andrew On Fri, Jun 22, 2012 at 2:26 PM, Bob Ippolito wrote: > On Fri, Jun 22, 2012 at 5:19 PM, Andrew Berman wrote: > >> I'm trying to use gproc as a very basic pub/sub. Here is example code I >> have (very basic): >> >> -module(test). >> >> -compile([export_all]). >> >> new() -> >> Pid = spawn_link(?MODULE, apply, []), >> gproc_ps:subscribe(l, test), >> Pid. >> >> apply() -> >> receive >> {gproc_ps_event, test, Msg} -> io:put_chars(Msg), >> apply() >> end. >> >> publish() -> >> gproc_ps:publish(l, test, "My test message"). >> >> In the shell I then do: >> >> Pid = test:new(). >> test:publish(). >> >> My expectation was that I would get an output of "My test message", but >> all I get is {gproc_ps_event,test,"My test message"}. When I do a flush(), >> I do get the Shell got {gproc_ps_event,test,"My test message"}. What am I >> doing wrong? I obviously don't understand something, can someone explain >> how to get this simple example to work. I know I can do Pid ! {....} in >> this example, but I'm trying to play with gproc. >> >> Thanks for any help! >> > > You're subscribing to 'test' from the shell process, not from Pid, so that > is why the shell is receiving the message. > > -bob > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Jun 22 23:35:25 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 22 Jun 2012 17:35:25 -0400 Subject: [erlang-questions] gproc Help In-Reply-To: References: Message-ID: You call gproc_ps:subscribe(l, test) from the spawned process instead of from the shell process. For example: new() -> spawn_link(fun () -> gproc_ps:subscribe(l, test), ?MODULE:apply() end). On Fri, Jun 22, 2012 at 5:33 PM, Andrew Berman wrote: > Yes, but how do I have my new process subscribe to it? > > Thanks, > > Andrew > > On Fri, Jun 22, 2012 at 2:26 PM, Bob Ippolito wrote: > >> On Fri, Jun 22, 2012 at 5:19 PM, Andrew Berman wrote: >> >>> I'm trying to use gproc as a very basic pub/sub. Here is example code I >>> have (very basic): >>> >>> -module(test). >>> >>> -compile([export_all]). >>> >>> new() -> >>> Pid = spawn_link(?MODULE, apply, []), >>> gproc_ps:subscribe(l, test), >>> Pid. >>> >>> apply() -> >>> receive >>> {gproc_ps_event, test, Msg} -> io:put_chars(Msg), >>> apply() >>> end. >>> >>> publish() -> >>> gproc_ps:publish(l, test, "My test message"). >>> >>> In the shell I then do: >>> >>> Pid = test:new(). >>> test:publish(). >>> >>> My expectation was that I would get an output of "My test message", but >>> all I get is {gproc_ps_event,test,"My test message"}. When I do a flush(), >>> I do get the Shell got {gproc_ps_event,test,"My test message"}. What am I >>> doing wrong? I obviously don't understand something, can someone explain >>> how to get this simple example to work. I know I can do Pid ! {....} in >>> this example, but I'm trying to play with gproc. >>> >>> Thanks for any help! >>> >> >> You're subscribing to 'test' from the shell process, not from Pid, so >> that is why the shell is receiving the message. >> >> -bob >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Jun 22 23:36:17 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Jun 2012 14:36:17 -0700 Subject: [erlang-questions] gproc Help In-Reply-To: References: Message-ID: Ahhhhh, of course! Stupid me. Thanks for your help, Bob! On Fri, Jun 22, 2012 at 2:35 PM, Bob Ippolito wrote: > You call gproc_ps:subscribe(l, test) from the spawned process instead of > from the shell process. > > For example: > > new() -> > spawn_link(fun () -> gproc_ps:subscribe(l, test), ?MODULE:apply() end). > > > On Fri, Jun 22, 2012 at 5:33 PM, Andrew Berman wrote: > >> Yes, but how do I have my new process subscribe to it? >> >> Thanks, >> >> Andrew >> >> On Fri, Jun 22, 2012 at 2:26 PM, Bob Ippolito wrote: >> >>> On Fri, Jun 22, 2012 at 5:19 PM, Andrew Berman wrote: >>> >>>> I'm trying to use gproc as a very basic pub/sub. Here is example code >>>> I have (very basic): >>>> >>>> -module(test). >>>> >>>> -compile([export_all]). >>>> >>>> new() -> >>>> Pid = spawn_link(?MODULE, apply, []), >>>> gproc_ps:subscribe(l, test), >>>> Pid. >>>> >>>> apply() -> >>>> receive >>>> {gproc_ps_event, test, Msg} -> io:put_chars(Msg), >>>> apply() >>>> end. >>>> >>>> publish() -> >>>> gproc_ps:publish(l, test, "My test message"). >>>> >>>> In the shell I then do: >>>> >>>> Pid = test:new(). >>>> test:publish(). >>>> >>>> My expectation was that I would get an output of "My test message", but >>>> all I get is {gproc_ps_event,test,"My test message"}. When I do a flush(), >>>> I do get the Shell got {gproc_ps_event,test,"My test message"}. What am I >>>> doing wrong? I obviously don't understand something, can someone explain >>>> how to get this simple example to work. I know I can do Pid ! {....} in >>>> this example, but I'm trying to play with gproc. >>>> >>>> Thanks for any help! >>>> >>> >>> You're subscribing to 'test' from the shell process, not from Pid, so >>> that is why the shell is receiving the message. >>> >>> -bob >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Sat Jun 23 00:35:33 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Fri, 22 Jun 2012 15:35:33 -0700 Subject: [erlang-questions] CouchDB backend for Lager Message-ID: Hi List, I've just pushed a very quick and dirty Lager backend using CouchDB to github: https://github.com/kofron/lager_couchdb_backend I literally wrote it in 10 minutes, so it's a pretty rough cut. Any and all feedback welcome. Jared Kofron -------------- next part -------------- An HTML attachment was scrubbed... URL: From wojtek@REDACTED Sat Jun 23 00:40:14 2012 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Sat, 23 Jun 2012 00:40:14 +0200 Subject: [erlang-questions] Erlang is touch of genius In-Reply-To: References: <4FE4AEA3.3020404@power.com.pl> Message-ID: <4FE4F44E.2050900@power.com.pl> On 2012-06-22 21:11, Ingo Jaeckel wrote: > What about tool support e.g. refactoring, (remote) debugging, > deployment, monitoring, profiling, code analysis tools, etc. There are > so many tools supporting the development process for more popular > languages like Java and C#. Do you think that (due to the language > itself?) those are not needed for Erlang? Would you choose to build your house out of straw reinforced poo, driven by the perceived maturity of the tools for that kind of craftsmanship? I do think that, because of the superior conceputal integrity of Erlang/OTP, less tools are necessary. And I do find the available tools adequate. > Compared to the free/commercial tools available for Java, the Erlang tool chain looks > not mature enough to me. Think again. The tools are good enough for Ericsson to achieve those 9s, but not mature enough for you. > I *love* the language itself and achieving nine 9s is great, too [1]. > But don't you need so much more than great language concepts on a > day-to-day basis when you are choosing the language for your next > super-cool project? Yes, you need great libraries, and Erlang/OTP + community effort has that. --Regards, Wojtek Narczynski From max.lapshin@REDACTED Sat Jun 23 01:00:58 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 23 Jun 2012 03:00:58 +0400 Subject: [erlang-questions] Erlang Factory Moscow Message-ID: Thanks to all organizators and participants! From olivier.girondel@REDACTED Sat Jun 23 03:30:16 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Sat, 23 Jun 2012 03:30:16 +0200 Subject: [erlang-questions] erlang mailing lists stats / erlang "popularity-contest"-like Message-ID: Hello, I came across these statistics through some search engine: http://www.erlang-solutions.com/upload/docs/44/erlangstatistics.pdf Is there a "fresh" version of this kind of statistics somewhere ? -- Olivier From ckawatak@REDACTED Sat Jun 23 13:02:00 2012 From: ckawatak@REDACTED (Chiharu Kawatake) Date: Sat, 23 Jun 2012 20:02:00 +0900 Subject: [erlang-questions] [ANN] Waste 0.1.0 Message-ID: Hi all, I just released an initial version of Waste, an experimental library for RPC using Thrift and AMQP (RabbitMQ is used in the implementation). https://github.com/mille-printemps/waste Motivation behind Waste is to utilize services in a fault-tolerant way from clients that are implemented in different programming languages. Waste extends a transport layer of Thrift so that it can put requests from clients to a queue and dispatch the requests to a service. If multiple services (on multiple physical servers) are subscribing the queue, the load of the requests is balanced among the servers. Currently, Waste allows you to call a service implemented in C++ from a client implemented in Erlang. Chiharu -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji.rikitake@REDACTED Sun Jun 24 05:03:44 2012 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 24 Jun 2012 12:03:44 +0900 Subject: [erlang-questions] R15B01 dtrace support on FreeBSD 9.0 broken In-Reply-To: References: Message-ID: <20120624030344.GA9534@k2r.org> See https://github.com/jj1bdx/erlang-freebsd-port for the DTrace config. Kenji Rikitake From kenji.rikitake@REDACTED Sun Jun 24 05:25:31 2012 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 24 Jun 2012 12:25:31 +0900 Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: References: <4FE23958.80401@hyber.org> <4FE23E00.8040401@hyber.org> <75746237-E3D3-46BF-8CC0-8762B6721618@cs.otago.ac.nz> <4FE2D548.80909@hyber.org> Message-ID: <20120624032531.GB9534@k2r.org> FYI: * Mersenne Twister PRNGs are NOT cryptographically safe either, although the random number generation period is much much longer (approx. 2^43 on AS183, (2^19937) - 1 for SFMT) and the state space is far less easier to be exploited. * sfmt-erlang is now runable on non-NIF environment (though it's slow) * I've been working on a lightweight variant of MT called TinyMT (period: (2^127) - 1, internal state: 28 bytes), including compatibility functions to the random module, and is capable of generating ~2^58 different RNG streams. It's at https://github.com/jj1bdx/tinymt-erlang ++> Tuncer Ayaz [2012-06-21 12:47:46 +0200]: > It should probably be replaced with Kenji's sfmt-erlang or an > implementation of (C)MWC. > > https://groups.google.com/group/comp.soft-sys.math.mathematica/msg/95a94c3b2aa5f077 Kenji Rikitake From mabrek@REDACTED Sun Jun 24 10:14:15 2012 From: mabrek@REDACTED (Anton Lebedevich) Date: Sun, 24 Jun 2012 12:14:15 +0400 Subject: [erlang-questions] which distel fork to use? Message-ID: <4FE6CC57.5030309@gmail.com> Hello! There are main distel repository https://github.com/massemanet/distel and two actively maintaned forks that contain a lot of changes: https://github.com/rost/distel https://github.com/jixiuf/distel I'm trying to understand what features do these forks introduce and which one should I use. Regards, Anton Lebedevich. From ronny.meeus@REDACTED Sun Jun 24 10:47:01 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Sun, 24 Jun 2012 10:47:01 +0200 Subject: [erlang-questions] SMP for IO-bound applications In-Reply-To: References: <4fe42c05.7067980a.1893.6fb7@mx.google.com> Message-ID: On Fri, Jun 22, 2012 at 1:51 PM, Erisa Dervishi wrote: > Hey, thank you for your reply, but I use the same mysql driver even for my > evaluations in? Erlang OTP R12 and do not get the same CPU usage (for R12 is > up to 45%). And the tests are exactly the same (same load). I have also > checked the message queue of the process that supervises the connections, > and it looks like there isn't a bottleneck in there (the queue length is > most of the time 0 and up to 4 messages at most) > And even for Mnesia the CPU usage (though low), doubles from R12 to R15 with > smp enabled from 6% to 12% for write tests > > > > On Fri, Jun 22, 2012 at 10:22 AM, be.dmitry@REDACTED > wrote: >> >> I think that much of CPU usage is created by your MySQL driver. >> You should have a look at driver details. There may be copying of the >> query you send to MySQL between your benchmarking process and the real >> process that owns the socket. The same with result - parsing response and >> sending result back to benchmarking process. There may be a bottleneck in >> choosing MySQL connection from connection pool if it is made through >> supervisor. >> Try to use etop to see what functions are at the bottlenecks. >> I mean your io-intensive test may be just the test showing driver >> bottlenecks and not the problems in smp. >> >> >> >> >> ----- Reply message ----- >> From: "Erisa Dervishi" >> To: >> Subject: [erlang-questions] SMP for IO-bound applications >> Date: Thu, Jun 21, 2012 23:28 >> >> >> One additonal comment I forgot to mention: >> >> For all Mnesia tests, the CPU usage is low in general. Only for the >> transactional read test in R15 with SMP enabled i get a CPU usage of 70%. >> From the load generator tool, i call the read(N) function that is showed >> below: >> >> exec_read(0) -> done_reading; >> >> exec_read(N) ?-> >> >> ? ? ? ?Id=random:uniform(7000000), >> >> ? ? ? ?%%io:format("~p~n",[Id]), >> >> ? ? ? ?%%[_Row] = mnesia:dirty_read({subscriber,Id}), >> >> ? ? ? ?Fun = fun() -> >> >> ? ? ? ? ? mnesia:read({subscriber,Id}) >> >> ? ? ? ?end, >> >> ? ? ? ?{atomic,[Row]}=mnesia:transaction(Fun), >> >> ? ? ? ?%io:format("~p~n",[Row]), >> >> ? ? ? ?exec_read(N-1). >> >> read(N) -> >> >> ? ? ? ?{A1,A2,A3} = now(), >> >> ? ? ? ?random:seed(A1, A2, A3), >> >> ? ? ? ?exec_read(N). >> >> >> I am using mpstat while my tests are running and beside the CPU usage, >> another difference of this test from the other Mnesia tests is that the >> syscl (system calls) column of mpstat has a value which is 20 times higher >> than the other test cases. >> >> >> >> On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi >> wrote: >> >> > Hi, >> > As part of my studies, I have recently been doing some performance >> > evaluations on Erlang SMP improvements ?for IO-bound applications. >> > >> > The applications I considered for the evaluation were Emysql driver ( >> > https://github.com/Eonblast/Emysql) and Mnesia. >> > I have created an Erlang module for performing reads/writes from/to >> > Mysql >> > DB through Emysql driver, and another Erlang module for communicating >> > with >> > Mnesia. >> > Both the modules have two methods write(nr_records) which writes >> > nr_records records to the database (MySQL, or Mnesia), and >> > read(nr_records) >> > which reads nr_records random records from the db. (both the reads and >> > writes are consecutive, not in chunks) >> > I have a load generator tool which simulates simultaneous requests to >> > both >> > the modules and gathers performance statistics. I generate as much load >> > as >> > needed to stress the applications (near the saturation point) >> > The metrics I uses are the throughput (no. of sessions/sec), and the >> > session duration. A session is a call to write(nr_records) >> > or read(nr_records) function depending on the test case (read or write). >> > So >> > if I call write(500), I measure the duration of a session that does 500 >> > hundred inserts into the database. >> > >> > The tests I run have these parameters: >> > >> > A)Type of operations: 1- Reads 2-Writes >> > I have just one table with 20 fields, and the reads and writes are just >> > select and insert operations in that table >> > >> > B) I/O applications: 1- Emysql 2-Mnesia >> > >> > C) SMP parameters: >> > 1- SMP enabled, no. of schedulers = no. of cpu cores >> > 2-SMP ?enabled, no. of schedulers = 1 >> > 3- SMP disabled >> > >> > D) Erlang OTP versions: >> > 1- Erlang R12B (The SMP has only one run-queue and multiple schedulers) >> > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) >> > >> > The tests included all possible combinations from A,B,C,D. They were run >> > in Solaris 10 x86 (a 16 cores machine). >> > In general these were the results I got: >> > >> > *Emysql driver:* >> >> > 1- There is a a big difference in performance between SMP enabled and >> > disabled in both the Erlang versions (R12 and R15) ?for both read and >> > write >> > tests. ?So I can say that SMP rocks! However, you have to have enough >> > load >> > to achieve that (for low load I could not see any difference, sometimes >> > it >> > was even better to disable SMP) >> > >> > 2- I was expecting a much better performance for SMP enabled (no. of >> > cores >> > = no. of schedulers) in R15 than in R12, since the schedulers' logic has >> > changed a lot from R12 to R15. But the results were more or less the >> > same. >> > ?I was thinking since I had a multithreaded db like MySql, a multiple >> > db-connections driver as Emysql, and SMP with multiple schedulers with >> > their own run queues, I could get better results than in R12 where there >> > is >> > only one run-queue and multiple schedulers (more lock contention). >> > Is it maybe because the processes are just doing IO and nothing >> > CPU-intensive? >> > >> > 3- I realized that CPU usage in R15 SMP enabled is twice higher than the >> > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, and >> > I >> > can see that the beam process is using up to 75-80% of the CPU, the rest >> > is >> > the mysql daemon process. Is it because in R15 the scheduling algorithm >> > has >> > become more complex? >> > >> > *Mnesia* >> > * >> >> > * >> > I have only one table (as I said before) created with the attribute >> > disc_copies, and it is stored only in one node. The whole table fits in >> > RAM, and I tried to keep it simple by controlling the size of the table >> > during my tests, in order not exceed RAM capacity >> > >> > 4- ?For the write tests in Mnesia I see the same behavior as in Mysql. >> > However, since the data are all loaded in RAM, the session duration is >> > shorter, and the gain in performance between SMP enabled and disabled is >> > not as huge as in Mysql. >> > >> > 5- For the read tests, I have two observations. First, the dirty reads >> > are >> > 10 times faster than the reads that use transactions. Second, I have >> > better >> > results for SMP disabled (twice faster, though in both cases the session >> > duration is in order of millisecods, 70 msec vs 30 msec). >> > >> > 6- Same as in Mysql, no difference in performance between R12 and R15 >> > with >> > SMP enabled, for both read and write tests. >> > >> > If you have read so far, and have a comment on my results, you are >> > welcome. Especially about the no difference btw R12 and R15 and the high >> > CPU usage in R15 when the applications are IO-bound. >> > >> > Thanks, >> > Erisa >> > >> > >> > >> > >> > >> > >> > >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Hello I'm very interested also in the behavior of Erlang applications in an SMP environment (so also my post "Strange observation running in an SMP environment." on this mailing list). Is the code that you have created somewhere available so that I can also play with it? In case not: are you willing to share it with the community? Thanks -- Best regards, Ronny From erisa85d@REDACTED Sun Jun 24 20:11:58 2012 From: erisa85d@REDACTED (Erisa Dervishi) Date: Sun, 24 Jun 2012 20:11:58 +0200 Subject: [erlang-questions] SMP for IO-bound applications In-Reply-To: References: <4fe42c05.7067980a.1893.6fb7@mx.google.com> Message-ID: Yes sure, even though I am using Tsung (a load generator tool written in Erlang) to generate my load and get some results back. I have added a new Tsung plugin for the purpose of my tests. I am a bit busy right now but If you are interested I will share what I have done so far. However, I would really like to have an opinion from an expert about my comments in here On Sun, Jun 24, 2012 at 10:47 AM, Ronny Meeus wrote: > On Fri, Jun 22, 2012 at 1:51 PM, Erisa Dervishi > wrote: > > Hey, thank you for your reply, but I use the same mysql driver even for > my > > evaluations in Erlang OTP R12 and do not get the same CPU usage (for > R12 is > > up to 45%). And the tests are exactly the same (same load). I have also > > checked the message queue of the process that supervises the connections, > > and it looks like there isn't a bottleneck in there (the queue length is > > most of the time 0 and up to 4 messages at most) > > And even for Mnesia the CPU usage (though low), doubles from R12 to R15 > with > > smp enabled from 6% to 12% for write tests > > > > > > > > On Fri, Jun 22, 2012 at 10:22 AM, be.dmitry@REDACTED < > be.dmitry@REDACTED> > > wrote: > >> > >> I think that much of CPU usage is created by your MySQL driver. > >> You should have a look at driver details. There may be copying of the > >> query you send to MySQL between your benchmarking process and the real > >> process that owns the socket. The same with result - parsing response > and > >> sending result back to benchmarking process. There may be a bottleneck > in > >> choosing MySQL connection from connection pool if it is made through > >> supervisor. > >> Try to use etop to see what functions are at the bottlenecks. > >> I mean your io-intensive test may be just the test showing driver > >> bottlenecks and not the problems in smp. > >> > >> > >> > >> > >> ----- Reply message ----- > >> From: "Erisa Dervishi" > >> To: > >> Subject: [erlang-questions] SMP for IO-bound applications > >> Date: Thu, Jun 21, 2012 23:28 > >> > >> > >> One additonal comment I forgot to mention: > >> > >> For all Mnesia tests, the CPU usage is low in general. Only for the > >> transactional read test in R15 with SMP enabled i get a CPU usage of > 70%. > >> From the load generator tool, i call the read(N) function that is showed > >> below: > >> > >> exec_read(0) -> done_reading; > >> > >> exec_read(N) -> > >> > >> Id=random:uniform(7000000), > >> > >> %%io:format("~p~n",[Id]), > >> > >> %%[_Row] = mnesia:dirty_read({subscriber,Id}), > >> > >> Fun = fun() -> > >> > >> mnesia:read({subscriber,Id}) > >> > >> end, > >> > >> {atomic,[Row]}=mnesia:transaction(Fun), > >> > >> %io:format("~p~n",[Row]), > >> > >> exec_read(N-1). > >> > >> read(N) -> > >> > >> {A1,A2,A3} = now(), > >> > >> random:seed(A1, A2, A3), > >> > >> exec_read(N). > >> > >> > >> I am using mpstat while my tests are running and beside the CPU usage, > >> another difference of this test from the other Mnesia tests is that the > >> syscl (system calls) column of mpstat has a value which is 20 times > higher > >> than the other test cases. > >> > >> > >> > >> On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi > >> wrote: > >> > >> > Hi, > >> > As part of my studies, I have recently been doing some performance > >> > evaluations on Erlang SMP improvements for IO-bound applications. > >> > > >> > The applications I considered for the evaluation were Emysql driver ( > >> > https://github.com/Eonblast/Emysql) and Mnesia. > >> > I have created an Erlang module for performing reads/writes from/to > >> > Mysql > >> > DB through Emysql driver, and another Erlang module for communicating > >> > with > >> > Mnesia. > >> > Both the modules have two methods write(nr_records) which writes > >> > nr_records records to the database (MySQL, or Mnesia), and > >> > read(nr_records) > >> > which reads nr_records random records from the db. (both the reads and > >> > writes are consecutive, not in chunks) > >> > I have a load generator tool which simulates simultaneous requests to > >> > both > >> > the modules and gathers performance statistics. I generate as much > load > >> > as > >> > needed to stress the applications (near the saturation point) > >> > The metrics I uses are the throughput (no. of sessions/sec), and the > >> > session duration. A session is a call to write(nr_records) > >> > or read(nr_records) function depending on the test case (read or > write). > >> > So > >> > if I call write(500), I measure the duration of a session that does > 500 > >> > hundred inserts into the database. > >> > > >> > The tests I run have these parameters: > >> > > >> > A)Type of operations: 1- Reads 2-Writes > >> > I have just one table with 20 fields, and the reads and writes are > just > >> > select and insert operations in that table > >> > > >> > B) I/O applications: 1- Emysql 2-Mnesia > >> > > >> > C) SMP parameters: > >> > 1- SMP enabled, no. of schedulers = no. of cpu cores > >> > 2-SMP enabled, no. of schedulers = 1 > >> > 3- SMP disabled > >> > > >> > D) Erlang OTP versions: > >> > 1- Erlang R12B (The SMP has only one run-queue and multiple > schedulers) > >> > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) > >> > > >> > The tests included all possible combinations from A,B,C,D. They were > run > >> > in Solaris 10 x86 (a 16 cores machine). > >> > In general these were the results I got: > >> > > >> > *Emysql driver:* > >> > >> > 1- There is a a big difference in performance between SMP enabled and > >> > disabled in both the Erlang versions (R12 and R15) for both read and > >> > write > >> > tests. So I can say that SMP rocks! However, you have to have enough > >> > load > >> > to achieve that (for low load I could not see any difference, > sometimes > >> > it > >> > was even better to disable SMP) > >> > > >> > 2- I was expecting a much better performance for SMP enabled (no. of > >> > cores > >> > = no. of schedulers) in R15 than in R12, since the schedulers' logic > has > >> > changed a lot from R12 to R15. But the results were more or less the > >> > same. > >> > I was thinking since I had a multithreaded db like MySql, a multiple > >> > db-connections driver as Emysql, and SMP with multiple schedulers with > >> > their own run queues, I could get better results than in R12 where > there > >> > is > >> > only one run-queue and multiple schedulers (more lock contention). > >> > Is it maybe because the processes are just doing IO and nothing > >> > CPU-intensive? > >> > > >> > 3- I realized that CPU usage in R15 SMP enabled is twice higher than > the > >> > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, > and > >> > I > >> > can see that the beam process is using up to 75-80% of the CPU, the > rest > >> > is > >> > the mysql daemon process. Is it because in R15 the scheduling > algorithm > >> > has > >> > become more complex? > >> > > >> > *Mnesia* > >> > * > >> > >> > * > >> > I have only one table (as I said before) created with the attribute > >> > disc_copies, and it is stored only in one node. The whole table fits > in > >> > RAM, and I tried to keep it simple by controlling the size of the > table > >> > during my tests, in order not exceed RAM capacity > >> > > >> > 4- For the write tests in Mnesia I see the same behavior as in Mysql. > >> > However, since the data are all loaded in RAM, the session duration is > >> > shorter, and the gain in performance between SMP enabled and disabled > is > >> > not as huge as in Mysql. > >> > > >> > 5- For the read tests, I have two observations. First, the dirty reads > >> > are > >> > 10 times faster than the reads that use transactions. Second, I have > >> > better > >> > results for SMP disabled (twice faster, though in both cases the > session > >> > duration is in order of millisecods, 70 msec vs 30 msec). > >> > > >> > 6- Same as in Mysql, no difference in performance between R12 and R15 > >> > with > >> > SMP enabled, for both read and write tests. > >> > > >> > If you have read so far, and have a comment on my results, you are > >> > welcome. Especially about the no difference btw R12 and R15 and the > high > >> > CPU usage in R15 when the applications are IO-bound. > >> > > >> > Thanks, > >> > Erisa > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > Hello > > I'm very interested also in the behavior of Erlang applications in an > SMP environment (so also my post "Strange observation running in an > SMP environment." on this mailing list). Is the code that you have > created somewhere available so that I can also play with it? In case > not: are you willing to share it with the community? > > Thanks > > -- > Best regards, > Ronny > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sun Jun 24 20:37:12 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Sun, 24 Jun 2012 20:37:12 +0200 Subject: [erlang-questions] SMP for IO-bound applications In-Reply-To: References: <4fe42c05.7067980a.1893.6fb7@mx.google.com> Message-ID: <746752638638550424@unknownmsgid> Hi Erisa, In R15 we try to spin schedulers even more in an effort to reduce their sleep. This is especially useful when we have jitter in runnability among processes. If a queue is empty a scheduler is forced to sleep, the latency in wake up is something we want avoid, hence we spin. The system is at its best when it is fully saturated. Another feature in R15 is delayed dealloc. Memory deallocation in the wrong thread is instead scheduled to be deallocated. This feature lessens lock contentions on memory allocators and improves scalability but will in all likelihood also increase CPU utilization (I have seen this on other measurements). In the future the schedulers will probably take this into account among other constraints and rules. If you want to investigate this further I recommend using a gprof-enabled vm to examine where the time is spent. Regards, Bj?rn-Egil 24 jun 2012 kl. 20:12 skrev Erisa Dervishi : Yes sure, even though I am using Tsung (a load generator tool written in Erlang) to generate my load and get some results back. I have added a new Tsung plugin for the purpose of my tests. I am a bit busy right now but If you are interested I will share what I have done so far. However, I would really like to have an opinion from an expert about my comments in here On Sun, Jun 24, 2012 at 10:47 AM, Ronny Meeus wrote: > On Fri, Jun 22, 2012 at 1:51 PM, Erisa Dervishi > wrote: > > Hey, thank you for your reply, but I use the same mysql driver even for > my > > evaluations in Erlang OTP R12 and do not get the same CPU usage (for > R12 is > > up to 45%). And the tests are exactly the same (same load). I have also > > checked the message queue of the process that supervises the connections, > > and it looks like there isn't a bottleneck in there (the queue length is > > most of the time 0 and up to 4 messages at most) > > And even for Mnesia the CPU usage (though low), doubles from R12 to R15 > with > > smp enabled from 6% to 12% for write tests > > > > > > > > On Fri, Jun 22, 2012 at 10:22 AM, be.dmitry@REDACTED < > be.dmitry@REDACTED> > > wrote: > >> > >> I think that much of CPU usage is created by your MySQL driver. > >> You should have a look at driver details. There may be copying of the > >> query you send to MySQL between your benchmarking process and the real > >> process that owns the socket. The same with result - parsing response > and > >> sending result back to benchmarking process. There may be a bottleneck > in > >> choosing MySQL connection from connection pool if it is made through > >> supervisor. > >> Try to use etop to see what functions are at the bottlenecks. > >> I mean your io-intensive test may be just the test showing driver > >> bottlenecks and not the problems in smp. > >> > >> > >> > >> > >> ----- Reply message ----- > >> From: "Erisa Dervishi" > >> To: > >> Subject: [erlang-questions] SMP for IO-bound applications > >> Date: Thu, Jun 21, 2012 23:28 > >> > >> > >> One additonal comment I forgot to mention: > >> > >> For all Mnesia tests, the CPU usage is low in general. Only for the > >> transactional read test in R15 with SMP enabled i get a CPU usage of > 70%. > >> From the load generator tool, i call the read(N) function that is showed > >> below: > >> > >> exec_read(0) -> done_reading; > >> > >> exec_read(N) -> > >> > >> Id=random:uniform(7000000), > >> > >> %%io:format("~p~n",[Id]), > >> > >> %%[_Row] = mnesia:dirty_read({subscriber,Id}), > >> > >> Fun = fun() -> > >> > >> mnesia:read({subscriber,Id}) > >> > >> end, > >> > >> {atomic,[Row]}=mnesia:transaction(Fun), > >> > >> %io:format("~p~n",[Row]), > >> > >> exec_read(N-1). > >> > >> read(N) -> > >> > >> {A1,A2,A3} = now(), > >> > >> random:seed(A1, A2, A3), > >> > >> exec_read(N). > >> > >> > >> I am using mpstat while my tests are running and beside the CPU usage, > >> another difference of this test from the other Mnesia tests is that the > >> syscl (system calls) column of mpstat has a value which is 20 times > higher > >> than the other test cases. > >> > >> > >> > >> On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi > >> wrote: > >> > >> > Hi, > >> > As part of my studies, I have recently been doing some performance > >> > evaluations on Erlang SMP improvements for IO-bound applications. > >> > > >> > The applications I considered for the evaluation were Emysql driver ( > >> > https://github.com/Eonblast/Emysql) and Mnesia. > >> > I have created an Erlang module for performing reads/writes from/to > >> > Mysql > >> > DB through Emysql driver, and another Erlang module for communicating > >> > with > >> > Mnesia. > >> > Both the modules have two methods write(nr_records) which writes > >> > nr_records records to the database (MySQL, or Mnesia), and > >> > read(nr_records) > >> > which reads nr_records random records from the db. (both the reads and > >> > writes are consecutive, not in chunks) > >> > I have a load generator tool which simulates simultaneous requests to > >> > both > >> > the modules and gathers performance statistics. I generate as much > load > >> > as > >> > needed to stress the applications (near the saturation point) > >> > The metrics I uses are the throughput (no. of sessions/sec), and the > >> > session duration. A session is a call to write(nr_records) > >> > or read(nr_records) function depending on the test case (read or > write). > >> > So > >> > if I call write(500), I measure the duration of a session that does > 500 > >> > hundred inserts into the database. > >> > > >> > The tests I run have these parameters: > >> > > >> > A)Type of operations: 1- Reads 2-Writes > >> > I have just one table with 20 fields, and the reads and writes are > just > >> > select and insert operations in that table > >> > > >> > B) I/O applications: 1- Emysql 2-Mnesia > >> > > >> > C) SMP parameters: > >> > 1- SMP enabled, no. of schedulers = no. of cpu cores > >> > 2-SMP enabled, no. of schedulers = 1 > >> > 3- SMP disabled > >> > > >> > D) Erlang OTP versions: > >> > 1- Erlang R12B (The SMP has only one run-queue and multiple > schedulers) > >> > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler) > >> > > >> > The tests included all possible combinations from A,B,C,D. They were > run > >> > in Solaris 10 x86 (a 16 cores machine). > >> > In general these were the results I got: > >> > > >> > *Emysql driver:* > >> > >> > 1- There is a a big difference in performance between SMP enabled and > >> > disabled in both the Erlang versions (R12 and R15) for both read and > >> > write > >> > tests. So I can say that SMP rocks! However, you have to have enough > >> > load > >> > to achieve that (for low load I could not see any difference, > sometimes > >> > it > >> > was even better to disable SMP) > >> > > >> > 2- I was expecting a much better performance for SMP enabled (no. of > >> > cores > >> > = no. of schedulers) in R15 than in R12, since the schedulers' logic > has > >> > changed a lot from R12 to R15. But the results were more or less the > >> > same. > >> > I was thinking since I had a multithreaded db like MySql, a multiple > >> > db-connections driver as Emysql, and SMP with multiple schedulers with > >> > their own run queues, I could get better results than in R12 where > there > >> > is > >> > only one run-queue and multiple schedulers (more lock contention). > >> > Is it maybe because the processes are just doing IO and nothing > >> > CPU-intensive? > >> > > >> > 3- I realized that CPU usage in R15 SMP enabled is twice higher than > the > >> > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%, > and > >> > I > >> > can see that the beam process is using up to 75-80% of the CPU, the > rest > >> > is > >> > the mysql daemon process. Is it because in R15 the scheduling > algorithm > >> > has > >> > become more complex? > >> > > >> > *Mnesia* > >> > * > >> > >> > * > >> > I have only one table (as I said before) created with the attribute > >> > disc_copies, and it is stored only in one node. The whole table fits > in > >> > RAM, and I tried to keep it simple by controlling the size of the > table > >> > during my tests, in order not exceed RAM capacity > >> > > >> > 4- For the write tests in Mnesia I see the same behavior as in Mysql. > >> > However, since the data are all loaded in RAM, the session duration is > >> > shorter, and the gain in performance between SMP enabled and disabled > is > >> > not as huge as in Mysql. > >> > > >> > 5- For the read tests, I have two observations. First, the dirty reads > >> > are > >> > 10 times faster than the reads that use transactions. Second, I have > >> > better > >> > results for SMP disabled (twice faster, though in both cases the > session > >> > duration is in order of millisecods, 70 msec vs 30 msec). > >> > > >> > 6- Same as in Mysql, no difference in performance between R12 and R15 > >> > with > >> > SMP enabled, for both read and write tests. > >> > > >> > If you have read so far, and have a comment on my results, you are > >> > welcome. Especially about the no difference btw R12 and R15 and the > high > >> > CPU usage in R15 when the applications are IO-bound. > >> > > >> > Thanks, > >> > Erisa > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > Hello > > I'm very interested also in the behavior of Erlang applications in an > SMP environment (so also my post "Strange observation running in an > SMP environment." on this mailing list). Is the code that you have > created somewhere available so that I can also play with it? In case > not: are you willing to share it with the community? > > Thanks > > -- > Best regards, > Ronny > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Mon Jun 25 01:16:52 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 24 Jun 2012 17:16:52 -0600 Subject: [erlang-questions] Large sparse/dense matrix server Message-ID: Greetings, I have mostly a mental question as, although been using erlang for a few minor servers for a few years now, this is my first 'large' project that does not seem to fit functional programming perfectly well for efficiency reasons, so I am curious at any and all thoughts that may help the design of this system. I have been writing an Erlang server that handles multiple very large matrices. By very large they extend to an unbounded distance, but 'chunks' of it are only generated upon initial access of said chunk. Each cell is an integer of 32 bits in size (although I may need to expand it to 48 or 64 bits soon). Effectively the cells will be read quite often in large neighborhoods of chunks and sent out. Occasionally there will be a single cell change requested by a client, but this cell change may cause a ripple of other nearby cell changes and updates need to be sent out to clients that are watching a chunk for updates. They need to be persisted to disk, but it is not imperative that it is flushed on every change, even being cached for for a couple of minutes would not be bad (although no more than a few seconds is best). Right now I have it set up like this: - If a client watches an area: - - All chunks in such an area are loaded - - client gets a full dump of all the chunk data in the area - When a chunk is loaded, a new process is started and it loads a file from disk that contains its gzipped saved state of its entire chunk area - - If no state file exists then it generates one from its initial generation function, then stores that data to disk so the costly generation is not done again - - Currently a chunk is 16x16x16=4096 cells in size and multiple chunks are stored in a single gzipped file to prevent tiny file creation explosion on disk - - If nothing is watching a chunk anymore, then after a time delay based on various things (especially proximity to other nearby loaded chunks and if a time for an inside cell is soon to expire) then the process will do a full flush of its data to disk then die. - If a cell is requested to change then the change request is sent to the chunk handling process - - The chunk handling process runs a few things in turn when a cell request change is set: - - - That the client is authorized to make a change to that cell (a 3d range check for permissions in another process) - - - Pre-Listeners for that cell are called to allow them to veto a change - - - The cell is changed, the specific integer value of the cell, the first 16 bits are used as an index lookup for a module name - - - - The module for that new cell type is called, it returns a full cell sized integer of what needs to be stored or if it needs to deny the change - - - - The module for that new cell type may also set extra data for that chunk that is a key/value with the key being the {Matrix, X, Y, Z} of the cell changed - - - - Any pending timers in the scheduler for the cell are killed. - - - Post-Listeners for that cell are called that allow them to react to that change, potentially making other changes to other cells following this same pattern - - - - The 'client' for rippling changes is basically just {'system', Client} to signify that it is a system caused change, but initially started by whatever Client - A scheduler, sometimes a cell wants to update itself, sometimes once every second, perhaps once in 5 seconds then again in a minute then never again until its state changes again or so - - When time elapses, which may not happen 'instantly' when it wants it to happen but 'soon', the cell id is looked up and its module update function is called with the coordinates, full cell value, delta of how 'late' the callback is, and any data given to the scheduler to be passed back in. Pre-listeners are synchronous callback funs, they are supposed to be short and very simple checks, but of course may still require a call to other processes, authentication, etc... so it has not yet always gone as planned. Post-listeners are PIDs that messages are sent to, to enforce that they must happen 'later', although I am about to change that to submit an update to the scheduler instead. There are a few other minor things, but this is the big part. Right now it is running in two Erlang VM's on one 6-core server with 16 gigs of ram, 4.2TB storage, and it is quite not efficient (it grew up in this way, needs a redesign...). First change I was thinking of was potentially using riak-core to handle creation and management of the chunk processes, since I do want to grow this server from two VM's on one computer to multiple VM's across multiple computers. My main issue with this is that I would prefer 'nearby' chunks to try to be loaded on the same node, and far away chunks to be on different nodes, some areas in the entire matrix may have massive areas loaded so that would need to be distributed as well as possible. And of course just as timers/schedulers elapse new chunks will need to be loaded to do a rather miniscule amount of processing then to unload again. Second change, to facilitate the non-central storage is to use Riak (unless there is something better suited for this purpose) as the storage. A Key/value row in Riak would hold an entire chunk, plus some extra handling data as the value, and the key would just be the chunk coordinates, a 4-tuple {MatrixLayer, X/16, Y/16, Z/16} where MatrixLayer is just one of multiple different unbounded matrices of some erlang term (likely atom, sometimes integers, but may be something else for others later on). Third change (maybe), as costly as it is to do initial generation, maybe I only want to store a chunk of there were any changes in it, meaning I may want to prevent initial updates unless caused by a client, disk space has not yet been an issue however, no where close actually. Plus who knows what else. Preferably the system needs to always remain up, and hot code loading has been quite useful here (not been using release updates, probably should, but right now this is still a 'dev' system so I actually do my programming on the live system, later on when actually rolled out for its heavy usage that will change). I have even thought of something as radical as doing a row per cell in riak, I could put the access data and all in the function for it, it would need read access to its neighbor cells, but not write (as writes would just be pushed for 'later'), and I 'think' I can do that in riak using its mapreduce interfaces, but not checked, and I need it to be quite fast. However it sounds costly in that it would be difficult to batch cell updates together as I do now (especially at generation time). I need the time guarantee of the system to be within a few seconds response time for the clients, timer callbacks, pretty much everything for a very high percentile. And I can expand out to more servers, however communication overhead seems like it may get substantial rather quickly based on past projects with Riak. So what would be the best design for such a system? And the system itself has a lot of parts and a lot of rather nasty interactions, I am not opposed to a complete rewrite if it will help -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Mon Jun 25 01:56:40 2012 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 25 Jun 2012 00:56:40 +0100 (BST) Subject: [erlang-questions] Yaws security alert - Yaws 1.93 In-Reply-To: <20120624032531.GB9534@k2r.org> Message-ID: <8ebec6f7-3d0b-41b5-b575-4ef4297c69f0@knuth> The original random module was never meant to be cryptographically safe, it was just a simple and reasonably fast PRNG. The bug, if there is one, was that this was never explicitly stated in the documentation. And as Kenji said even the MT are not cryptographically safe either, and were never meant to be, even though they are much better PRNG. The best solution is Richard's to create an explicit module for cryptographically safe random numbers. And include a better "normal" PRNG than random, for example Kenji's. Which can be made drop-in compatible with the existing one. Robert ----- Original Message ----- > FYI: > > * Mersenne Twister PRNGs are NOT cryptographically safe either, > although > the random number generation period is much much longer (approx. > 2^43 > on AS183, (2^19937) - 1 for SFMT) and the state space is far less > easier to be exploited. > > * sfmt-erlang is now runable on non-NIF environment (though it's > slow) > > * I've been working on a lightweight variant of MT called TinyMT > (period: > (2^127) - 1, internal state: 28 bytes), including compatibility > functions to the random module, and is capable of generating ~2^58 > different RNG streams. It's at > https://github.com/jj1bdx/tinymt-erlang > > ++> Tuncer Ayaz [2012-06-21 12:47:46 +0200]: > > It should probably be replaced with Kenji's sfmt-erlang or an > > implementation of (C)MWC. > > > > https://groups.google.com/group/comp.soft-sys.math.mathematica/msg/95a94c3b2aa5f077 > > Kenji Rikitake > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Mon Jun 25 03:27:03 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 24 Jun 2012 18:27:03 -0700 Subject: [erlang-questions] [ANN] CloudI 1.0.0 Released with Load Tests Message-ID: <4FE7BE67.5030906@gmail.com> Download 1.0.0 from http://sourceforge.net/projects/cloudi/files/latest/download CloudI (http://cloudi.org) development has reached a successful beta, after some testing, fixes and release candidates. If you find any lingering issues or improvements, please report them. I have done some load tests as part of the 1.0.0 release so that the performance characteristics are more transparent, as an open source project. The load tests test a scenario with incoming HTTP traffic exercising the supported programming languages independently, when each programming language is given a single OS process with no threads (in the case of Erlang, it is a single Erlang process). I understand that people like to minimize load testing because they do not feel it is applicable to their unique software stack, but I definitely see establishing a baseline for performance as critical, for further improvements. The nature of this particular set of load tests provides a view of the relative response rates coming from separate programming languages with the same dynamic XML requests (over HTTP). If you want to find out more about the load tests, you can look at http://cloudi.org/faq.html#5_LoadTesting or within the release in src/tests/http_req/loadtest/results/201206_20k/. Thanks, Michael From icfp.publicity@REDACTED Mon Jun 25 09:12:03 2012 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Mon, 25 Jun 2012 09:12:03 +0200 Subject: [erlang-questions] ICFP Student Research Competition Message-ID: ===================================================================== Student Research Competition Associated with the The 17th ACM SIGPLAN International Conference on Functional Programming (ICFP 2012) and affiliated events http://www.icfpconference.org/icfp2012/src.html Copenhagen, Denmark, Sep 9-15, 2012 ===================================================================== This year ICFP will host a Student Research Competition where undergraduate and postgraduate students can present posters. The SRC at the ICFP 2012 consists of three rounds: - Extended abstract round: All students are encouraged to submit an extended abstract outlining their research (800 words). - Poster session at ICFP 2012: Based on the abstracts, a panel of judges will select the most promising entrants to participate in the poster session which will take place at ICFP. Students who make it to this round will be supported to attend the conference, to a maximum of $500 for travel and housing. If your total costs are higher than these $500 your conference fee may be waived too. In the poster session, students will have the opportunity to present their work to the judges, who will select three finalists in each category* (graduate/undergraduate) to advance to the next round. - ICFP presentation: The next round consists of an oral presentation at the ICFP to compete for the final awards in each category. ** Prizes ** Both the top three graduate and the top three undergraduate contestants will receive prizes of $500, $300, and $200, respectively. All six winners will receive award medals and a two-year complimentary ACM student membership, including a subscription to ACM?s Digital Library. The names of the winners will be posted on the SRC web site. The winners in each category will be invited to participate in the ACM SRC Grand Finals, an on-line round of competitions among the winners of other conference-hosted SRCs. Grand Finalists and their advisors will be invited to the Annual ACM Awards Banquet for an all-expenses-paid trip, where they will be recognized for their accomplishments along with other prestigious ACM award winners, including the winner of the Turing Award (also known as the Nobel Prize of Computing). The top three graduate Grand Finalists will receive an additional $500, $300, and $200. Likewise, the top three undergraduate Grand Finalists will receive an additional $500, $300, and $200. All six Grand Finalists will receive Grand Finalist certificates. ** Eligibility ** The SRC is open to both undergraduate and graduate students. Upon submission, entrants must be enrolled as a student at their university. The abstract must describe the student?s individual research and must be authored solely by the student. If the work is collaborative with others and/or part of a larger group project, the abstract should make clear what the student?s role was and should focus on that portion of the work. The extended abstract must not exceed 800 words and must not be longer than 2 pages. The reference list does not count towards these limits. To submit an abstract, please register through the submission page and follow the instructions. Abstracts submitted after the deadline may be considered at the committee's discretion, but only after decisions have been made on all abstracts submitted before the deadline. If you have any problems, don't hesitate to contact the competition chair. More information about the submission process can be found online at: http://www.icfpconference.org/icfp2012/src.html ** Important Dates ** Deadline for submission: June 29th Notification of acceptance: July 8th ** Selection Committee ** Koen Claessen, Chalmers University of Technology Robby Findler (ICFP Program chair), Northwestern University Ken Friis Larsen, IT University of Copenhangen Jacques Garrigue, Nagoya University Doaitse Swierstra (Chair), Utrecht University From robert.virding@REDACTED Mon Jun 25 12:17:28 2012 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 25 Jun 2012 11:17:28 +0100 (BST) Subject: [erlang-questions] JavaScript compiler for Erlang In-Reply-To: <20120622171508.GA332@aluminum.motivity.ca> Message-ID: <3c863604-27d1-46d9-b763-c1c2f8e7a3d0@knuth> Sorry, sent this locally first time: Running Javascript on the BEAM is not a trivial problem, not at least if you want to do it in pure erlang. The BEAM is specially targeted for Erlang and it is not a general purpose VM. It can do many things much more efficiently just because it IS a special purpose VM. The main is problem is that js has mutable data which the beam does not directly support, so it has to be be emulated. I have done a Lua implementation which has exactly the same problem. Doing global data does cost, unfortunately. I chose Lua as it was a simpler language but JS would be fun. An alternative would be to use NIFs to try an emulate mutable data. You could not most likely not use the BEAM process heaps as destructive updates would seriously confuse the BEAM memory management. And if you start using NIFs then maybe it is better to connect to a "real" JS as is done in erlv8. It really depends on how you would like to use JS. Robert ----- Original Message ----- > On Thu, Jun 21, 2012 at 11:12:26AM -0400, Ingo Jaeckel wrote: > } Just out of curiosity: Do you want to compile JavaScript code to > } Erlang or Erlang to JavaScript? > > I want to run Javascript in the beam emulator. > > } In either case - Why? What is the application behind this? > > At the moment I am working with VoiceXML which incorporates > ECMAScript. > Although they call a VoiceXML interpreter a "voice browser" if you > consider where it is typically used in the network it is a server > side > application. > > } What about https://github.com/beamjs/erlv8/ and > http://beamjs.org/? > > Yes, there are a couple bindings to external engines. Erlyjs was > taking the approach I would prefer, interpreting the ECMAscript on > the beam machine. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From info@REDACTED Mon Jun 25 14:02:09 2012 From: info@REDACTED (info) Date: Mon, 25 Jun 2012 14:02:09 +0200 Subject: [erlang-questions] How to restart odbc ? Message-ID: <201206251402079054627@its3.ch> ? How can I supervise the ODBC application in my personal application in order to restart it automatically in case of crash ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastien.tandel@REDACTED Mon Jun 25 15:04:48 2012 From: sebastien.tandel@REDACTED (=?ISO-8859-1?Q?S=E9bastien_Tandel?=) Date: Mon, 25 Jun 2012 10:04:48 -0300 Subject: [erlang-questions] Obtaining current frame bindings ? Message-ID: Hi all, I have written a simple fault injection server in order to test my code. I can inject exception, sleep and other some minor perturbations into my code (at some chosen places, let's call them test points) at runtime and it's configured thanks to a simple configuration file written in json. These perturbations are injected after meeting some conditions. For example, right now, it's based on a counter per test point (if enabled). It this counter reaches zero, there is an action triggered (exception, sleep, ...). If this mechanism already provides me lot of testing case, I would like though extend it to be able to write conditions which are code injection like, for example, "if N == [].". I'm able to use correctly erl_scan and erl_parse to obtain the ast. However, when calling erl_eval:exprs(), I need the bindings of the current function in which this code runs and I've not been able to find out a way to obtain these bindings. Does anyone know whether it's possible? And if it is, how can I obtain it? :) Regards, Sebastien Tandel -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Mon Jun 25 15:16:05 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 25 Jun 2012 06:16:05 -0700 (PDT) Subject: [erlang-questions] How to restart odbc ? In-Reply-To: <201206251402079054627@its3.ch> References: <201206251402079054627@its3.ch> Message-ID: <842ddcd8-6049-4d30-9b93-e0b697c61a44@googlegroups.com> I would expect that 'odbc:start(permanent)' would do what you need... the default is 'temporary'. hth, /s -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Mon Jun 25 15:17:53 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Mon, 25 Jun 2012 09:17:53 -0400 Subject: [erlang-questions] JavaScript compiler for Erlang In-Reply-To: <3c863604-27d1-46d9-b763-c1c2f8e7a3d0@knuth> References: <3c863604-27d1-46d9-b763-c1c2f8e7a3d0@knuth> Message-ID: <82EA4C4F-4F0B-47B9-8FCB-AC492A1820C8@dieswaytoofast.com> Vance: On a bit of a tangent here, if the JS part of the VoiceBrowser consists of "external" code that is subject to constant change, then, well, you might be a bit out of luck. On the other hand, if this is all pre-defined stuff, you might want to consider just rewriting most of the VXML interpreter in erlang. We did something similar a while back, and it worked pretty remarkably well. You tend to end up with a fairly small (and somewhat tightly controllable) set of core JS stuff that you can farm out to a NIF, or just leave as JS if you so choose... Cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Blog | Twitter | Google+ On Jun 25, 2012, at 6:17 AM, Robert Virding wrote: > Sorry, sent this locally first time: > > Running Javascript on the BEAM is not a trivial problem, not at least if you want to do it in pure erlang. The BEAM is specially targeted for Erlang and it is not a general purpose VM. It can do many things much more efficiently just because it IS a special purpose VM. The main is problem is that js has mutable data which the beam does not directly support, so it has to be be emulated. I have done a Lua implementation which has exactly the same problem. Doing global data does cost, unfortunately. I chose Lua as it was a simpler language but JS would be fun. > > An alternative would be to use NIFs to try an emulate mutable data. You could not most likely not use the BEAM process heaps as destructive updates would seriously confuse the BEAM memory management. And if you start using NIFs then maybe it is better to connect to a "real" JS as is done in erlv8. > > It really depends on how you would like to use JS. > > Robert > > > ----- Original Message ----- >> On Thu, Jun 21, 2012 at 11:12:26AM -0400, Ingo Jaeckel wrote: >> } Just out of curiosity: Do you want to compile JavaScript code to >> } Erlang or Erlang to JavaScript? >> >> I want to run Javascript in the beam emulator. >> >> } In either case - Why? What is the application behind this? >> >> At the moment I am working with VoiceXML which incorporates >> ECMAScript. >> Although they call a VoiceXML interpreter a "voice browser" if you >> consider where it is typically used in the network it is a server >> side >> application. >> >> } What about https://github.com/beamjs/erlv8/ and >> http://beamjs.org/? >> >> Yes, there are a couple bindings to external engines. Erlyjs was >> taking the approach I would prefer, interpreting the ECMAscript on >> the beam machine. >> >> -- >> -Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cstomi.levlist@REDACTED Mon Jun 25 15:23:59 2012 From: cstomi.levlist@REDACTED (Tamas.Cseke ) Date: Mon, 25 Jun 2012 15:23:59 +0200 Subject: [erlang-questions] run_erl: Could not open pty master Message-ID: <4FE8666F.8020401@gmail.com> Hi, We can start run_erl only with root user (it seems we have a permission problem otherwise) R15B02 Here is strace output http://pastebin.com/Vp3gcaNH After a lot of debugging it seems grantpt is failing. Any idea is appreciated. Thanks, Tamas From essen@REDACTED Mon Jun 25 18:12:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 25 Jun 2012 18:12:23 +0200 Subject: [erlang-questions] SSL certificate's subject too long Message-ID: <4FE88DE7.20401@ninenines.eu> Hey, I'm running into the exact issue described here: http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html The certificate I have has a too long Subject line and Erlang just fails when trying to load it. This is a RapidSSL certificate, with a CA. Not sure I can just modify the subject directly while keeping it valid (don't know how anyway). Does anyone know how I could manage to use this certificate? If I need to patch OTP, any pointers as to where this fails is more than welcome. Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines From ingela.andin@REDACTED Mon Jun 25 22:27:10 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 25 Jun 2012 22:27:10 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: <4FE88DE7.20401@ninenines.eu> References: <4FE88DE7.20401@ninenines.eu> Message-ID: Hi! 2012/6/25 Lo?c Hoguin : > Hey, > > I'm running into the exact issue described here: > http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html > > The certificate I have has a too long Subject line and Erlang just fails > when trying to load it. This is a RapidSSL certificate, with a CA. Not sure > I can just modify the subject directly while keeping it valid (don't know > how anyway). > > Does anyone know how I could manage to use this certificate? If I need to > patch OTP, any pointers as to where this fails is more than welcome. > > Thanks. > The ecertfile is a backwards compatibility error code that you will get if ssl has a problem reading the certfile. It could be a file-error or a file-format error, or a bug in OTP application public_key. So if you want to pinpoint the error you can do: {ok, PemBin} = file:read_file("Cert.pem"). PemEntries = public_key:pem_decode(Pembin). public_key:pem_entry_decode(hd(PemEntries)). If your cert is breaking the ASN-1 spec you have a problem. To be able to fix the certificate you need to change the certificate and sign the new certificate with the private key that signed your certificate. If you have access to the key it is doable. I am not familiar with RapidSSL but if they are breaking the specs I think they should create a new and valid certificate for you. You could workaround it by changing the public_key applications ASN1-spec but it feels wrong... Regards Ingela Erlang/OTP team - Ericsson AB From jameschurchman@REDACTED Tue Jun 26 00:55:44 2012 From: jameschurchman@REDACTED (James Churchman) Date: Mon, 25 Jun 2012 23:55:44 +0100 Subject: [erlang-questions] nested variable assignment : bug or the expected behaviour ? Message-ID: Small syntax related thing If i have a function like : f()->[J=100,J]. Or put in the shell: {J=200,J}. Or erlang:'+'(J=10,J). Or any other combination of where i try to set a variable nested inside somewhere else, it will say "variable 'J' is unbound" .. is this the expected behaviour ? ( if i do bind the value before tho it will it will fail with no match of right hand side value XX which is to be expected ) I only ask as i want to type : [ {1, N1=now() ,0 }, {2, N2=now() ,N1 }, {3, N3=now() ,N2 } ] I realise I could generate all the NXX now's before, or write a function to put each now result in the flowing tuple but its just for some simple test code and the behaviour was not as i wanted :-) James From michael.eugene.turner@REDACTED Tue Jun 26 05:04:28 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 26 Jun 2012 12:04:28 +0900 Subject: [erlang-questions] nested variable assignment : bug or the expected behaviour ? In-Reply-To: References: Message-ID: Nice question. I'd like to know the answer myself. My guess: committing to first-lexical-use-of-matching for variable binding in expressions reduces opportunities to optimize the expression, since it implies an order of evaluation that might not be optimal. This can't (quite) be the explanation though, since Erlang is so lisplike in its approach to the question "What's an expression?" -michael turner On Tue, Jun 26, 2012 at 7:55 AM, James Churchman wrote: > Small syntax related thing > > If i have a function like : > > f()->[J=100,J]. > > Or put in the shell: > {J=200,J}. > > Or > > erlang:'+'(J=10,J). > > Or any other combination of where i try to set a variable nested > inside somewhere else, it will say "variable 'J' is unbound" .. is > this the expected behaviour ? > > ( if i do bind the value before tho it will it will fail with no match > of right hand side value XX which is to be expected ) > > I only ask as i want to type : > > [ > {1, N1=now() ,0 }, > {2, N2=now() ,N1 }, > {3, N3=now() ,N2 } > ] > > I realise I could generate all the NXX now's before, or write a > function to put each now result in the flowing tuple but its just for > some simple test code and the behaviour was not as i wanted :-) > > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue Jun 26 05:43:33 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 26 Jun 2012 15:43:33 +1200 Subject: [erlang-questions] nested variable assignment : bug or the expected behaviour ? In-Reply-To: References: Message-ID: On 26/06/2012, at 10:55 AM, James Churchman wrote: > Small syntax related thing > > If i have a function like : > > f()->[J=100,J]. > > Or put in the shell: > {J=200,J}. The question came up many years ago about how precisely evaluation order in Erlang was or should be defined. It's not just variable bindings: if you write f(X)+g(X) and both f(X) and g(X) are certain to raise an exception, which one do you get? One model: Scheme. Scheme tries very hard to leave the execution order undefined. If I write (list (f X) (g X)) in Scheme I have no idea which function will be called first. Another model: Common Lisp. Common Lisp defines left-to- right execution. My recollection was that with exceptions and message passing and I/O in Erlang, the decision was to follow Common Lisp rather than Scheme. Indeed, [io:write(a),io:write(b)] does write ab instead of the more natural ba (think about how lists work to see why I said that), and so does {io:write(a),io:write(b)}. And nested variable bindings per se are not rejected by Erlang. Try > io:write(X=a), X. Although many people feel that this is very poor practice. Erlang is stricter than it needs to be to ensure that the source code has a feasible semantics, but not strict enough to improve readability as much as some people (including me) would like. What exactly is wrong with f() -> J = 100, [J,J]. or J = 200, {J,J}? > > Or any other combination of where i try to set a variable nested > inside somewhere else, it will say "variable 'J' is unbound" .. is > this the expected behaviour ? Yes. But it's not "nested inside somewhere ELSE"; that _is_ allowed. It's "nested inside the SAME construct other than a serial clause". > [ > {1, N1=now() ,0 }, > {2, N2=now() ,N1 }, > {3, N3=now() ,N2 } > ] This is tricky enough to deserve a function with a name and a comment to generate it. From rexxe98@REDACTED Tue Jun 26 07:52:40 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 25 Jun 2012 22:52:40 -0700 Subject: [erlang-questions] gen_server durable queue Message-ID: Hey all, I'm wondering if anyone has done any work on building a gen_server replacement which uses a durable queue. I'm looking to build a command bus and I cannot lose messages if the node goes down. I was thinking of using Rabbit, but it seems like overkill for me and just another component I have to worry about. Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Tue Jun 26 08:11:54 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Tue, 26 Jun 2012 08:11:54 +0200 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: > I'm wondering if anyone has done any work on building a gen_server > replacement which uses a durable queue. ?I'm looking to build a command bus > and I cannot lose messages if the node goes down. ?I was thinking of using > Rabbit, but it seems like overkill for me and just another component I have > to worry about. Did you look at ZeroMQ? Look at the last reliability pattern, called titanic in the guide (http://zguide.zeromq.org). 0MQ is a library hence it will not seem like an overkill. That is of course just one reason to use 0MQ for others read the guide, it is actually very much worth it. From bourinov@REDACTED Tue Jun 26 09:16:27 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 26 Jun 2012 11:16:27 +0400 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: <488F7C08-2A5E-4D15-9B66-0F9B9A3BC58C@gmail.com> Hi Andrew, My xp with rabbit shows that you don't have to worry about it after installation. Go with rabbit. Sent from my iPad On 26.06.2012, at 9:52, Andrew Berman wrote: > Hey all, > > I'm wondering if anyone has done any work on building a gen_server replacement which uses a durable queue. I'm looking to build a command bus and I cannot lose messages if the node goes down. I was thinking of using Rabbit, but it seems like overkill for me and just another component I have to worry about. > > Thanks, > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Tue Jun 26 09:19:34 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 26 Jun 2012 11:19:34 +0400 Subject: [erlang-questions] How should I create minor patch to OTP? Message-ID: Hi. I'm trying to follow instructions on https://github.com/erlang/otp/wiki/submitting-patches In short: they don't work out of the box. You ask to use "maint" branch for minor improvement, but I really don't understand what exactly should I do to make patch properly. Trying to follow supplied instructions end in "CONFLICT" while making git rebase Maybe it would be better to add simple, but working snippet of code, how exactly you want to prepare git repo? From henrik@REDACTED Tue Jun 26 11:13:52 2012 From: henrik@REDACTED (Henrik Nord) Date: Tue, 26 Jun 2012 11:13:52 +0200 Subject: [erlang-questions] How should I create minor patch to OTP? In-Reply-To: References: Message-ID: <4FE97D50.2020507@erlang.org> Hi You should not have to do a rebase if you are already on the maint branch. I think I understand why you are confused though. It says on the github wiki that you should pull from 'master' and then do 'git rebase upstream master' if you have been working on your patch for a long time. This is based on the assumption that you are actually working on a patch for the 'master' branch. If you are working on a 'maint' based patch, then you should substitute 'master' with 'maint' in the instructions on the wiki: git clone git@REDACTED:your_username/otp.git git remote add upstream git://github.com/erlang/otp.git git checkout maint and continue as instructed in the wiki and if you work on the patch along time and want to rebase it upon the latest version on 'maint' git fetch upstream git checkout new_feature_name git rebase upstream/maint If this is not your problem, and maybe you are already based on 'master' and want to rebase you branch upon 'maint' I suggest the following: checkout your branch. open gitk on your branch. in the terminal reset your branch to the tip of the 'maint' branch (git reset --hard maint) then rightklick on your commit(s) in gitk, and chose "cherry-pick this commit" This will take your changes and apply them on a fresh version of 'maint' without requiring you to handle any merge/rebase conflicts! I hope this have been helpful in some way. Do not hesitate to send me any questions regarding patches or the preparation of them! Thank you for your future contributions! On 06/26/2012 09:19 AM, Max Lapshin wrote: > Hi. > > I'm trying to follow instructions on > https://github.com/erlang/otp/wiki/submitting-patches > > In short: they don't work out of the box. > > > You ask to use "maint" branch for minor improvement, but I really > don't understand what exactly should I do to > make patch properly. Trying to follow supplied instructions end in > "CONFLICT" while making git rebase > > Maybe it would be better to add simple, but working snippet of code, > how exactly you want to prepare git repo? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- /Henrik Nord Erlang/OTP From max.lapshin@REDACTED Tue Jun 26 11:43:32 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 26 Jun 2012 13:43:32 +0400 Subject: [erlang-questions] =?utf-8?q?supervisor_with_obstinate_restart_po?= =?utf-8?q?licy_=E2=80=94_are_there_any_implementations=3F?= Message-ID: Hi. I think that many people have met with OTP supervisor problem: if your supervisor must work with external resource, and this resource is down, you get system, brought down after some restarts. I think that there are many implementations of trackers, that restart such jobs and thus reimplement OTP supervisors. Have anyone implemented supervisor that is OTP compatible, and doesn't fail on frequent worker restarts, but starts to restart less and less frequent? From baliulia@REDACTED Tue Jun 26 13:04:57 2012 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Tue, 26 Jun 2012 14:04:57 +0300 Subject: [erlang-questions] remsh, lager and local output Message-ID: <4FE99759.1020403@gmail.com> Hi Erlangers, I am attaching to a remote node via 'erl -remsh' and I'd like to be able to see all of it's output in the REPL. For logging I am running lager on the remote node. From what I've found, I see only one way to proceed: Write a custom lager backend which could take as a parameter the Pid of the group_leader() on the node from which I am attaching and use it as the IoDevice for the io calls. But this seems a bit odd to me. Has anyone had/tried to solve this problem? If so, I'd be very interested to hear with what you came up. Best wishes, Ignas From Sergey_Zhemzhitsky@REDACTED Tue Jun 26 13:09:48 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Tue, 26 Jun 2012 11:09:48 +0000 Subject: [erlang-questions] Parameterized module initialization Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Hi there, I?d like to use parameterized module to save some readonly staff and use it later at every function call. As I understand the compiler implicitly makes every function have parameters of the module, so even the function with zero arguments will be exported as a function with arity that corresponds to the number of module parameters. Is it possible to have a function in a parameterized module that can be exported without any parameters of the module? Best Regards, Sergey _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jun 26 14:05:21 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 15:05:21 +0300 Subject: [erlang-questions] nested variable assignment : bug or the expected behaviour ? In-Reply-To: References: Message-ID: <4FE9A581.7030108@gmail.com> On 06/26/2012 01:55 AM, James Churchman wrote: > If i have a function like : > > f()->[J=100,J]. > > Or put in the shell: > {J=200,J}. > > Or > > erlang:'+'(J=10,J). > > Or any other combination of where i try to set a variable nested > inside somewhere else, it will say "variable 'J' is unbound" .. is > this the expected behaviour ? Yes, it's expected. Each expression and subexpression in Erlang has an input environment and an output environment (which is the same as the input environment unless the expression creates additional bindings). Environments are chained so that in the comma-separated body of a clause, the output from one expression becomes the input of the next. However, arguments to a single expression, such as f(A, B) or {A, B}, are always evaluated in the same input environment. Hence, even if you write f(X=1, X), where X is a previously unbound variable, both arguments are evaluated in the environment that does not contain X, and the compiler complains about the second X being unknown. The output environment from an expression is the combination of the output environments from its subexpressions, so f(X=1, Y=2) exports both X and Y. To preserve the constraint that a variable occurring in more than one pattern has the same value independent of evaluation order, if you write f(X=a(), X=b()), the compiler will actually insert a test after the evaluation of the arguments to check that the intermediate results of a() and b() are in fact the same value and so either can be taken as the value of X. As Richard O'Keefe said in his reply, Erlang uses strict left-to-right evaluation order also in function arguments (even where it to someone writing a compiler might seem to be backwards). This is to ensure that an Erlang program behaves the same regardless of compiler implementation, and to eliminate a source of confusion: programmers shouldn't have to remember a bunch of special cases where evaluation order is right-to-left or undefined. What you see is what you get. But at the same time, the known evaluation order (with respect to side effects) should not be something that programmers can abuse to write unreadable code like f(1+g(X=h(Y),42),{foo,X}). Therefore, the scoping rules enforce that bindings are effectively treated as if the evaluation order of arguments was unknown, or could be changed from one version of Erlang to the next without changing the meaning of the program. /Richard From erlang@REDACTED Tue Jun 26 14:53:18 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 26 Jun 2012 14:53:18 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: There is an undocumented way of doing this (which one day will become standard) Suppose we define X as follows: > X = {mod1, a,b,c}. {mod1, a,b,c}. X is now an instance of a parameterised module. if we now call the function X:func1(x,y) then what actually gets called is the function mod1:func1(x,y,{mod1,a,b,c}) So for example if we define a module sneaky.erl like this: -module(sneaky). -export([test/2, hide/1]). test(X, Y) -> {sneaky, X, Y}. hide(Something) -> {sneaky, Something}. Then in the shell we can do this: 1> c(sneaky). {ok,sneaky} 2> X = sneaky:hide({rectange,10,file,"foo"}). {sneaky,{rectange,10,file,"foo"}} 3> X:test(abc). {sneaky,abc,{sneaky,{rectange,10,file,"foo"}}} Cheers /Joe On Tue, Jun 26, 2012 at 1:09 PM, Zhemzhitsky Sergey wrote: > Hi there, > > > > I?d like to use parameterized module to save some readonly staff and use it > later at every function call. > > > > As I understand the compiler implicitly makes every function have parameters > of the module, so even the function with zero arguments will be exported as > a function with arity that corresponds to the number of module parameters. > > > > Is it possible to have a function in a parameterized module that can be > exported without any parameters of the module? > > > > Best Regards, > > Sergey > > > > _______________________________________________________ > > > > The information contained in this message may be privileged and conf > idential and protected from disclosure. If you are not the original intended > recipient, you are hereby notified that any review, retransmission, > dissemination, or other use of, or taking of any action in reliance upon, > this information is prohibited. If you have received this communication in > error, please notify the sender immediately by replying to this message and > delete it from your computer. Thank you for your cooperation. Troika Dialog, > Russia. > > If you need assistance please contact our Contact Center (+7495) 258 0500 or > go to www.troika.ru/eng/Contacts/system.wbp > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vladdu55@REDACTED Tue Jun 26 15:14:22 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 26 Jun 2012 15:14:22 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: Hi, On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong wrote: > There is an undocumented way of doing this (which one day will become standard) > Suppose we define X as follows: >> X = {mod1, a,b,c}. > {mod1, a,b,c}. > X is now an instance of a parameterised module. if we now call the > function X:func1(x,y) then what actually gets called is the function > mod1:func1(x,y,{mod1,a,b,c}) I hope that you are expressing some form of sarcasm, Joe. If I may use such language, it is horrible. Just as we got (almost) rid of {M,F} to the benefit of proper function references, are we going to get another kludge? A proper module reference is the way to go here. Then there are plenty of nice things that can be supported in a clean way. best regards, Vlad From Sergey_Zhemzhitsky@REDACTED Tue Jun 26 16:16:48 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Tue, 26 Jun 2012 14:16:48 +0000 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: <06139A918ACCA041BF46A0F36940C7FA39CCBCA6@exch-mbx2.msk.trd.ru> Hi guys, Thanks a lot for answers. I suppose I will not use any workarounds for now. Best Regards, Sergey -----Original Message----- From: Vlad Dumitrescu [mailto:vladdu55@REDACTED] Sent: Tuesday, June 26, 2012 5:14 PM To: Joe Armstrong Cc: Zhemzhitsky Sergey; Erlang-Questions Questions Subject: Re: [erlang-questions] Parameterized module initialization Hi, On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong wrote: > There is an undocumented way of doing this (which one day will become > standard) Suppose we define X as follows: >> X = {mod1, a,b,c}. > {mod1, a,b,c}. > X is now an instance of a parameterised module. if we now call the > function X:func1(x,y) then what actually gets called is the function > mod1:func1(x,y,{mod1,a,b,c}) I hope that you are expressing some form of sarcasm, Joe. If I may use such language, it is horrible. Just as we got (almost) rid of {M,F} to the benefit of proper function references, are we going to get another kludge? A proper module reference is the way to go here. Then there are plenty of nice things that can be supported in a clean way. best regards, Vlad _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From erlang@REDACTED Tue Jun 26 16:56:02 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 26 Jun 2012 16:56:02 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: On Tue, Jun 26, 2012 at 3:14 PM, Vlad Dumitrescu wrote: > Hi, > > On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong wrote: >> There is an undocumented way of doing this (which one day will become standard) >> Suppose we define X as follows: >>> X = {mod1, a,b,c}. >> {mod1, a,b,c}. >> X is now an instance of a parameterised module. if we now call the >> function X:func1(x,y) then what actually gets called is the function >> mod1:func1(x,y,{mod1,a,b,c}) > > I hope that you are expressing some form of sarcasm, Joe. No - there is no sarcasm implied. I like this mechanism. This way of hiding additional data is very flexible. I've used this in several programs, and the resulting code is easy to understand. This nice thing is that using this you can make very nice client APIs. For example, for string processing you might say Str = {ansi_string, "abc"}, or Str = {utf8_string, "u+1234"} And then when you call Str:substr(1,4) Get the correct behaviour depending upon the underlying string type ie either ansi_string:substr(1,4,{ansi_string,"abc"}) or utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called If you write a program where strings might be in latin1, or utf8 or something else you'd still have to tag the string (as in say {latin1, "abc"}) so you knew what the encoding was and still have to call different modules depending upon the tag. This is what the above mechanism does so it's very nice for hiding the mess in polymorphic interfaces. /Joe > > If I may use such language, it is horrible. Just as we got (almost) > rid of {M,F} to the benefit of proper function references, are we > going to get another kludge? A proper module reference is the way to > go here. Then there are plenty of nice things that can be supported in > a clean way. > > best regards, > Vlad From n.oxyde@REDACTED Tue Jun 26 18:18:22 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 26 Jun 2012 18:18:22 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Le 26 juin 2012 ? 16:56, Joe Armstrong a ?crit : > I like this mechanism. This way of hiding additional data is very flexible. > > I've used this in several programs, and the resulting code is easy to > understand. This nice thing is that using this you can make very nice > client APIs. For example, for string processing you might say > > Str = {ansi_string, "abc"}, or > Str = {utf8_string, "u+1234"} > > And then when you call > > Str:substr(1,4) > > Get the correct behaviour depending upon the underlying string type ie > either ansi_string:substr(1,4,{ansi_string,"abc"}) or > utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called > > If you write a program where strings might be in latin1, or utf8 or > something else you'd still > have to tag the string (as in say {latin1, "abc"}) so you knew what > the encoding was and still > have to call different modules depending upon the tag. This is what > the above mechanism does > so it's very nice for hiding the mess in polymorphic interfaces. > > /Joe I think it is quite ugly to have terms as module parameters; a more beautiful way in my opinion would be to have something like Caml functors, which are mappings from structures (modules in Erlang) to structures. You would have a string functor which takes as a parameter a string implementation module, either ansi_string or utf8_string. You could then abstract the common stuff from ansi_ and utf8_ in string. Regards, -- Anthony Ramine From rexxe98@REDACTED Tue Jun 26 18:31:20 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 26 Jun 2012 09:31:20 -0700 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: Thanks Alex, I've looked at 0MQ before, but I'm not clear what it provides me over just using standard Erlang message passing or using something like gproc. Can you shed any light on this? Thanks, Andrew On Mon, Jun 25, 2012 at 11:11 PM, Alex Shneyderman wrote: > > I'm wondering if anyone has done any work on building a gen_server > > replacement which uses a durable queue. I'm looking to build a command > bus > > and I cannot lose messages if the node goes down. I was thinking of > using > > Rabbit, but it seems like overkill for me and just another component I > have > > to worry about. > > Did you look at ZeroMQ? Look at the last reliability pattern, called > titanic in the > guide (http://zguide.zeromq.org). 0MQ is a library hence it will not > seem like an > overkill. That is of course just one reason to use 0MQ for others read > the guide, > it is actually very much worth it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Tue Jun 26 18:32:25 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 26 Jun 2012 09:32:25 -0700 Subject: [erlang-questions] gen_server durable queue In-Reply-To: <488F7C08-2A5E-4D15-9B66-0F9B9A3BC58C@gmail.com> References: <488F7C08-2A5E-4D15-9B66-0F9B9A3BC58C@gmail.com> Message-ID: Hey Max, Yeah, I have experience with Rabbit too and I just didn't want to turn to it for this use case, but I may have to. Thanks for your advice! Andrew On Tue, Jun 26, 2012 at 12:16 AM, Max Bourinov wrote: > Hi Andrew, > > My xp with rabbit shows that you don't have to worry about it after > installation. Go with rabbit. > > Sent from my iPad > > On 26.06.2012, at 9:52, Andrew Berman wrote: > > > Hey all, > > > > I'm wondering if anyone has done any work on building a gen_server > replacement which uses a durable queue. I'm looking to build a command bus > and I cannot lose messages if the node goes down. I was thinking of using > Rabbit, but it seems like overkill for me and just another component I have > to worry about. > > > > Thanks, > > > > Andrew > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Tue Jun 26 18:53:58 2012 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Tue, 26 Jun 2012 18:53:58 +0200 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: > I've looked at 0MQ before, but I'm not clear what it provides me over just > using standard Erlang message passing or using something like gproc. ?Can > you shed any light on this? Not over Erlang, no. But you mentioned you wanted to use Rabbit. I find Rabbit a bit problematic as it will require an extra node to worry about, if you want it to be reliable you will need more than one. 0MQ is a library that has no such requirements it will live on any node you start but you will have to encode reliability on your own. Hence my reference to zguide, which explains exactly how to do it. Cheers, Alex. From watson.timothy@REDACTED Tue Jun 26 19:25:06 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 26 Jun 2012 18:25:06 +0100 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: <32361BD8-0829-4E6D-8129-0C55D3CCF212@gmail.com> Why do you say rabbit requires an extra node to be reliable? On 26 Jun 2012, at 17:53, Alex Shneyderman wrote: >> I've looked at 0MQ before, but I'm not clear what it provides me over just >> using standard Erlang message passing or using something like gproc. Can >> you shed any light on this? > > Not over Erlang, no. But you mentioned you wanted to use Rabbit. I > find Rabbit a bit > problematic as it will require an extra node to worry about, if you > want it to be reliable > you will need more than one. 0MQ is a library that has no such > requirements it will > live on any node you start but you will have to encode reliability on > your own. Hence > my reference to zguide, which explains exactly how to do it. > > Cheers, > Alex. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Tue Jun 26 19:26:13 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 26 Jun 2012 18:26:13 +0100 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <734372BD-959C-40DA-9519-61DF76425205@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Message-ID: I agree that something like functors would be a really nice abstraction for erlang. On 26 Jun 2012, at 17:18, Anthony Ramine wrote: > Le 26 juin 2012 ? 16:56, Joe Armstrong a ?crit : > >> I like this mechanism. This way of hiding additional data is very flexible. >> >> I've used this in several programs, and the resulting code is easy to >> understand. This nice thing is that using this you can make very nice >> client APIs. For example, for string processing you might say >> >> Str = {ansi_string, "abc"}, or >> Str = {utf8_string, "u+1234"} >> >> And then when you call >> >> Str:substr(1,4) >> >> Get the correct behaviour depending upon the underlying string type ie >> either ansi_string:substr(1,4,{ansi_string,"abc"}) or >> utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called >> >> If you write a program where strings might be in latin1, or utf8 or >> something else you'd still >> have to tag the string (as in say {latin1, "abc"}) so you knew what >> the encoding was and still >> have to call different modules depending upon the tag. This is what >> the above mechanism does >> so it's very nice for hiding the mess in polymorphic interfaces. >> >> /Joe > > I think it is quite ugly to have terms as module parameters; a more beautiful way in my opinion would be to have something like Caml functors, which are mappings from structures (modules in Erlang) to structures. > > You would have a string functor which takes as a parameter a string implementation module, either ansi_string or utf8_string. You could then abstract the common stuff from ansi_ and utf8_ in string. > > Regards, > > -- > Anthony Ramine > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rexxe98@REDACTED Tue Jun 26 19:30:41 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 26 Jun 2012 10:30:41 -0700 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: Actually, found two articles relating to the difference: 1. http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ 2. http://jlouisramblings.blogspot.com/2011/10/one-major-difference-zeromq-and-erlang.html The way I see it, if you're using Erlang, there's no point to using 0MQ. It really provides the same features, just differently. I feel like 0MQ would be great for a language that doesn't provide what Erlang does natively. --Andrew On Tue, Jun 26, 2012 at 9:53 AM, Alex Shneyderman wrote: > > I've looked at 0MQ before, but I'm not clear what it provides me over > just > > using standard Erlang message passing or using something like gproc. Can > > you shed any light on this? > > Not over Erlang, no. But you mentioned you wanted to use Rabbit. I > find Rabbit a bit > problematic as it will require an extra node to worry about, if you > want it to be reliable > you will need more than one. 0MQ is a library that has no such > requirements it will > live on any node you start but you will have to encode reliability on > your own. Hence > my reference to zguide, which explains exactly how to do it. > > Cheers, > Alex. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Tue Jun 26 19:31:28 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 26 Jun 2012 10:31:28 -0700 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Message-ID: I agree too, that would be really nice to get something like that in Erlang. On Tue, Jun 26, 2012 at 10:26 AM, Tim Watson wrote: > I agree that something like functors would be a really nice abstraction > for erlang. > > On 26 Jun 2012, at 17:18, Anthony Ramine wrote: > > > Le 26 juin 2012 ? 16:56, Joe Armstrong a ?crit : > > > >> I like this mechanism. This way of hiding additional data is very > flexible. > >> > >> I've used this in several programs, and the resulting code is easy to > >> understand. This nice thing is that using this you can make very nice > >> client APIs. For example, for string processing you might say > >> > >> Str = {ansi_string, "abc"}, or > >> Str = {utf8_string, "u+1234"} > >> > >> And then when you call > >> > >> Str:substr(1,4) > >> > >> Get the correct behaviour depending upon the underlying string type ie > >> either ansi_string:substr(1,4,{ansi_string,"abc"}) or > >> utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called > >> > >> If you write a program where strings might be in latin1, or utf8 or > >> something else you'd still > >> have to tag the string (as in say {latin1, "abc"}) so you knew what > >> the encoding was and still > >> have to call different modules depending upon the tag. This is what > >> the above mechanism does > >> so it's very nice for hiding the mess in polymorphic interfaces. > >> > >> /Joe > > > > I think it is quite ugly to have terms as module parameters; a more > beautiful way in my opinion would be to have something like Caml functors, > which are mappings from structures (modules in Erlang) to structures. > > > > You would have a string functor which takes as a parameter a string > implementation module, either ansi_string or utf8_string. You could then > abstract the common stuff from ansi_ and utf8_ in string. > > > > Regards, > > > > -- > > Anthony Ramine > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Tue Jun 26 19:33:46 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 26 Jun 2012 10:33:46 -0700 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Message-ID: I'd also like to know why there are "hidden" features of Erlang in the first place. I don't understand why they cannot be documented and accepted in the community and if not, just get rid of them. No one wants to use something that might possibly be removed in the future or unsupported. Just put it in the language or get rid of it. --Andrew On Tue, Jun 26, 2012 at 10:31 AM, Andrew Berman wrote: > I agree too, that would be really nice to get something like that in > Erlang. > > > On Tue, Jun 26, 2012 at 10:26 AM, Tim Watson wrote: > >> I agree that something like functors would be a really nice abstraction >> for erlang. >> >> On 26 Jun 2012, at 17:18, Anthony Ramine wrote: >> >> > Le 26 juin 2012 ? 16:56, Joe Armstrong a ?crit : >> > >> >> I like this mechanism. This way of hiding additional data is very >> flexible. >> >> >> >> I've used this in several programs, and the resulting code is easy to >> >> understand. This nice thing is that using this you can make very nice >> >> client APIs. For example, for string processing you might say >> >> >> >> Str = {ansi_string, "abc"}, or >> >> Str = {utf8_string, "u+1234"} >> >> >> >> And then when you call >> >> >> >> Str:substr(1,4) >> >> >> >> Get the correct behaviour depending upon the underlying string type ie >> >> either ansi_string:substr(1,4,{ansi_string,"abc"}) or >> >> utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called >> >> >> >> If you write a program where strings might be in latin1, or utf8 or >> >> something else you'd still >> >> have to tag the string (as in say {latin1, "abc"}) so you knew what >> >> the encoding was and still >> >> have to call different modules depending upon the tag. This is what >> >> the above mechanism does >> >> so it's very nice for hiding the mess in polymorphic interfaces. >> >> >> >> /Joe >> > >> > I think it is quite ugly to have terms as module parameters; a more >> beautiful way in my opinion would be to have something like Caml functors, >> which are mappings from structures (modules in Erlang) to structures. >> > >> > You would have a string functor which takes as a parameter a string >> implementation module, either ansi_string or utf8_string. You could then >> abstract the common stuff from ansi_ and utf8_ in string. >> > >> > Regards, >> > >> > -- >> > Anthony Ramine >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jun 26 20:32:22 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 21:32:22 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <734372BD-959C-40DA-9519-61DF76425205@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Message-ID: <4FEA0036.6020508@gmail.com> On 06/26/2012 07:18 PM, Anthony Ramine wrote: > I think it is quite ugly to have terms as module parameters; a more > beautiful way in my opinion would be to have something like Caml > functors, which are mappings from structures (modules in Erlang) to > structures. Parameterized modules in Erlang already correspond to what ML/Caml calls "functors": an abstraction over modules. What you decide to parameterize over: terms or other modules (or possibly even types, though I'm not sure what would be needed to support that), is up to you. > You would have a string functor which takes as a parameter a string > implementation module, either ansi_string or utf8_string. You could > then abstract the common stuff from ansi_ and utf8_ in string. And that is just as straightforward to do in Erlang as in Caml: -module(string, [StringImpl]). -export([reverse/1]). reverse(S) -> StringImpl:reverse(S) and then you should probably hide the details of instantiating such a module behind some factory module or facade: -module(string_maker). -export([...]). ansi_string_module() -> string:new(ansi_string). utf8_string_module() -> string:new(utf8_string). /Richard From carlsson.richard@REDACTED Tue Jun 26 20:41:10 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 21:41:10 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: <4FEA0246.70705@gmail.com> On 06/26/2012 02:09 PM, Zhemzhitsky Sergey wrote: > Hi there, > > I?d like to use parameterized module to save some readonly staff and use > it later at every function call. > > As I understand the compiler implicitly makes every function have > parameters of the module, so even the function with zero arguments will > be exported as a function with arity that corresponds to the number of > module parameters. > > Is it possible to have a function in a parameterized module that can be > exported without any parameters of the module? My intention was that there should be a "static" declaration to be used with parameterized modules; something like: -module(foo, [Bar]). -export([...]). -static([baz/2]). baz(A,B) -> ... where baz/2 would not get any automatic hidden variable, so you could call foo:baz(X,Y) directly without creating an instance of foo first. I just never got time to add this. If parameterized modules become a full-fledged language feature, some form of static-declarations should definitely be part of that. For anyone wondering why nothing is happening with this stuff, it's just a question of time and priorities. I'm not working for Ericsson, I have very little spare time these days, and the OTP team have lots of other higher prioritized things to do. /Richard From carlsson.richard@REDACTED Tue Jun 26 20:51:46 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 21:51:46 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: <4FEA04C2.8040305@gmail.com> On 06/26/2012 05:56 PM, Joe Armstrong wrote: > I've used this in several programs, and the resulting code is easy to > understand. This nice thing is that using this you can make very nice > client APIs. For example, for string processing you might say > > Str = {ansi_string, "abc"}, or > Str = {utf8_string, "u+1234"} > > And then when you call > > Str:substr(1,4) > > Get the correct behaviour depending upon the underlying string type If you want to do it properly, Joe, then use the automatically generated instantiator functions "new/N" instead of building tuple literals like that: Str = ansi_string:new("abc"), or Str = utf8_string:new("u+1234") then the code will keep working even if module instances become a separate data type (like funs), be easier to understand when you or anyone else reads it later on, be possible to process using tools like Dialyzer and Wrangler, and be easier to grep for "ansi_string:" etc. /Richard From mononcqc@REDACTED Tue Jun 26 21:09:00 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 26 Jun 2012 15:09:00 -0400 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEA0036.6020508@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> Message-ID: <4FEA08CC.9050909@ferd.ca> The problem I have with all of these things is that parametrized modules will automatically mess up with the arities of function when they're perceived and used. This means most tracing is going to be wrong. I can't trace on {PMod, Fun, Arity} where PMod is a parametrized module, because the current underlying representation ({Mod, Data}) isn't accepted by tools such as dbg. I can't trace on {Mod, Fun, Arity} where Mod is the actual module I think I use, because the Arity is expanded by PMods to pass hidden arguments. What I actually need to do is {Mod, Fun, Arity+1} while knowing what I have is a parametrized module. Messing with arity is not something I'm a fan of. I'm also not sure what impact we have when using PMods and the fun M:F/A expression, but I can already say it isn't obvious if I should expect it to break or not. They can be an abstraction, but at best they're a leaky one that can be rather confusing when using well known and supported tools. I have no solution to offer, though. On 12-06-26 2:32 PM, Richard Carlsson wrote: > > Parameterized modules in Erlang already correspond to what ML/Caml > calls "functors": an abstraction over modules. What you decide to > parameterize over: terms or other modules (or possibly even types, > though I'm not sure what would be needed to support that), is up to you. > >> You would have a string functor which takes as a parameter a string >> implementation module, either ansi_string or utf8_string. You could >> then abstract the common stuff from ansi_ and utf8_ in string. > > And that is just as straightforward to do in Erlang as in Caml: > > -module(string, [StringImpl]). > -export([reverse/1]). > reverse(S) -> StringImpl:reverse(S) > > and then you should probably hide the details of instantiating such a > module behind some factory module or facade: > > -module(string_maker). > -export([...]). > ansi_string_module() -> string:new(ansi_string). > utf8_string_module() -> string:new(utf8_string). > > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jose.valim@REDACTED Tue Jun 26 21:34:33 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Tue, 26 Jun 2012 21:34:33 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEA08CC.9050909@ferd.ca> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <4FEA08CC.9050909@ferd.ca> Message-ID: > > I have no solution to offer, though. > A possible solution, which would also fit well with Richard's static proposal, is something along the same lines as Python which requires you to explicitly pass *self*: some_fun(A, B, self) -> ... some_static_fun(A, B) -> ... Then the perceived arity is correct and you get static support for free. Of course, we cannot use "self" as keyword as Python, but maybe ?SELF or another special token will suit well. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jun 26 22:36:00 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 23:36:00 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEA08CC.9050909@ferd.ca> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <4FEA08CC.9050909@ferd.ca> Message-ID: <4FEA1D30.900@gmail.com> On 06/26/2012 10:09 PM, Fred Hebert wrote: > The problem I have with all of these things is that parametrized modules > will automatically mess up with the arities of function when they're > perceived and used. This means most tracing is going to be wrong. > > I can't trace on {PMod, Fun, Arity} where PMod is a parametrized module, > because the current underlying representation ({Mod, Data}) isn't > accepted by tools such as dbg. > > I can't trace on {Mod, Fun, Arity} where Mod is the actual module I > think I use, because the Arity is expanded by PMods to pass hidden > arguments. > > What I actually need to do is {Mod, Fun, Arity+1} while knowing what I > have is a parametrized module. Messing with arity is not something I'm a > fan of. I'm also not sure what impact we have when using PMods and the > fun M:F/A expression, but I can already say it isn't obvious if I should > expect it to break or not. > > They can be an abstraction, but at best they're a leaky one that can be > rather confusing when using well known and supported tools. > > I have no solution to offer, though. Yes, in the current, proof-of-concept implementation, there are a bunch of weird effects like this. In a full blown implementation, other tools should be taught to recognize parameterized modules (I think they currently get the attribute 'abstract'=true) and handle them appropriately, so that the arities are presented as they look in the source code, not as they are implemented under the hood. /Richard From carlsson.richard@REDACTED Tue Jun 26 22:40:09 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jun 2012 23:40:09 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <4FEA08CC.9050909@ferd.ca> Message-ID: <4FEA1E29.9010305@gmail.com> On 06/26/2012 10:34 PM, Jos? Valim wrote: > A possible solution, which would also fit well with Richard's static > proposal, is something along the same lines as Python which requires you > to explicitly pass *self*: > > some_fun(A, B, self) -> ... > some_static_fun(A, B) -> ... > > Then the perceived arity is correct and you get static support for free. > Of course, we cannot use "self" as keyword as Python, but maybe ?SELF or > another special token will suit well. No need for a special token; you can use any variable name. The current implementation of parameterized modules already binds the name THIS to the automatic extra argument. I always disliked the explicit self-parameter in Python, though. But yes, it's an easy cop-out. /Richard From steven.charles.davis@REDACTED Wed Jun 27 02:15:01 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 26 Jun 2012 17:15:01 -0700 (PDT) Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: <1561c77d-a1c4-4c3e-ac7d-e7dc2ff1c38e@googlegroups.com> Is it just me, or is hidden state in a function that appears to the client as stateless always bad news... eventually? /s > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Jun 27 02:40:02 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 27 Jun 2012 12:40:02 +1200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> Message-ID: <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> On 27/06/2012, at 5:33 AM, Andrew Berman wrote: > I'd also like to know why there are "hidden" features of Erlang in the first place. Because it isn't dead. New ideas get tried, and sane designers don't take it for granted that their precious new idea is going to be right first time. So initial implementations get short-cuts so that people can get some experience with a new feature before designing and documenting the Real Thing. Strictly speaking, in this case it isn't the *feature* (parameterised modules) that is hidden but the *implementation* of that feature. From ok@REDACTED Wed Jun 27 02:47:02 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 27 Jun 2012 12:47:02 +1200 Subject: [erlang-questions] Exams are over and I've forgotten who I promised to read things for Message-ID: Two people in this mailing list asked me to read something for them. One was an article and the other was a web page. Exam marking is over and I now have time to do this, but I've lost track of what it was I was supposed to read. The web page looked promising but could benefit from a couple of minor things. The article looked excellent; there's a journal where that kind of stuff would fit perfectly. So I want to say nice things to you, if you will kindly tell me who you are! From bourinov@REDACTED Wed Jun 27 07:11:11 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 27 Jun 2012 09:11:11 +0400 Subject: [erlang-questions] gen_server durable queue In-Reply-To: References: Message-ID: <76006424-E2E8-4D77-AF6E-1DBE3B169CD2@gmail.com> Hi Andrew, Did you find the answer for your question? What is your approach? I have similar task and I do it with Rabbit but I want to know about all possibilities. Sent from my iPad On 26.06.2012, at 21:30, Andrew Berman wrote: > Actually, found two articles relating to the difference: > http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ > http://jlouisramblings.blogspot.com/2011/10/one-major-difference-zeromq-and-erlang.html > The way I see it, if you're using Erlang, there's no point to using 0MQ. It really provides the same features, just differently. I feel like 0MQ would be great for a language that doesn't provide what Erlang does natively. > > --Andrew > > On Tue, Jun 26, 2012 at 9:53 AM, Alex Shneyderman wrote: > > I've looked at 0MQ before, but I'm not clear what it provides me over just > > using standard Erlang message passing or using something like gproc. Can > > you shed any light on this? > > Not over Erlang, no. But you mentioned you wanted to use Rabbit. I > find Rabbit a bit > problematic as it will require an extra node to worry about, if you > want it to be reliable > you will need more than one. 0MQ is a library that has no such > requirements it will > live on any node you start but you will have to encode reliability on > your own. Hence > my reference to zguide, which explains exactly how to do it. > > Cheers, > Alex. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Wed Jun 27 08:19:42 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 27 Jun 2012 08:19:42 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: Hi, With the chance to be considered "ugly", I've been using the solution given here by Joe along with some others (using list instead of tuple - Erlang is good at list manipulation and, by the way, doesn't the great tail recursion use this kind of variable shadowing? i see no ugliness in it; list of tuples - feels like working with JSON's; record - working with server state; nested records - complex states, "one state to rule them all"; string hiding the variables - easy to convert into/from data transfer stream over a tcp connection; and maybe others). Each approach has its own advantages and disadvantages. The solution given by Joe is close to defining a state of a gen_server or whatever (pretty often in documentation one can find a state defined by a record). Maybe when Erlang will have the records replaced by something else, this practice will be obsolete, Joe's solution gets +1 from me. That's my 2c opinion. CGS On Tue, Jun 26, 2012 at 4:56 PM, Joe Armstrong wrote: > On Tue, Jun 26, 2012 at 3:14 PM, Vlad Dumitrescu > wrote: > > Hi, > > > > On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong wrote: > >> There is an undocumented way of doing this (which one day will become > standard) > >> Suppose we define X as follows: > >>> X = {mod1, a,b,c}. > >> {mod1, a,b,c}. > >> X is now an instance of a parameterised module. if we now call the > >> function X:func1(x,y) then what actually gets called is the function > >> mod1:func1(x,y,{mod1,a,b,c}) > > > > I hope that you are expressing some form of sarcasm, Joe. > > No - there is no sarcasm implied. > > I like this mechanism. This way of hiding additional data is very > flexible. > > I've used this in several programs, and the resulting code is easy to > understand. This nice thing is that using this you can make very nice > client APIs. For example, for string processing you might say > > Str = {ansi_string, "abc"}, or > Str = {utf8_string, "u+1234"} > > And then when you call > > Str:substr(1,4) > > Get the correct behaviour depending upon the underlying string type ie > either ansi_string:substr(1,4,{ansi_string,"abc"}) or > utf8_sting:substr(1,4,{utf8_string, "u+1234"} ) will be called > > If you write a program where strings might be in latin1, or utf8 or > something else you'd still > have to tag the string (as in say {latin1, "abc"}) so you knew what > the encoding was and still > have to call different modules depending upon the tag. This is what > the above mechanism does > so it's very nice for hiding the mess in polymorphic interfaces. > > /Joe > > > > > > > > If I may use such language, it is horrible. Just as we got (almost) > > rid of {M,F} to the benefit of proper function references, are we > > going to get another kludge? A proper module reference is the way to > > go here. Then there are plenty of nice things that can be supported in > > a clean way. > > > > > > > > best regards, > > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 27 09:09:53 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 27 Jun 2012 09:09:53 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> Message-ID: <4FEAB1C1.6080301@ninenines.eu> On 06/27/2012 02:40 AM, Richard O'Keefe wrote: > > On 27/06/2012, at 5:33 AM, Andrew Berman wrote: > >> I'd also like to know why there are "hidden" features of Erlang in the first place. > > > Because it isn't dead. New ideas get tried, and sane designers don't take it > for granted that their precious new idea is going to be right first time. > So initial implementations get short-cuts so that people can get some > experience with a new feature before designing and documenting the Real Thing. > > Strictly speaking, in this case it isn't the *feature* (parameterised modules) > that is hidden but the *implementation* of that feature. Sure, but these kind of things are better kept in an experimental branch, not in the real thing where popular projects start using them and people deploy them in production. -- Lo?c Hoguin Erlang Cowboy Nine Nines From bob@REDACTED Wed Jun 27 09:25:54 2012 From: bob@REDACTED (Bob Ippolito) Date: Wed, 27 Jun 2012 11:25:54 +0400 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEAB1C1.6080301@ninenines.eu> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> <4FEAB1C1.6080301@ninenines.eu> Message-ID: On Wednesday, June 27, 2012, Lo?c Hoguin wrote: > On 06/27/2012 02:40 AM, Richard O'Keefe wrote: > >> >> On 27/06/2012, at 5:33 AM, Andrew Berman wrote: >> >> I'd also like to know why there are "hidden" features of Erlang in the >>> first place. >>> >> >> >> Because it isn't dead. New ideas get tried, and sane designers don't >> take it >> for granted that their precious new idea is going to be right first time. >> So initial implementations get short-cuts so that people can get some >> experience with a new feature before designing and documenting the Real >> Thing. >> >> Strictly speaking, in this case it isn't the *feature* (parameterised >> modules) >> that is hidden but the *implementation* of that feature. >> > > Sure, but these kind of things are better kept in an experimental branch, > not in the real thing where popular projects start using them and people > deploy them in production. To be fair, all of these features were introduced before Erlang had public version control. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 27 09:39:17 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 27 Jun 2012 09:39:17 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: References: <4FE88DE7.20401@ninenines.eu> Message-ID: <4FEAB8A5.60201@ninenines.eu> On 06/25/2012 10:27 PM, Ingela Andin wrote: > Hi! > > 2012/6/25 Lo?c Hoguin : >> Hey, >> >> I'm running into the exact issue described here: >> http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html >> >> The certificate I have has a too long Subject line and Erlang just fails >> when trying to load it. This is a RapidSSL certificate, with a CA. Not sure >> I can just modify the subject directly while keeping it valid (don't know >> how anyway). >> >> Does anyone know how I could manage to use this certificate? If I need to >> patch OTP, any pointers as to where this fails is more than welcome. >> >> Thanks. >> > > The ecertfile is a backwards compatibility error code that you will > get if ssl has a problem reading the certfile. > It could be a file-error or a file-format error, or a bug in OTP > application public_key. So if you want to pinpoint the error you can > do: > {ok, PemBin} = file:read_file("Cert.pem"). > PemEntries = public_key:pem_decode(Pembin). > public_key:pem_entry_decode(hd(PemEntries)). Getting {'RSAPrivateKey','two-prime', ... All entries decode fine too. What next? Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines From essen@REDACTED Wed Jun 27 09:40:01 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 27 Jun 2012 09:40:01 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> <4FEAB1C1.6080301@ninenines.eu> Message-ID: <4FEAB8D1.5080600@ninenines.eu> On 06/27/2012 09:25 AM, Bob Ippolito wrote: > On Wednesday, June 27, 2012, Lo?c Hoguin wrote: > > On 06/27/2012 02:40 AM, Richard O'Keefe wrote: > > > On 27/06/2012, at 5:33 AM, Andrew Berman wrote: > > I'd also like to know why there are "hidden" features of > Erlang in the first place. > > > > Because it isn't dead. New ideas get tried, and sane designers > don't take it > for granted that their precious new idea is going to be right > first time. > So initial implementations get short-cuts so that people can get > some > experience with a new feature before designing and documenting > the Real Thing. > > Strictly speaking, in this case it isn't the *feature* > (parameterised modules) > that is hidden but the *implementation* of that feature. > > > Sure, but these kind of things are better kept in an experimental > branch, not in the real thing where popular projects start using > them and people deploy them in production. > > > To be fair, all of these features were introduced before Erlang had > public version control. I see, makes more sense. ;) -- Lo?c Hoguin Erlang Cowboy Nine Nines From ingela.andin@REDACTED Wed Jun 27 09:59:15 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 27 Jun 2012 09:59:15 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: <4FEAB8A5.60201@ninenines.eu> References: <4FE88DE7.20401@ninenines.eu> <4FEAB8A5.60201@ninenines.eu> Message-ID: Hi! 2012/6/27, Lo?c Hoguin : > On 06/25/2012 10:27 PM, Ingela Andin wrote: >> Hi! >> >> 2012/6/25 Lo?c Hoguin : >>> Hey, >>> >>> I'm running into the exact issue described here: >>> http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html >>> >>> The certificate I have has a too long Subject line and Erlang just fails >>> when trying to load it. This is a RapidSSL certificate, with a CA. Not >>> sure >>> I can just modify the subject directly while keeping it valid (don't >>> know >>> how anyway). >>> >>> Does anyone know how I could manage to use this certificate? If I need >>> to >>> patch OTP, any pointers as to where this fails is more than welcome. >>> >>> Thanks. >>> >> >> The ecertfile is a backwards compatibility error code that you will >> get if ssl has a problem reading the certfile. >> It could be a file-error or a file-format error, or a bug in OTP >> application public_key. So if you want to pinpoint the error you can >> do: >> {ok, PemBin} = file:read_file("Cert.pem"). >> PemEntries = public_key:pem_decode(Pembin). >> public_key:pem_entry_decode(hd(PemEntries)). > > Getting {'RSAPrivateKey','two-prime', ... > > All entries decode fine too. > > What next? All entries, is there more than one certificate entry? You can try tracing? (Use observer if you are not comfortable using dbg-module directly) Regards Ingela Erlang/OTP team - Ericsson AB From essen@REDACTED Wed Jun 27 10:46:21 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 27 Jun 2012 10:46:21 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: References: <4FE88DE7.20401@ninenines.eu> <4FEAB8A5.60201@ninenines.eu> Message-ID: <4FEAC85D.3070609@ninenines.eu> On 06/27/2012 09:59 AM, Ingela Andin wrote: > Hi! > > 2012/6/27, Lo?c Hoguin : >> On 06/25/2012 10:27 PM, Ingela Andin wrote: >>> Hi! >>> >>> 2012/6/25 Lo?c Hoguin : >>>> Hey, >>>> >>>> I'm running into the exact issue described here: >>>> http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html >>>> >>>> The certificate I have has a too long Subject line and Erlang just fails >>>> when trying to load it. This is a RapidSSL certificate, with a CA. Not >>>> sure >>>> I can just modify the subject directly while keeping it valid (don't >>>> know >>>> how anyway). >>>> >>>> Does anyone know how I could manage to use this certificate? If I need >>>> to >>>> patch OTP, any pointers as to where this fails is more than welcome. >>>> >>>> Thanks. >>>> >>> >>> The ecertfile is a backwards compatibility error code that you will >>> get if ssl has a problem reading the certfile. >>> It could be a file-error or a file-format error, or a bug in OTP >>> application public_key. So if you want to pinpoint the error you can >>> do: >>> {ok, PemBin} = file:read_file("Cert.pem"). >>> PemEntries = public_key:pem_decode(Pembin). >>> public_key:pem_entry_decode(hd(PemEntries)). >> >> Getting {'RSAPrivateKey','two-prime', ... >> >> All entries decode fine too. >> >> What next? > > All entries, is there more than one certificate entry? 1 rsa private key entry followed by 3 certificate entries. There's one for domain.com, one for *.domain.com and IIRC one for *.*.domain.com. > You can try tracing? (Use observer if you are not comfortable using > dbg-module directly) Will do. -- Lo?c Hoguin Erlang Cowboy Nine Nines From ingela.andin@REDACTED Wed Jun 27 11:10:06 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 27 Jun 2012 11:10:06 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: <4FEAC85D.3070609@ninenines.eu> References: <4FE88DE7.20401@ninenines.eu> <4FEAB8A5.60201@ninenines.eu> <4FEAC85D.3070609@ninenines.eu> Message-ID: Hi! 2012/6/27, Lo?c Hoguin : > On 06/27/2012 09:59 AM, Ingela Andin wrote: >> Hi! >> >> 2012/6/27, Lo?c Hoguin : >>> On 06/25/2012 10:27 PM, Ingela Andin wrote: >>>> Hi! >>>> >>>> 2012/6/25 Lo?c Hoguin : >>>>> Hey, >>>>> >>>>> I'm running into the exact issue described here: >>>>> http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html >>>>> >>>>> The certificate I have has a too long Subject line and Erlang just >>>>> fails >>>>> when trying to load it. This is a RapidSSL certificate, with a CA. Not >>>>> sure >>>>> I can just modify the subject directly while keeping it valid (don't >>>>> know >>>>> how anyway). >>>>> >>>>> Does anyone know how I could manage to use this certificate? If I need >>>>> to >>>>> patch OTP, any pointers as to where this fails is more than welcome. >>>>> >>>>> Thanks. >>>>> >>>> >>>> The ecertfile is a backwards compatibility error code that you will >>>> get if ssl has a problem reading the certfile. >>>> It could be a file-error or a file-format error, or a bug in OTP >>>> application public_key. So if you want to pinpoint the error you can >>>> do: >>>> {ok, PemBin} = file:read_file("Cert.pem"). >>>> PemEntries = public_key:pem_decode(Pembin). >>>> public_key:pem_entry_decode(hd(PemEntries)). >>> >>> Getting {'RSAPrivateKey','two-prime', ... >>> >>> All entries decode fine too. >>> >>> What next? >> >> All entries, is there more than one certificate entry? > > 1 rsa private key entry followed by 3 certificate entries. Well that is your problem then, currently there is only support for one certificate in the certifcate file (the key can be there but only one cert) as there is now algorithm to choose a specific cert. The cacert file may of course have many cert entries. Regards Ingela Erlang/OTP team - Ericsson AB From n.oxyde@REDACTED Wed Jun 27 11:18:30 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 27 Jun 2012 11:18:30 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEA0036.6020508@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> Message-ID: <8507A709-7DF3-4598-A4F1-2FEDBB605862@gmail.com> I find it way saner for a parameterized module to be a mapping from modules to modules than to be a way to encapsulate some arbitrary terms. I may be wrong but I think that it would be easier to implement and reason about them in a non leaky way if their arguments were only module names. -- Anthony Ramine Le 26 juin 2012 ? 20:32, Richard Carlsson a ?crit : > Parameterized modules in Erlang already correspond to what ML/Caml calls "functors": an abstraction over modules. What you decide to parameterize over: terms or other modules (or possibly even types, though I'm not sure what would be needed to support that), is up to you. From essen@REDACTED Wed Jun 27 11:29:06 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 27 Jun 2012 11:29:06 +0200 Subject: [erlang-questions] SSL certificate's subject too long In-Reply-To: References: <4FE88DE7.20401@ninenines.eu> <4FEAB8A5.60201@ninenines.eu> <4FEAC85D.3070609@ninenines.eu> Message-ID: <4FEAD262.8040802@ninenines.eu> On 06/27/2012 11:10 AM, Ingela Andin wrote: > Hi! > > 2012/6/27, Lo?c Hoguin : >> On 06/27/2012 09:59 AM, Ingela Andin wrote: >>> Hi! >>> >>> 2012/6/27, Lo?c Hoguin : >>>> On 06/25/2012 10:27 PM, Ingela Andin wrote: >>>>> Hi! >>>>> >>>>> 2012/6/25 Lo?c Hoguin : >>>>>> Hey, >>>>>> >>>>>> I'm running into the exact issue described here: >>>>>> http://www.mentby.com/Group/rabbitmq-discuss/ssl-certificate-error.html >>>>>> >>>>>> The certificate I have has a too long Subject line and Erlang just >>>>>> fails >>>>>> when trying to load it. This is a RapidSSL certificate, with a CA. Not >>>>>> sure >>>>>> I can just modify the subject directly while keeping it valid (don't >>>>>> know >>>>>> how anyway). >>>>>> >>>>>> Does anyone know how I could manage to use this certificate? If I need >>>>>> to >>>>>> patch OTP, any pointers as to where this fails is more than welcome. >>>>>> >>>>>> Thanks. >>>>>> >>>>> >>>>> The ecertfile is a backwards compatibility error code that you will >>>>> get if ssl has a problem reading the certfile. >>>>> It could be a file-error or a file-format error, or a bug in OTP >>>>> application public_key. So if you want to pinpoint the error you can >>>>> do: >>>>> {ok, PemBin} = file:read_file("Cert.pem"). >>>>> PemEntries = public_key:pem_decode(Pembin). >>>>> public_key:pem_entry_decode(hd(PemEntries)). >>>> >>>> Getting {'RSAPrivateKey','two-prime', ... >>>> >>>> All entries decode fine too. >>>> >>>> What next? >>> >>> All entries, is there more than one certificate entry? >> >> 1 rsa private key entry followed by 3 certificate entries. > > Well that is your problem then, currently there is only support for > one certificate in the certifcate file (the key can be there but only > one cert) as there is now algorithm to choose > a specific cert. The cacert file may of course have many cert entries. OK that was the issue, but the reason wasn't the one I expected. I didn't have more than one domain certificate, it seems I just had the cacert bundled in the file (and assumed wrong). Separating the two seems to make it work, although I'll have to try elsewhere than localhost. Thanks a lot for the help! -- Lo?c Hoguin Erlang Cowboy Nine Nines From g@REDACTED Wed Jun 27 12:28:51 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 27 Jun 2012 05:28:51 -0500 Subject: [erlang-questions] Exams are over and I've forgotten who I promised to read things for In-Reply-To: References: Message-ID: On Tue, Jun 26, 2012 at 7:47 PM, Richard O'Keefe wrote: > Two people in this mailing list asked me to read something for them. > One was an article and the other was a web page. > Exam marking is over and I now have time to do this, > but I've lost track of what it was I was supposed to read. > The web page looked promising but could benefit from a couple > of minor things. > The article looked excellent; there's a journal where that kind > of stuff would fit perfectly. > So I want to say nice things to you, if you will kindly tell me > who you are! /me raises his hand This was the blog post I was asking for notes on: http://gar1t.com/blog/2012/06/10/solving-embarrassingly-obvious-problems-in-erlang/ Garrett From henrik@REDACTED Wed Jun 27 14:06:10 2012 From: henrik@REDACTED (Henrik Nord) Date: Wed, 27 Jun 2012 14:06:10 +0200 Subject: [erlang-questions] Erlang.org back on track Message-ID: <4FEAF732.7000702@erlang.org> The frontpage of www.erlang.org is now back online after this mornings problems. Thank you -- /Henrik Nord Erlang/OTP From carlsson.richard@REDACTED Wed Jun 27 14:35:14 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 27 Jun 2012 15:35:14 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <1561c77d-a1c4-4c3e-ac7d-e7dc2ff1c38e@googlegroups.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <1561c77d-a1c4-4c3e-ac7d-e7dc2ff1c38e@googlegroups.com> Message-ID: <4FEAFE02.2050009@gmail.com> On 06/27/2012 03:15 AM, Steve Davis wrote: > Is it just me, or is hidden state in a function that appears to the > client as stateless always bad news... eventually? Yes, but this isn't about state. That's what makes it different from object oriented programming. Once you create M=mymodule:new(42), M is always the same M. Nothing that anybody else does can change the module instance behind your back. This is just as referentially transparent as the following: make_adder(X) -> fun (Y) -> Y+X end. add(A,B) -> F = make_adder(B), F(A). Using parameterized modules, this could be written -module(adder, [X]). -export([add/1]). add(Y) -> Y+X. -module(foo). add(A,B) -> Adder=adder:new(B), Adder:add(A). The main difference between funs and parameterized modules are that modules allow you to have several differently named entry points with possibly different arities. But you can do that with funs as well if you want; it's just more clumsy: make_stepper(C) -> fun ({forward, N}) -> C*N; ({backward, N}) -> -C*N end. compare: -module(stepper, [C]). -export([forward/1,backward/1]). forward(N) -> C*N. backward(N) -> -C*N. Also important is that parameterized modules are by design not tied to a particular version of the code (they always call the latest version of the module). Funs don't survive reloading the code for the module twice, but a module instance can be kept around for a long time. A caveat is that (just like funs) existing instances stop working if you change the interface and reload the code - for example, changing -module(foo, [X]) to -module([X,Y]). That's why I recommend that client code should probably never call your_module:new(...) directly, but always via some kind of facade, so clients don't know the actual names of the modules they instantiate and thus you can update the module name along with the interface if you need to to that sort of incompatible change: e.g., "make_mod(X) -> mod_v1:new(X)." becomes "make_mod(X,Y) -> mod_v2:new(X,Y)." /Richard From robert.virding@REDACTED Wed Jun 27 14:41:58 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 27 Jun 2012 13:41:58 +0100 (BST) Subject: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 In-Reply-To: <201206221104.38808.clist@uah.es> Message-ID: I don't really think that there is an inconsistency here. In all the cases where something is to be started, supervisor, application and proc_lib, the specifier for this is the Mod,Fun,Args of apply and spawn. Though I will admit that in the application case the function name is predefined, almost like a call back function. And for callbacks all the arguments, number and meaning, are predefined. I personally think the main problems are: - Inconsistency in the documentation caused by using "Args" to mean different things. So in supervisor. application and proc_lib "Args" really is a list of arguments in apply/spawn style where the call is built using the length of the list to determine the arity of the called function. In gen_XXX the ONE argument to Mod:init/1 is also called "Args" where it can be anything and even if it is a list it will only become one argument. Calling it "Data" would have made it easier to understand. - The second argument supervisor:start_child/2 to completely different, both type and meaning, depending on whether the supervisor is a simple_one_for_one or anything else. Yes, I know that it is documented but it IS inconsistent. I think that in general making all the start functions arity 1 and calling them with a list of arguments is of no real benefit. Apart from simple_one_for_one supervisors the arguments to the start function are anyway fixed so there would be no real gain here anyway. Also it would be a completely backwards incompatible change which would cause a lot of rewriting, so I don't see it being applied. We will just have to live with it. If it is of any consolation there are changes I would like to make which would also never be made. :-) Robert ----- Original Message ----- > > i did'nt find any example of mixing args from the supervirsor > childspec > and start_child(...) so im going to make my own digging in this > issue: > > Let see How the simple_one_by_one works... > > handle_call({start_child, EArgs}, _From, State) when > ?is_simple(State) -> > Child = hd(State#state.children), > #child{mfargs = {M, F, A}} = Child, > Args = A ++ EArgs, > case do_start_child_i(M, F, Args) of > <------>{ok, undefined} when Child#child.restart_type =:= temporary > -> > <------> {reply, {ok, undefined}, State}; > <------>{ok, Pid} -> > <------> NState = save_dynamic_child(Child#child.restart_type, > Pid, Args, State), > <------> {reply, {ok, Pid}, NState}; > <------>{ok, Pid, Extra} -> > <------> NState = save_dynamic_child(Child#child.restart_type, > Pid, Args, State), > <------> {reply, {ok, Pid, Extra}, NState}; > <------>What -> > <------> {reply, What, State} > end; > > > Shouldnt be do_start_child_i(M,F,EArgs) of the form : > > do_start_child(M,F,[EArgs]) ?? > > or the inner call apply(M,F,[A])? > > Provided the docs state that your unique exported funcion must be of > arity one you should > expect that code will take measures in order to enforce arity one, > enclosing whatever you pass it in a List on the apply phase. > > The semanctics of apply use a list to KNOW how many args the target > call has, while the > semantics of sup childspec + sup start_child use a list ++ op to know > how many args will be passed > to your callback of arity ONE. > > So for the apply part all your functions are arity ONE > > Without the ability to join Args from the ChildSpec and from the > start child call i would concur that you should > enclose in a list whatever you want to be as a sole argument and this > code be ok, but as soon as you can put > two or more args one on the ChildSpec and one on the call Sup code > must enforce that only one arg APPLY > call is made so you end calling start_link/1 whatever you pass on the > args... > > Even the DOCS state "[ term() ]" as the MFA on the ChildSpec > > > My stdlib-1.18.1.pdf says (Pag 369) > > "start_child(SupRef, ChildSpec) -> startchild_ret() > Types: > SupRef = sup_ref() > ChildSpec = child_spec() | (List :: [term()]) > ... > ... > > ...If the case of a simple_one_for_one supervisor, the child > specification defined in Module:init/1 will be > used and ChildSpec should instead be an arbitrary list of terms List. > The child process will then be started by > appending List to the existing start function arguments, i.e. by > calling apply(M, F, A++List) where {M,F,A} > is the start function defined in the child specification." > > Here IMHO lies the error, as A is a list we have [any()...] ++ > [any()...] is a list [any()...,any()...] and thats get us an apply > call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} and > you try provide a [..] in your start_child it will fail > unless both argslists are void. > > So a combination of: > > init(Opts) -> > io:format("[GROUP SUP] Ready to manage MuC with opts: > ~p\'s\n",[Opts]), > > {ok, { > {simple_one_for_one, 1, 60}, > [ > {mucfsm, {sim_group_fsm, start_link, > [arg1,arg2]}, transient, 60, worker, [sim_group]} > ] > } > }. > > > and a client that makes a call like: > ... > {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), > ... > > Gets you a final apply call {M,F,A} where A is in the form > [arg1,arg2,arg3,arg4] > not [[arg1,arg2,argf3,arg4]] at is should be when you intent to call > a start_link/1.... > > > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > [QuickCheck] [PropEr] [Erl0MQ] > Starting simple_one_by_one group supervisor... > [GROUP SUP] staring with argument: [{option1,40}] > Eshell V5.9.1 (abort with ^G) > 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s > Adding a child dynamically... > {"init terminating in > do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.erl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{line,344}]}, > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]}]}} > > > /Angel > > On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel > escribi?: > > That's right... > > > > But start_link it is expected to be or arity 3 and Opts is a > > property list. > > > > but then on test:main/0 when i do > > supervisor:start_child(SupPid,Opts2) > > > > being Opts2 another property list, then suppervisor should call: > > > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 which > > in turn > > calls gen_fsm:start_link, thus providing a Pid in the end... > > > > but what it really does is apply(sim_group_fsm,start_link,Opts ++ > > Opts2,[ > > ]) and then crashes unless i declare a start_link/2 on my > > gen_fsm... > > > > { > > {badmatch, > > {error, > > {'EXIT', > > {undef, > > [ > > {sim_group_fsm,start_link,[[{option1,40}],[{option29,100}]],[]}, > > {supervisor,do_start_child_i,3,[ > > {file,"supervisor.erl"}, > > {line,319} > > ]}...... > > > > i dont see why i finish having a apply/4 call in supervisor: 319 > > instead of > > an apply/3... > > > > I want the sup to propagate one property list to children specs > > while i > > will provide another during children spawns providing that sup > > combines > > two in any way children will manage to demunge ...both property > > lists > > > > So cant just see what im doing wrong.... > > > > /angel > > > > > > PS: i changed a bit the test code to force sup crash > > > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez Miguel > > > wrote: > > > } While calling a simple_one_by_one supervisor to start a gen > > > fsm I > > > found } it causes calling a start_link/2 function on the > > > callback > > > module of the } gen_fsm. > > > > > > You are specifying that function to be called to start the worker > > > yourself. In your module group_supervisor.erl you provide a > > > child_spec() > > > in the > > > > > > return value of init/1: > > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, > > > worker, > > > > > > [sim_group]} > > > > > > The form of which is: > > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > > > The second argument, StartFunc, has the form: > > > {Module, Function, Arguments} > > > > > > You provided: > > > {sim_group_fsm, start_link, [Opts]} > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gustav.simonsson@REDACTED Wed Jun 27 16:28:27 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Wed, 27 Jun 2012 15:28:27 +0100 (BST) Subject: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 In-Reply-To: Message-ID: <3a05a3fc-b9da-47f6-8536-8b77918996ff@knuth> This is a post talking about why you should not name things "Data" :P http://petdance.com/2012/04/the-worlds-two-worst-variable-names/ Perhaps a better name for what you send to Mod:init/1 in gen_XXX is "Argument" so it's more clear from the variable name that it is a single term. // Gustav Sent from my PC ----- Original Message ----- > From: "Robert Virding" > To: clist@REDACTED > Cc: erlang-questions@REDACTED > Sent: Wednesday, June 27, 2012 2:41:58 PM > Subject: Re: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 > > I don't really think that there is an inconsistency here. > > In all the cases where something is to be started, supervisor, > application and proc_lib, the specifier for this is the Mod,Fun,Args > of apply and spawn. Though I will admit that in the application case > the function name is predefined, almost like a call back function. > And for callbacks all the arguments, number and meaning, are > predefined. > > I personally think the main problems are: > > - Inconsistency in the documentation caused by using "Args" to mean > different things. So in supervisor. application and proc_lib "Args" > really is a list of arguments in apply/spawn style where the call is > built using the length of the list to determine the arity of the > called function. In gen_XXX the ONE argument to Mod:init/1 is also > called "Args" where it can be anything and even if it is a list it > will only become one argument. Calling it "Data" would have made it > easier to understand. > > - The second argument supervisor:start_child/2 to completely > different, both type and meaning, depending on whether the > supervisor is a simple_one_for_one or anything else. Yes, I know > that it is documented but it IS inconsistent. > > I think that in general making all the start functions arity 1 and > calling them with a list of arguments is of no real benefit. Apart > from simple_one_for_one supervisors the arguments to the start > function are anyway fixed so there would be no real gain here > anyway. > > Also it would be a completely backwards incompatible change which > would cause a lot of rewriting, so I don't see it being applied. > > We will just have to live with it. If it is of any consolation there > are changes I would like to make which would also never be made. :-) > > Robert > > ----- Original Message ----- > > > > i did'nt find any example of mixing args from the supervirsor > > childspec > > and start_child(...) so im going to make my own digging in this > > issue: > > > > Let see How the simple_one_by_one works... > > > > handle_call({start_child, EArgs}, _From, State) when > > ?is_simple(State) -> > > Child = hd(State#state.children), > > #child{mfargs = {M, F, A}} = Child, > > Args = A ++ EArgs, > > case do_start_child_i(M, F, Args) of > > <------>{ok, undefined} when Child#child.restart_type =:= temporary > > -> > > <------> {reply, {ok, undefined}, State}; > > <------>{ok, Pid} -> > > <------> NState = save_dynamic_child(Child#child.restart_type, > > Pid, Args, State), > > <------> {reply, {ok, Pid}, NState}; > > <------>{ok, Pid, Extra} -> > > <------> NState = save_dynamic_child(Child#child.restart_type, > > Pid, Args, State), > > <------> {reply, {ok, Pid, Extra}, NState}; > > <------>What -> > > <------> {reply, What, State} > > end; > > > > > > Shouldnt be do_start_child_i(M,F,EArgs) of the form : > > > > do_start_child(M,F,[EArgs]) ?? > > > > or the inner call apply(M,F,[A])? > > > > Provided the docs state that your unique exported funcion must be > > of > > arity one you should > > expect that code will take measures in order to enforce arity one, > > enclosing whatever you pass it in a List on the apply phase. > > > > The semanctics of apply use a list to KNOW how many args the target > > call has, while the > > semantics of sup childspec + sup start_child use a list ++ op to > > know > > how many args will be passed > > to your callback of arity ONE. > > > > So for the apply part all your functions are arity ONE > > > > Without the ability to join Args from the ChildSpec and from the > > start child call i would concur that you should > > enclose in a list whatever you want to be as a sole argument and > > this > > code be ok, but as soon as you can put > > two or more args one on the ChildSpec and one on the call Sup code > > must enforce that only one arg APPLY > > call is made so you end calling start_link/1 whatever you pass on > > the > > args... > > > > Even the DOCS state "[ term() ]" as the MFA on the ChildSpec > > > > > > My stdlib-1.18.1.pdf says (Pag 369) > > > > "start_child(SupRef, ChildSpec) -> startchild_ret() > > Types: > > SupRef = sup_ref() > > ChildSpec = child_spec() | (List :: [term()]) > > ... > > ... > > > > ...If the case of a simple_one_for_one supervisor, the child > > specification defined in Module:init/1 will be > > used and ChildSpec should instead be an arbitrary list of terms > > List. > > The child process will then be started by > > appending List to the existing start function arguments, i.e. by > > calling apply(M, F, A++List) where {M,F,A} > > is the start function defined in the child specification." > > > > Here IMHO lies the error, as A is a list we have [any()...] ++ > > [any()...] is a list [any()...,any()...] and thats get us an apply > > call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} > > and > > you try provide a [..] in your start_child it will fail > > unless both argslists are void. > > > > So a combination of: > > > > init(Opts) -> > > io:format("[GROUP SUP] Ready to manage MuC with opts: > > ~p\'s\n",[Opts]), > > > > {ok, { > > {simple_one_for_one, 1, 60}, > > [ > > {mucfsm, {sim_group_fsm, start_link, > > [arg1,arg2]}, transient, 60, worker, [sim_group]} > > ] > > } > > }. > > > > > > and a client that makes a call like: > > ... > > {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), > > ... > > > > Gets you a final apply call {M,F,A} where A is in the form > > [arg1,arg2,arg3,arg4] > > not [[arg1,arg2,argf3,arg4]] at is should be when you intent to > > call > > a start_link/1.... > > > > > > > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > > [async-threads:0] [hipe] [kernel-poll:false] > > > > [QuickCheck] [PropEr] [Erl0MQ] > > Starting simple_one_by_one group supervisor... > > [GROUP SUP] staring with argument: [{option1,40}] > > Eshell V5.9.1 (abort with ^G) > > 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s > > Adding a child dynamically... > > {"init terminating in > > do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.erl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{line,344}]}, > > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]}]}} > > > > > > /Angel > > > > On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel > > escribi?: > > > That's right... > > > > > > But start_link it is expected to be or arity 3 and Opts is a > > > property list. > > > > > > but then on test:main/0 when i do > > > supervisor:start_child(SupPid,Opts2) > > > > > > being Opts2 another property list, then suppervisor should call: > > > > > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 > > > which > > > in turn > > > calls gen_fsm:start_link, thus providing a Pid in the end... > > > > > > but what it really does is apply(sim_group_fsm,start_link,Opts ++ > > > Opts2,[ > > > ]) and then crashes unless i declare a start_link/2 on my > > > gen_fsm... > > > > > > { > > > {badmatch, > > > {error, > > > {'EXIT', > > > {undef, > > > [ > > > {sim_group_fsm,start_link,[[{option1,40}],[{option29,100}]],[]}, > > > {supervisor,do_start_child_i,3,[ > > > {file,"supervisor.erl"}, > > > {line,319} > > > ]}...... > > > > > > i dont see why i finish having a apply/4 call in supervisor: 319 > > > instead of > > > an apply/3... > > > > > > I want the sup to propagate one property list to children specs > > > while i > > > will provide another during children spawns providing that sup > > > combines > > > two in any way children will manage to demunge ...both property > > > lists > > > > > > So cant just see what im doing wrong.... > > > > > > /angel > > > > > > > > > PS: i changed a bit the test code to force sup crash > > > > > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez > > > > Miguel > > > > wrote: > > > > } While calling a simple_one_by_one supervisor to start a gen > > > > fsm I > > > > found } it causes calling a start_link/2 function on the > > > > callback > > > > module of the } gen_fsm. > > > > > > > > You are specifying that function to be called to start the > > > > worker > > > > yourself. In your module group_supervisor.erl you provide a > > > > child_spec() > > > > in the > > > > > > > > return value of init/1: > > > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, > > > > worker, > > > > > > > > [sim_group]} > > > > > > > > The form of which is: > > > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > > > > > The second argument, StartFunc, has the form: > > > > {Module, Function, Arguments} > > > > > > > > You provided: > > > > {sim_group_fsm, start_link, [Opts]} > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Wed Jun 27 17:01:30 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 27 Jun 2012 16:01:30 +0100 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <8507A709-7DF3-4598-A4F1-2FEDBB605862@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <8507A709-7DF3-4598-A4F1-2FEDBB605862@gmail.com> Message-ID: Yes, wouldn't existing stuff like xref and dialyzer and cover and so on, require less patching this way also? On 27 Jun 2012, at 10:18, Anthony Ramine wrote: > I find it way saner for a parameterized module to be a mapping from modules to modules than to be a way to encapsulate some arbitrary terms. > > I may be wrong but I think that it would be easier to implement and reason about them in a non leaky way if their arguments were only module names. > > -- > Anthony Ramine > > Le 26 juin 2012 ? 20:32, Richard Carlsson a ?crit : > >> Parameterized modules in Erlang already correspond to what ML/Caml calls "functors": an abstraction over modules. What you decide to parameterize over: terms or other modules (or possibly even types, though I'm not sure what would be needed to support that), is up to you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Wed Jun 27 18:08:13 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 27 Jun 2012 19:08:13 +0300 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <8507A709-7DF3-4598-A4F1-2FEDBB605862@gmail.com> Message-ID: <4FEB2FED.8030500@gmail.com> On 06/27/2012 06:01 PM, Tim Watson wrote: > Yes, wouldn't existing stuff like xref and dialyzer and cover and so > on, require less patching this way also? > > On 27 Jun 2012, at 10:18, Anthony Ramine wrote: > >> I find it way saner for a parameterized module to be a mapping from >> modules to modules than to be a way to encapsulate some arbitrary >> terms. >> >> I may be wrong but I think that it would be easier to implement and >> reason about them in a non leaky way if their arguments were only >> module names. In a statically typed language like ML, restricting the type of structure parameters to also be structures makes sense. It means that the functor is something that takes one or more smaller structures implementing some specific interfaces (classic examples are generic data types like a queue), and returns a new, larger structure that builds on the given structures. As far as I know, functor instantiation is done at compile time only, but using separate compilation for each involved structure. This basically means that instantiating a structure by another structure can be done by patching code pointers (and nobody needs to pass around an object/vtable). Since the signatures have been checked, nothing can "go wrong". Allowing other data types than structures as parameters would make separate compilation difficult (compare C++ templates). But it also means that the only way to parameterize a structure over a constant - say, the gravitational constant for a simulation - is to create a separate GravityConstant interface with a single function that returns the value to be used. That's a bit more boilerplate than I would like. In Erlang, however, there is no reason to limit the type of a module parameter. Indeed, the only way to enforce it would be to add runtime checks at each entry point. The implementation is not simplified - it gets more complicated. In practice, it is probably a good idea to mostly parameterize modules with respect to other modules (i.e., callbacks), but there's no point in trying to enforce that. /Richard From rexxe98@REDACTED Wed Jun 27 18:32:32 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Wed, 27 Jun 2012 09:32:32 -0700 Subject: [erlang-questions] gen_server durable queue In-Reply-To: <76006424-E2E8-4D77-AF6E-1DBE3B169CD2@gmail.com> References: <76006424-E2E8-4D77-AF6E-1DBE3B169CD2@gmail.com> Message-ID: No, not yet, Max. I'll let you know what I decide to do. Thanks, Andrew On Tue, Jun 26, 2012 at 10:11 PM, Max Bourinov wrote: > Hi Andrew, > > Did you find the answer for your question? What is your approach? I have > similar task and I do it with Rabbit but I want to know about all > possibilities. > > Sent from my iPad > > On 26.06.2012, at 21:30, Andrew Berman wrote: > > Actually, found two articles relating to the difference: > > 1. http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ > 2. > http://jlouisramblings.blogspot.com/2011/10/one-major-difference-zeromq-and-erlang.html > > The way I see it, if you're using Erlang, there's no point to using 0MQ. > It really provides the same features, just differently. I feel like 0MQ > would be great for a language that doesn't provide what Erlang does > natively. > > --Andrew > > On Tue, Jun 26, 2012 at 9:53 AM, Alex Shneyderman > wrote: > >> > I've looked at 0MQ before, but I'm not clear what it provides me over >> just >> > using standard Erlang message passing or using something like gproc. >> Can >> > you shed any light on this? >> >> Not over Erlang, no. But you mentioned you wanted to use Rabbit. I >> find Rabbit a bit >> problematic as it will require an extra node to worry about, if you >> want it to be reliable >> you will need more than one. 0MQ is a library that has no such >> requirements it will >> live on any node you start but you will have to encode reliability on >> your own. Hence >> my reference to zguide, which explains exactly how to do it. >> >> Cheers, >> Alex. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Wed Jun 27 19:54:31 2012 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 27 Jun 2012 18:54:31 +0100 Subject: [erlang-questions] nested variable assignment : bug or the expected behaviour ? In-Reply-To: <4FE9A581.7030108@gmail.com> References: <4FE9A581.7030108@gmail.com> Message-ID: Thanks everyone, especially Richard Carlsson for that incredibly detailed response I now realise that there are some very interesting subtitles to it .. So that the original {J=200,J}. has no correct meaning unless J is already bound, but {J=200,J=200}. will interestingly generate a pattern match (and succeed in this case) as will {J=200,J=100} but obviously fail in this case ! also in all cases the J assignment drops out of the expression ( just like case statements! and obv as long as no bad match happened ) which is why the pattern match happens, to ensure that there is a single value of J that can be bound to .. finally the pattern patching is only performed at the end of the expression, not at each place the = is found .. so : [J=begin erlang:display(1), 100 end, J=begin erlang:display(2), 200 end , J=begin erlang:display(3), 100 end ]. would fail after all 3 sub expressions have been evaluated, giving "1 2 3 " out , rather than the more expected "1 2" out .. wow ! Tho yes i do get you point that with excessive use, although the expression scoping rules are in fact very complex it does prevent excessively nested variable binding that could lead to difficult to understand code! James On 26 June 2012 13:05, Richard Carlsson wrote: > On 06/26/2012 01:55 AM, James Churchman wrote: >> >> If i have a function like : >> >> f()->[J=100,J]. >> >> Or put in the shell: >> {J=200,J}. >> >> Or >> >> erlang:'+'(J=10,J). >> >> Or any other combination of where i try to set a variable nested >> inside somewhere else, it will say "variable 'J' is unbound" .. is >> this the expected behaviour ? > > > Yes, it's expected. Each expression and subexpression in Erlang has an input > environment and an output environment (which is the same as the input > environment unless the expression creates additional bindings). Environments > are chained so that in the comma-separated body of a clause, the output from > one expression becomes the input of the next. > > However, arguments to a single expression, such as f(A, B) or {A, B}, are > always evaluated in the same input environment. Hence, even if you write > f(X=1, X), where X is a previously unbound variable, both arguments are > evaluated in the environment that does not contain X, and the compiler > complains about the second X being unknown. > > The output environment from an expression is the combination of the output > environments from its subexpressions, so f(X=1, Y=2) exports both X and Y. > To preserve the constraint that a variable occurring in more than one > pattern has the same value independent of evaluation order, if you write > f(X=a(), X=b()), the compiler will actually insert a test after the > evaluation of the arguments to check that the intermediate results of a() > and b() are in fact the same value and so either can be taken as the value > of X. > > As Richard O'Keefe said in his reply, Erlang uses strict left-to-right > evaluation order also in function arguments (even where it to someone > writing a compiler might seem to be backwards). This is to ensure that an > Erlang program behaves the same regardless of compiler implementation, and > to eliminate a source of confusion: programmers shouldn't have to remember a > bunch of special cases where evaluation order is right-to-left or undefined. > What you see is what you get. > > But at the same time, the known evaluation order (with respect to side > effects) should not be something that programmers can abuse to write > unreadable code like f(1+g(X=h(Y),42),{foo,X}). Therefore, the scoping rules > enforce that bindings are effectively treated as if the evaluation order of > arguments was unknown, or could be changed from one version of Erlang to the > next without changing the meaning of the program. > > ? ?/Richard From freeakk@REDACTED Wed Jun 27 20:15:38 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 27 Jun 2012 22:15:38 +0400 Subject: [erlang-questions] Mystery float binary matching Message-ID: Hi, What is the problem with this code? 1> <<_/native-float>> = <<0,0,0,0,0,0,240,255>>. ** exception error: no match of right hand side value <<0,0,0,0,0,0,240,255>> 2> <<_/native-float>> = <<0,0,0,0,0,0,0,0>>. <<0,0,0,0,0,0,0,0>> -- Best regards, Uvarov Michael From f355@REDACTED Wed Jun 27 21:01:05 2012 From: f355@REDACTED (Konstantin Tcepliaev) Date: Wed, 27 Jun 2012 23:01:05 +0400 Subject: [erlang-questions] Mystery float binary matching In-Reply-To: References: Message-ID: <835241340823665@web23h.yandex.ru> Hello Michael, There's no mystery here. The double-precision float you're trying to match represents -Infinity according to IEEE-754, which has no equivalent in Erlang and is usually represented by atom 'infinity', which is another datatype. More examples that don't match: <<0,0,0,0,0,0,240,127>> - +Infinity <<1,0,0,0,0,0,240,127>> - NaN I'm not sure what you should do if you need to parse the input which may contain those special values. The best I can think of is to match the field in question as an eight bytes binary, then check that binary for specials, and only then match it to float. Also, beware of endianness. Regards, -- Konstantin 27.06.2012, 22:15, "Michael Uvarov" : > Hi, > > What is the problem with this code? > > 1> <<_/native-float>> = <<0,0,0,0,0,0,240,255>>. > ** exception error: no match of right hand side value <<0,0,0,0,0,0,240,255>> > > 2> <<_/native-float>> = <<0,0,0,0,0,0,0,0>>. > <<0,0,0,0,0,0,0,0>> > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Wed Jun 27 21:18:00 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 27 Jun 2012 22:18:00 +0300 Subject: [erlang-questions] =?windows-1252?q?supervisor_with_obstinate_res?= =?windows-1252?q?tart_policy_=97_are_there_any_implementations=3F?= In-Reply-To: References: Message-ID: <4FEB5C68.7040809@gmail.com> On 06/26/2012 12:43 PM, Max Lapshin wrote: > Hi. > > I think that many people have met with OTP supervisor problem: if your > supervisor must work with external resource, > and this resource is down, you get system, brought down after some restarts. > > I think that there are many implementations of trackers, that restart > such jobs and thus reimplement OTP supervisors. > > Have anyone implemented supervisor that is OTP compatible, and doesn't > fail on frequent worker restarts, > but starts to restart less and less frequent? I did some work previously on adding incremental backoff to the OTP supervisors, but in this case you describe, I think that what you need is not a special supervisor, but a Circuit Breaker (see http://en.wikipedia.org/wiki/Circuit_breaker_design_pattern). The idea with supervision is that it is often the case that a restart will fix temporary problems and glitches, by resetting the workers to a known good state. But when it comes to depending on external resources, your supervisor cannot restart the external resource - it can only restart your connection to the resource. If that wasn't the problem, you're still screwed. One way of implementing a circuit breaker in Erlang is as a separate server that acts as a middleman. Anybody who wants to call the external service has to make the request via the circuit breaker. The circuit breaker runs the jobs, tracks status of jobs and detects timeouts, logs warnings, and can decide to block further requests for a while if the external resource seems to be misbehaving, so your logs don't get flooded by a million workers simultaneously discovering that your SMS provider (or whatever) is unavailable. You should also be able to query the circuit breaker about the current status of the resources it monitors, force block/unblock, etc. A generic circuit breaker would be a nice addition to the Erlang libraries, but it's not a supervisor - it's a service, which in itself needs to be supervised (because the rest of the system depends on it). /Richard From freeakk@REDACTED Wed Jun 27 21:53:51 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 27 Jun 2012 23:53:51 +0400 Subject: [erlang-questions] Mystery float binary matching In-Reply-To: <835241340823665@web23h.yandex.ru> References: <835241340823665@web23h.yandex.ru> Message-ID: I use this: try <> = Bin, {Value, Bin2}; catch error:{badmatch, _Bin} -> BitCount = bit_size(<<0/native-float>>), <<_Skip:BitCount, Bin2/binary>> = Bin, {undefined, Bin2} end. From f355@REDACTED Wed Jun 27 22:17:56 2012 From: f355@REDACTED (Konstantin Tcepliaev) Date: Thu, 28 Jun 2012 00:17:56 +0400 Subject: [erlang-questions] Mystery float binary matching In-Reply-To: References: <835241340823665@web23h.yandex.ru> Message-ID: <681411340828276@web4h.yandex.ru> That'll work well with three obvious assumptions: 1. You need to match only one field and that wouldn't change in the future. 2. Your input is coming from the machine with the same endianness. 3. Erlang VM matches the Bin2 second time using pointer to binary heap and not the binary data itself (which seems quite logical, though I'm not completely sure). If these are met, then you're good to go. 27.06.2012, 23:53, "Michael Uvarov" : > I use this: > > ???try > ???????<> = Bin, > ???????{Value, Bin2}; > ????catch error:{badmatch, _Bin} -> > ????????BitCount = bit_size(<<0/native-float>>), > ????????<<_Skip:BitCount, Bin2/binary>> = Bin, > ????????{undefined, Bin2} > ????end. From robert.virding@REDACTED Thu Jun 28 01:08:38 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Jun 2012 00:08:38 +0100 (BST) Subject: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 In-Reply-To: <3a05a3fc-b9da-47f6-8536-8b77918996ff@knuth> Message-ID: <106933cc-7032-42d5-8a0c-7ac351ec6569@knuth> In that case "InitData" is probably a better name than "Argument", which says about as much as "Data". Like the post though. Anything but "Args" as ther is realy only one of it. Robert ----- Original Message ----- > > This is a post talking about why you should not name things "Data" :P > http://petdance.com/2012/04/the-worlds-two-worst-variable-names/ > > Perhaps a better name for what you send to Mod:init/1 in gen_XXX is > "Argument" > so it's more clear from the variable name that it is a single term. > > // Gustav > > Sent from my PC > > ----- Original Message ----- > > From: "Robert Virding" > > To: clist@REDACTED > > Cc: erlang-questions@REDACTED > > Sent: Wednesday, June 27, 2012 2:41:58 PM > > Subject: Re: [erlang-questions] More on this: simple_one_by_one > > trouble and rare start_link/2 > > > > I don't really think that there is an inconsistency here. > > > > In all the cases where something is to be started, supervisor, > > application and proc_lib, the specifier for this is the > > Mod,Fun,Args > > of apply and spawn. Though I will admit that in the application > > case > > the function name is predefined, almost like a call back function. > > And for callbacks all the arguments, number and meaning, are > > predefined. > > > > I personally think the main problems are: > > > > - Inconsistency in the documentation caused by using "Args" to mean > > different things. So in supervisor. application and proc_lib "Args" > > really is a list of arguments in apply/spawn style where the call > > is > > built using the length of the list to determine the arity of the > > called function. In gen_XXX the ONE argument to Mod:init/1 is also > > called "Args" where it can be anything and even if it is a list it > > will only become one argument. Calling it "Data" would have made it > > easier to understand. > > > > - The second argument supervisor:start_child/2 to completely > > different, both type and meaning, depending on whether the > > supervisor is a simple_one_for_one or anything else. Yes, I know > > that it is documented but it IS inconsistent. > > > > I think that in general making all the start functions arity 1 and > > calling them with a list of arguments is of no real benefit. Apart > > from simple_one_for_one supervisors the arguments to the start > > function are anyway fixed so there would be no real gain here > > anyway. > > > > Also it would be a completely backwards incompatible change which > > would cause a lot of rewriting, so I don't see it being applied. > > > > We will just have to live with it. If it is of any consolation > > there > > are changes I would like to make which would also never be made. > > :-) > > > > Robert > > > > ----- Original Message ----- > > > > > > i did'nt find any example of mixing args from the supervirsor > > > childspec > > > and start_child(...) so im going to make my own digging in this > > > issue: > > > > > > Let see How the simple_one_by_one works... > > > > > > handle_call({start_child, EArgs}, _From, State) when > > > ?is_simple(State) -> > > > Child = hd(State#state.children), > > > #child{mfargs = {M, F, A}} = Child, > > > Args = A ++ EArgs, > > > case do_start_child_i(M, F, Args) of > > > <------>{ok, undefined} when Child#child.restart_type =:= > > > temporary > > > -> > > > <------> {reply, {ok, undefined}, State}; > > > <------>{ok, Pid} -> > > > <------> NState = save_dynamic_child(Child#child.restart_type, > > > Pid, Args, State), > > > <------> {reply, {ok, Pid}, NState}; > > > <------>{ok, Pid, Extra} -> > > > <------> NState = save_dynamic_child(Child#child.restart_type, > > > Pid, Args, State), > > > <------> {reply, {ok, Pid, Extra}, NState}; > > > <------>What -> > > > <------> {reply, What, State} > > > end; > > > > > > > > > Shouldnt be do_start_child_i(M,F,EArgs) of the form : > > > > > > do_start_child(M,F,[EArgs]) ?? > > > > > > or the inner call apply(M,F,[A])? > > > > > > Provided the docs state that your unique exported funcion must > > > be > > > of > > > arity one you should > > > expect that code will take measures in order to enforce arity > > > one, > > > enclosing whatever you pass it in a List on the apply phase. > > > > > > The semanctics of apply use a list to KNOW how many args the > > > target > > > call has, while the > > > semantics of sup childspec + sup start_child use a list ++ op to > > > know > > > how many args will be passed > > > to your callback of arity ONE. > > > > > > So for the apply part all your functions are arity ONE > > > > > > Without the ability to join Args from the ChildSpec and from the > > > start child call i would concur that you should > > > enclose in a list whatever you want to be as a sole argument and > > > this > > > code be ok, but as soon as you can put > > > two or more args one on the ChildSpec and one on the call Sup > > > code > > > must enforce that only one arg APPLY > > > call is made so you end calling start_link/1 whatever you pass on > > > the > > > args... > > > > > > Even the DOCS state "[ term() ]" as the MFA on the ChildSpec > > > > > > > > > My stdlib-1.18.1.pdf says (Pag 369) > > > > > > "start_child(SupRef, ChildSpec) -> startchild_ret() > > > Types: > > > SupRef = sup_ref() > > > ChildSpec = child_spec() | (List :: [term()]) > > > ... > > > ... > > > > > > ...If the case of a simple_one_for_one supervisor, the child > > > specification defined in Module:init/1 will be > > > used and ChildSpec should instead be an arbitrary list of terms > > > List. > > > The child process will then be started by > > > appending List to the existing start function arguments, i.e. by > > > calling apply(M, F, A++List) where {M,F,A} > > > is the start function defined in the child specification." > > > > > > Here IMHO lies the error, as A is a list we have [any()...] ++ > > > [any()...] is a list [any()...,any()...] and thats get us an > > > apply > > > call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} > > > and > > > you try provide a [..] in your start_child it will fail > > > unless both argslists are void. > > > > > > So a combination of: > > > > > > init(Opts) -> > > > io:format("[GROUP SUP] Ready to manage MuC with opts: > > > ~p\'s\n",[Opts]), > > > > > > {ok, { > > > {simple_one_for_one, 1, 60}, > > > [ > > > {mucfsm, {sim_group_fsm, start_link, > > > [arg1,arg2]}, transient, 60, worker, [sim_group]} > > > ] > > > } > > > }. > > > > > > > > > and a client that makes a call like: > > > ... > > > {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), > > > ... > > > > > > Gets you a final apply call {M,F,A} where A is in the form > > > [arg1,arg2,arg3,arg4] > > > not [[arg1,arg2,argf3,arg4]] at is should be when you intent to > > > call > > > a start_link/1.... > > > > > > > > > > > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > > > [QuickCheck] [PropEr] [Erl0MQ] > > > Starting simple_one_by_one group supervisor... > > > [GROUP SUP] staring with argument: [{option1,40}] > > > Eshell V5.9.1 (abort with ^G) > > > 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s > > > Adding a child dynamically... > > > {"init terminating in > > > do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.erl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{line,344}]}, > > > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]}]}} > > > > > > > > > /Angel > > > > > > On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel > > > escribi?: > > > > That's right... > > > > > > > > But start_link it is expected to be or arity 3 and Opts is a > > > > property list. > > > > > > > > but then on test:main/0 when i do > > > > supervisor:start_child(SupPid,Opts2) > > > > > > > > being Opts2 another property list, then suppervisor should > > > > call: > > > > > > > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 > > > > which > > > > in turn > > > > calls gen_fsm:start_link, thus providing a Pid in the end... > > > > > > > > but what it really does is apply(sim_group_fsm,start_link,Opts > > > > ++ > > > > Opts2,[ > > > > ]) and then crashes unless i declare a start_link/2 on my > > > > gen_fsm... > > > > > > > > { > > > > {badmatch, > > > > {error, > > > > {'EXIT', > > > > {undef, > > > > [ > > > > {sim_group_fsm,start_link,[[{option1,40}],[{option29,100}]],[]}, > > > > {supervisor,do_start_child_i,3,[ > > > > {file,"supervisor.erl"}, > > > > {line,319} > > > > ]}...... > > > > > > > > i dont see why i finish having a apply/4 call in supervisor: > > > > 319 > > > > instead of > > > > an apply/3... > > > > > > > > I want the sup to propagate one property list to children specs > > > > while i > > > > will provide another during children spawns providing that sup > > > > combines > > > > two in any way children will manage to demunge ...both property > > > > lists > > > > > > > > So cant just see what im doing wrong.... > > > > > > > > /angel > > > > > > > > > > > > PS: i changed a bit the test code to force sup crash > > > > > > > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > > > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez > > > > > Miguel > > > > > wrote: > > > > > } While calling a simple_one_by_one supervisor to start a > > > > > gen > > > > > fsm I > > > > > found } it causes calling a start_link/2 function on the > > > > > callback > > > > > module of the } gen_fsm. > > > > > > > > > > You are specifying that function to be called to start the > > > > > worker > > > > > yourself. In your module group_supervisor.erl you provide a > > > > > child_spec() > > > > > in the > > > > > > > > > > return value of init/1: > > > > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, > > > > > 60, > > > > > worker, > > > > > > > > > > [sim_group]} > > > > > > > > > > The form of which is: > > > > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > > > > > > > The second argument, StartFunc, has the form: > > > > > {Module, Function, Arguments} > > > > > > > > > > You provided: > > > > > {sim_group_fsm, start_link, [Opts]} > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu Jun 28 01:24:18 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Jun 2012 00:24:18 +0100 (BST) Subject: [erlang-questions] Parameterized module initialization In-Reply-To: Message-ID: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> I quite agree. Having finally got rid of {M,F} please,please,please don't replace it with something even worse. I am no great fan of parametrised modules but if they are to be properly added to the language then do it as an opaque data type and use the module:new function to initialise it. Otherwise we might as well open up funs as well so we can really have fun.* Robert * that was being sarcastic. ----- Original Message ----- > Hi, > > On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong > wrote: > > There is an undocumented way of doing this (which one day will > > become standard) > > Suppose we define X as follows: > >> X = {mod1, a,b,c}. > > {mod1, a,b,c}. > > X is now an instance of a parameterised module. if we now call the > > function X:func1(x,y) then what actually gets called is the > > function > > mod1:func1(x,y,{mod1,a,b,c}) > > I hope that you are expressing some form of sarcasm, Joe. > > If I may use such language, it is horrible. Just as we got (almost) > rid of {M,F} to the benefit of proper function references, are we > going to get another kludge? A proper module reference is the way to > go here. Then there are plenty of nice things that can be supported > in > a clean way. > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu Jun 28 01:30:37 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Jun 2012 00:30:37 +0100 (BST) Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <8CDCDD35-375D-447F-AB39-B8D1C34E8CB1@cs.otago.ac.nz> Message-ID: Definitely, things need to be tested before being added to the language. Many ideas which sound great when you describe them are terrible when you actually use them, or in the best case need some tuning before being added. I think the only thing we really miss is process for testing new things before deciding whether they go or stay. Something along the lines of deprecating things before removing them. Couldn't we have an "experimental" tag so the compiler/system verbosely flags things as experimental until a decision is made. In that way they are there, you can test them but you know not to depend on them as they may go away. The say nothing is experimental for more than one release. Robert ----- Original Message ----- > > On 27/06/2012, at 5:33 AM, Andrew Berman wrote: > > > I'd also like to know why there are "hidden" features of Erlang in > > the first place. > > > Because it isn't dead. New ideas get tried, and sane designers don't > take it > for granted that their precious new idea is going to be right first > time. > So initial implementations get short-cuts so that people can get some > experience with a new feature before designing and documenting the > Real Thing. > > Strictly speaking, in this case it isn't the *feature* (parameterised > modules) > that is hidden but the *implementation* of that feature. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From steven.charles.davis@REDACTED Thu Jun 28 01:49:25 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 27 Jun 2012 16:49:25 -0700 (PDT) Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEAFE02.2050009@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <1561c77d-a1c4-4c3e-ac7d-e7dc2ff1c38e@googlegroups.com> <4FEAFE02.2050009@gmail.com> Message-ID: Hi Richard, Accepted and thanks for taking the time to write the explanation for me (and hopefully it will clarify that question for others too). ...so this is more like currying than state. Thanks again, /s On Wednesday, June 27, 2012 7:35:14 AM UTC-5, Richard Carlsson wrote: > > On 06/27/2012 03:15 AM, Steve Davis wrote: > > Is it just me, or is hidden state in a function that appears to the > > client as stateless always bad news... eventually? > > Yes, but this isn't about state. That's what makes it different from > object oriented programming. Once you create M=mymodule:new(42), > M is always the same M. Nothing that anybody else does can change the > module instance behind your back. This is just as referentially > transparent as the following: > > make_adder(X) -> fun (Y) -> Y+X end. > > add(A,B) -> > F = make_adder(B), > F(A). > > Using parameterized modules, this could be written > > -module(adder, [X]). > -export([add/1]). > add(Y) -> Y+X. > > -module(foo). > add(A,B) -> > Adder=adder:new(B), > Adder:add(A). > > The main difference between funs and parameterized modules are that > modules allow you to have several differently named entry points with > possibly different arities. But you can do that with funs as well if you > want; it's just more clumsy: > > make_stepper(C) -> fun ({forward, N}) -> C*N; > ({backward, N}) -> -C*N > end. > > compare: > > -module(stepper, [C]). > -export([forward/1,backward/1]). > forward(N) -> C*N. > backward(N) -> -C*N. > > Also important is that parameterized modules are by design not tied to a > particular version of the code (they always call the latest version of > the module). Funs don't survive reloading the code for the module twice, > but a module instance can be kept around for a long time. > > A caveat is that (just like funs) existing instances stop working if you > change the interface and reload the code - for example, changing > -module(foo, [X]) to -module([X,Y]). That's why I recommend that client > code should probably never call your_module:new(...) directly, but > always via some kind of facade, so clients don't know the actual names > of the modules they instantiate and thus you can update the module name > along with the interface if you need to to that sort of incompatible > change: e.g., "make_mod(X) -> mod_v1:new(X)." becomes "make_mod(X,Y) -> > mod_v2:new(X,Y)." > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Thu Jun 28 03:23:06 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 27 Jun 2012 19:23:06 -0600 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> Message-ID: On Tue, Jun 26, 2012 at 6:53 AM, Joe Armstrong wrote: > There is an undocumented way of doing this (which one day will become > standard) > > Suppose we define X as follows: > > > X = {mod1, a,b,c}. > {mod1, a,b,c}. > > X is now an instance of a parameterised module. if we now call the > function X:func1(x,y) then what actually gets called is the function > mod1:func1(x,y,{mod1,a,b,c}) > > So for example if we define a module sneaky.erl like this: > > -module(sneaky). > -export([test/2, hide/1]). > > test(X, Y) -> {sneaky, X, Y}. > > hide(Something) -> > {sneaky, Something}. > > Then in the shell we can do this: > > 1> c(sneaky). > {ok,sneaky} > 2> X = sneaky:hide({rectange,10,file,"foo"}). > {sneaky,{rectange,10,file,"foo"}} > 3> X:test(abc). > {sneaky,abc,{sneaky,{rectange,10,file,"foo"}}} > > Cheers > Personally I love this form and use it my self. I like parametrized modules, but I do not use the normal decoration, I build it up myself, just like this. It is wonderfully obvious to me. Although I usually have a parent module use a helper module to handle all that, but I still do it explicitly as it is easy for me to forget about the decorations otherwise. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jun 28 08:11:24 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 28 Jun 2012 18:11:24 +1200 Subject: [erlang-questions] Intel Xeon Phi Message-ID: <2BCFF0DD-8435-4E79-939E-70C74FA4500D@cs.otago.ac.nz> http://gpuscience.com/hpc/intel-announces-xeon-phi-coprocessor-based-many-integrated-core-intel-mic-architecture/?goback=.gde_122502_member_125590167 "Made with Intel?s innovative 22nm, 3-D tri-gate transistors, the Intel Xeon Phi coprocessor, available in a PCIe form factor, contains more than 50 cores and a minimum of 8GB of GDDR5 memory." I don't expect to get my hands on one before they're obsolete, but I can't help wondering what if anything it will take to get Erlang running on one... From essen@REDACTED Thu Jun 28 10:00:56 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 10:00:56 +0200 Subject: [erlang-questions] Non-Erlang dependencies Message-ID: <4FEC0F38.7020700@ninenines.eu> Hey, I was wondering how you guys currently handle non-Erlang project dependencies, with regards to simple development but also releases handling. For example if you are writing a web application you are certainly going to depend on something like jquery+bootstrap. Do you simply put them in your own repos? Can't use rebar for this (yet) but maybe you use a nifty trick? Another example would be a C application that you communicate with through ports. Any feedback welcome, thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines From dva.traktorista@REDACTED Thu Jun 28 10:10:08 2012 From: dva.traktorista@REDACTED (=?utf-8?B?ZHZhLnRyYWt0b3Jpc3RhQGdtYWlsLmNvbQ==?=) Date: Thu, 28 Jun 2012 11:10:08 +0300 Subject: [erlang-questions] =?utf-8?q?Non-Erlang_dependencies?= Message-ID: <4fec1185.a86a980a.5cbe.4ce4@mx.google.com> I prefer to have a shell script that builds non-Erlang project environment and believe that most do so. In my opinion - having deps in one's repo is a bad practice. ----- Reply message ----- From: "Lo?c Hoguin" Date: Thu, Jun 28, 2012 11:00 Subject: [erlang-questions] Non-Erlang dependencies To: "Erlang Users' List" Hey, I was wondering how you guys currently handle non-Erlang project dependencies, with regards to simple development but also releases handling. For example if you are writing a web application you are certainly going to depend on something like jquery+bootstrap. Do you simply put them in your own repos? Can't use rebar for this (yet) but maybe you use a nifty trick? Another example would be a C application that you communicate with through ports. Any feedback welcome, thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From clist@REDACTED Thu Jun 28 10:10:34 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Thu, 28 Jun 2012 10:10:34 +0200 Subject: [erlang-questions] =?iso-8859-1?q?More_on_this=3A_simple=5Fone=5F?= =?iso-8859-1?q?by=5Fone_trouble_and_rare=09start=5Flink/2?= In-Reply-To: References: Message-ID: <201206281010.35303.clist@uah.es> How much is used this feature? How many people use it ? I dont think so many, after a couple of greps along my installed OTP packages i would say i dont think anyone is using this simple_one_for_one feaure as all MFA's in child specs seem to have the A part as [ ] so the args count in fixed by what you put on start_child/2. The best i cant say in my limited knowledge is that this feature is nullified buy this fact so apparently almost nobody is using it, hence the dumb patch i did. However i didnt meant having all callbacks as init/1 but in this case provided that start_child can have arbitrary args list this would make your posible callbacks arity any, at any time, upon what you put on the args. this is error prone at least In the best case scenario you can pass both args surrounded in list so you get a Module:init/2 callback to be called.... better than nothing. Do in the end maybe it is worth check if spec args are [] (the usual case) and keep doing a simple ++ but in other case do a better list joint and keep your callback arity sane as documented... I think it deserve at least be documented so other guys dont get stuck with this on the future . /Angel On Mi?rcoles, 27 de Junio de 2012 14:41:58 usted escribi?: > I don't really think that there is an inconsistency here. > > In all the cases where something is to be started, supervisor, application > and proc_lib, the specifier for this is the Mod,Fun,Args of apply and > spawn. Though I will admit that in the application case the function name > is predefined, almost like a call back function. And for callbacks all the > arguments, number and meaning, are predefined. > > I personally think the main problems are: > > - Inconsistency in the documentation caused by using "Args" to mean > different things. So in supervisor. application and proc_lib "Args" really > is a list of arguments in apply/spawn style where the call is built using > the length of the list to determine the arity of the called function. In > gen_XXX the ONE argument to Mod:init/1 is also called "Args" where it can > be anything and even if it is a list it will only become one argument. > Calling it "Data" would have made it easier to understand. > > - The second argument supervisor:start_child/2 to completely different, > both type and meaning, depending on whether the supervisor is a > simple_one_for_one or anything else. Yes, I know that it is documented but > it IS inconsistent. > > I think that in general making all the start functions arity 1 and calling > them with a list of arguments is of no real benefit. Apart from > simple_one_for_one supervisors the arguments to the start function are > anyway fixed so there would be no real gain here anyway. > > Also it would be a completely backwards incompatible change which would > cause a lot of rewriting, so I don't see it being applied. > > We will just have to live with it. If it is of any consolation there are > changes I would like to make which would also never be made. :-) > > Robert Ok ok, at least im not completely wrong, ill try to overcome this with other approach like passing a pid of something else so children can ask some adult whatever they know about their new playground. Maybe things like this deserve a place in a TODO list for a future rewrite or a OTP next-generation saga.... As always the good side of the history is a better understanding on the inner working of the OTP beats. Thanks Angel > > ----- Original Message ----- > > > i did'nt find any example of mixing args from the supervirsor > > childspec > > and start_child(...) so im going to make my own digging in this > > issue: > > > > Let see How the simple_one_by_one works... > > > > handle_call({start_child, EArgs}, _From, State) when > > ?is_simple(State) -> > > > > Child = hd(State#state.children), > > #child{mfargs = {M, F, A}} = Child, > > Args = A ++ EArgs, > > case do_start_child_i(M, F, Args) of > > > > <------>{ok, undefined} when Child#child.restart_type =:= temporary > > -> > > <------> {reply, {ok, undefined}, State}; > > <------>{ok, Pid} -> > > <------> NState = save_dynamic_child(Child#child.restart_type, > > Pid, Args, State), > > <------> {reply, {ok, Pid}, NState}; > > <------>{ok, Pid, Extra} -> > > <------> NState = save_dynamic_child(Child#child.restart_type, > > Pid, Args, State), > > <------> {reply, {ok, Pid, Extra}, NState}; > > <------>What -> > > <------> {reply, What, State} > > > > end; > > > > Shouldnt be do_start_child_i(M,F,EArgs) of the form : > > > > do_start_child(M,F,[EArgs]) ?? > > > > or the inner call apply(M,F,[A])? > > > > Provided the docs state that your unique exported funcion must be of > > arity one you should > > expect that code will take measures in order to enforce arity one, > > enclosing whatever you pass it in a List on the apply phase. > > > > The semanctics of apply use a list to KNOW how many args the target > > call has, while the > > semantics of sup childspec + sup start_child use a list ++ op to know > > how many args will be passed > > to your callback of arity ONE. > > > > So for the apply part all your functions are arity ONE > > > > Without the ability to join Args from the ChildSpec and from the > > start child call i would concur that you should > > enclose in a list whatever you want to be as a sole argument and this > > code be ok, but as soon as you can put > > two or more args one on the ChildSpec and one on the call Sup code > > must enforce that only one arg APPLY > > call is made so you end calling start_link/1 whatever you pass on the > > args... > > > > Even the DOCS state "[ term() ]" as the MFA on the ChildSpec > > > > > > My stdlib-1.18.1.pdf says (Pag 369) > > > > "start_child(SupRef, ChildSpec) -> startchild_ret() > > Types: > > SupRef = sup_ref() > > ChildSpec = child_spec() | (List :: [term()]) > > ... > > ... > > > > ...If the case of a simple_one_for_one supervisor, the child > > specification defined in Module:init/1 will be > > used and ChildSpec should instead be an arbitrary list of terms List. > > The child process will then be started by > > appending List to the existing start function arguments, i.e. by > > calling apply(M, F, A++List) where {M,F,A} > > is the start function defined in the child specification." > > > > Here IMHO lies the error, as A is a list we have [any()...] ++ > > [any()...] is a list [any()...,any()...] and thats get us an apply > > call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} and > > you try provide a [..] in your start_child it will fail > > unless both argslists are void. > > > > So a combination of: > > > > init(Opts) -> > > > > io:format("[GROUP SUP] Ready to manage MuC with opts: > > ~p\'s\n",[Opts]), > > > > {ok, { > > > > {simple_one_for_one, 1, 60}, > > [ > > > > {mucfsm, {sim_group_fsm, start_link, > > [arg1,arg2]}, transient, 60, worker, [sim_group]} > > > > ] > > } > > > > }. > > > > and a client that makes a call like: > > ... > > > > {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), > > > > ... > > > > Gets you a final apply call {M,F,A} where A is in the form > > [arg1,arg2,arg3,arg4] > > not [[arg1,arg2,argf3,arg4]] at is should be when you intent to call > > a start_link/1.... > > > > > > > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > > [async-threads:0] [hipe] [kernel-poll:false] > > > > [QuickCheck] [PropEr] [Erl0MQ] > > Starting simple_one_by_one group supervisor... > > [GROUP SUP] staring with argument: [{option1,40}] > > Eshell V5.9.1 (abort with ^G) > > 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s > > Adding a child dynamically... > > {"init terminating in > > do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg > > 1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.e > > rl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{lin > > e,344}]}, > > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib > > ,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0 > > ,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]} > > ]}} > > > > > > /Angel > > > > On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel > > > > escribi?: > > > That's right... > > > > > > But start_link it is expected to be or arity 3 and Opts is a > > > property list. > > > > > > but then on test:main/0 when i do > > > supervisor:start_child(SupPid,Opts2) > > > > > > being Opts2 another property list, then suppervisor should call: > > > > > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 which > > > in turn > > > calls gen_fsm:start_link, thus providing a Pid in the end... > > > > > > but what it really does is apply(sim_group_fsm,start_link,Opts ++ > > > Opts2,[ > > > ]) and then crashes unless i declare a start_link/2 on my > > > gen_fsm... > > > > > > { > > > > > > {badmatch, > > > > > > {error, > > > > > > {'EXIT', > > > > > > {undef, > > > > > > [ > > > {sim_group_fsm,start_link,[[{option1,40}], [{option29,100}]],[]}, > > > {supervisor,do_start_child_i,3,[ > > > > > > {file,"supervisor.erl"}, > > > {line,319} > > > ]}...... > > > > > > i dont see why i finish having a apply/4 call in supervisor: 319 > > > instead of > > > an apply/3... > > > > > > I want the sup to propagate one property list to children specs > > > while i > > > will provide another during children spawns providing that sup > > > combines > > > two in any way children will manage to demunge ...both property > > > lists > > > > > > So cant just see what im doing wrong.... > > > > > > /angel > > > > > > > > > PS: i changed a bit the test code to force sup crash > > > > > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez Miguel > > > > wrote: > > > > } While calling a simple_one_by_one supervisor to start a gen > > > > fsm I > > > > found } it causes calling a start_link/2 function on the > > > > callback > > > > module of the } gen_fsm. > > > > > > > > You are specifying that function to be called to start the worker > > > > yourself. In your module group_supervisor.erl you provide a > > > > child_spec() > > > > in the > > > > > > > > return value of init/1: > > > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, 60, > > > > worker, > > > > > > > > [sim_group]} > > > > > > > > The form of which is: > > > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > > > > > The second argument, StartFunc, has the form: > > > > {Module, Function, Arguments} > > > > > > > > You provided: > > > > {sim_group_fsm, start_link, [Opts]} > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From prof3ta@REDACTED Thu Jun 28 10:18:35 2012 From: prof3ta@REDACTED (Roberto Aloi) Date: Thu, 28 Jun 2012 10:18:35 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <4FEC0F38.7020700@ninenines.eu> References: <4FEC0F38.7020700@ninenines.eu> Message-ID: In the past, I've added bootstrap as a GIT Submodule and I simply included it in the release. This approach didn't work out too well (submodules can be painful). In the end, I've decided to go for the simple route and to add the bootstrap files to the repo. A bit of duplication there, but a lot of simplicity. @dva: Having a bash script which gets and builds stuff assumes that you have (Internet) access to the resources you have to build at building time. Sometimes, that's not the case. R On Thu, Jun 28, 2012 at 10:00 AM, Lo?c Hoguin wrote: > Hey, > > I was wondering how you guys currently handle non-Erlang project > dependencies, with regards to simple development but also releases handling. > > For example if you are writing a web application you are certainly going to > depend on something like jquery+bootstrap. Do you simply put them in your > own repos? Can't use rebar for this (yet) but maybe you use a nifty trick? > > Another example would be a C application that you communicate with through > ports. > > Any feedback welcome, thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Roberto Aloi --- Website: http://roberto-aloi.com Twitter: @robertoaloi From bchesneau@REDACTED Thu Jun 28 10:29:02 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 28 Jun 2012 10:29:02 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <4FEC0F38.7020700@ninenines.eu> References: <4FEC0F38.7020700@ninenines.eu> Message-ID: On Thu, Jun 28, 2012 at 10:00 AM, Lo=EFc Hoguin wrote: > Hey, > > I was wondering how you guys currently handle non-Erlang project > dependencies, with regards to simple development but also releases handli= ng. > > For example if you are writing a web application you are certainly going = to > depend on something like jquery+bootstrap. Do you simply put them in your > own repos? Can't use rebar for this (yet) but maybe you use a nifty trick= ? > Since you want to rely on one specific version of your libs you may want sometimes to have them in your own repo. Which is generally the case for such things like jquery and bootstrap. Depending on your need you can either separate the web part in its own repo or put in the priv dir of your app or watever. Using the rebar template for a release you can also move folders in your app. > Another example would be a C application that you communicate with throug= h > ports. > Against it depeds. Can the C app be installed with the system? Do you want to be multiplatform ? Doy you want to distribute all the code in your archive? At which point could you find these dependencies on the systems you target ? DO you want to build your application staticaly against this C code.... Then there are different patterns. Among them: - Put the c code in c_src and build your prot/nif against it - Get the C code on build (using curl or other) and build against it - Build your port against shared libs installed on your system - wrap your build with a script/program that will build/fetch the dependencies for you These days I'm investigating the use of repo [1] wich allows you to unify different repositories in your project and let you upload changes to them. (So you keep the repository separation). Difference with using rebar get-deps is that I can still upload changes in this repositories directly. Hope it helps, - beno=EEt [1] http://source.android.com/source/version-control.html From bchesneau@REDACTED Thu Jun 28 10:30:05 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 28 Jun 2012 10:30:05 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: References: <4FEC0F38.7020700@ninenines.eu> Message-ID: > Against it depeds. s/Against i depeds/Again it depends From n.oxyde@REDACTED Thu Jun 28 10:51:47 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 28 Jun 2012 10:51:47 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEB2FED.8030500@gmail.com> References: <06139A918ACCA041BF46A0F36940C7FA39CCBAEC@exch-mbx2.msk.trd.ru> <734372BD-959C-40DA-9519-61DF76425205@gmail.com> <4FEA0036.6020508@gmail.com> <8507A709-7DF3-4598-A4F1-2FEDBB605862@gmail.com> <4FEB2FED.8030500@gmail.com> Message-ID: There is no point in trying to enforce that with the current implementation, you're right; but what I would like is proper functors where BEAM knows about them and not just modules transformed at compile-time. A man can always dream, right? -- Anthony Ramine Le 27 juin 2012 ? 18:08, Richard Carlsson a ?crit : > In Erlang, however, there is no reason to limit the type of a module parameter. Indeed, the only way to enforce it would be to add runtime checks at each entry point. The implementation is not simplified - it gets more complicated. In practice, it is probably a good idea to mostly parameterize modules with respect to other modules (i.e., callbacks), but there's no point in trying to enforce that. From erlang@REDACTED Thu Jun 28 11:03:46 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 28 Jun 2012 11:03:46 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> Message-ID: On Thu, Jun 28, 2012 at 1:24 AM, Robert Virding wrote: > I quite agree. Having finally got rid of {M,F} please,please,please don't replace it with something even worse. I am no great fan of parametrised modules but if they are to be properly added to the language then do it as an opaque data type and use the module:new function to initialise it. But it's not worse. The "sneaky" mechanism has the additional advantages over paramterised module: 1) Variables in the body of a function "appear by magic" - ie they are not declared in the head of the function. So I can't just cut and paste code from one module to another. 2) You can only have one "new" constructor (I am correct here) suppose I have one group of function in a module that need access to module parameter variables A,B,C and some other set of functions needs different variables P,Q,R - If I understand correctly this can't be done (this should be solved by a letrec, but that's a different discussion) 3) This provides a very convenient syntax for the user of a module X:some_function(...) is very familiar to users of other programming languages (like Javascript) - the idea that X contains some state is "normal" for most programmers coming to Erlang from other languages /Joe > > Otherwise we might as well open up funs as well so we can really have fun.* > > Robert > > * that was being sarcastic. > > ----- Original Message ----- >> Hi, >> >> On Tue, Jun 26, 2012 at 2:53 PM, Joe Armstrong >> wrote: >> > There is an undocumented way of doing this (which one day will >> > become standard) >> > Suppose we define X as follows: >> >> X = {mod1, a,b,c}. >> > {mod1, a,b,c}. >> > X is now an instance of a parameterised module. if we now call the >> > function X:func1(x,y) then what actually gets called is the >> > function >> > mod1:func1(x,y,{mod1,a,b,c}) >> >> I hope that you are expressing some form of sarcasm, Joe. >> >> If I may use such language, it is horrible. Just as we got (almost) >> rid of {M,F} to the benefit of proper function references, are we >> going to get another kludge? A proper module reference is the way to >> go here. Then there are plenty of nice things that can be supported >> in >> a clean way. >> >> best regards, >> Vlad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From watson.timothy@REDACTED Thu Jun 28 11:09:14 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 28 Jun 2012 10:09:14 +0100 Subject: [erlang-questions] =?windows-1252?q?supervisor_with_obstinate_res?= =?windows-1252?q?tart_policy_=97_are_there_any_implementations=3F?= In-Reply-To: <4FEB5C68.7040809@gmail.com> References: <4FEB5C68.7040809@gmail.com> Message-ID: <6F133B91-AA62-4C86-BDDF-E2F5FDEB3C53@gmail.com> On 27 Jun 2012, at 20:18, Richard Carlsson wrote: > On 06/26/2012 12:43 PM, Max Lapshin wrote: >> Hi. >> >> I think that many people have met with OTP supervisor problem: if your >> supervisor must work with external resource, >> and this resource is down, you get system, brought down after some restarts. >> >> I think that there are many implementations of trackers, that restart >> such jobs and thus reimplement OTP supervisors. >> >> Have anyone implemented supervisor that is OTP compatible, and doesn't >> fail on frequent worker restarts, >> but starts to restart less and less frequent? > > I did some work previously on adding incremental backoff to the OTP supervisors, but in this case you describe, I think that what you need is not a special supervisor, but a Circuit Breaker (see http://en.wikipedia.org/wiki/Circuit_breaker_design_pattern). The idea with supervision is that it is often the case that a restart will fix temporary problems and glitches, by resetting the workers to a known good state. But when it comes to depending on external resources, your supervisor cannot restart the external resource - it can only restart your connection to the resource. If that wasn't the problem, you're still screwed. > There is a simple back-off (after delay) implementation available in http://hg.rabbitmq.com/rabbitmq-server/file/default/src/supervisor2.erl as well fyi. Cheers, Tim From essen@REDACTED Thu Jun 28 11:23:12 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 11:23:12 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> Message-ID: <4FEC2280.7010509@ninenines.eu> On 06/28/2012 11:03 AM, Joe Armstrong wrote: > On Thu, Jun 28, 2012 at 1:24 AM, Robert Virding > wrote: >> I quite agree. Having finally got rid of {M,F} please,please,please don't replace it with something even worse. I am no great fan of parametrised modules but if they are to be properly added to the language then do it as an opaque data type and use the module:new function to initialise it. > > But it's not worse. The "sneaky" mechanism has the additional advantages over > paramterised module: > > 1) Variables in the body of a function "appear by magic" - ie they are not > declared in the head of the function. So I can't just cut and > paste code from > one module to another. > 2) You can only have one "new" constructor (I am correct here) > suppose I have one group of function in a module that need access to > module parameter variables A,B,C and some other set of functions needs > different variables P,Q,R - If I understand correctly this can't be done > (this should be solved by a letrec, but that's a different discussion) > 3) This provides a very convenient syntax for the user of a module > X:some_function(...) is very familiar to users of other programming > languages (like Javascript) - the idea that X contains some state is > "normal" for most programmers coming to Erlang from other languages Sure, these people are used to X:some_function(...). But are they used to X2 = X:some_function(...)? When they do X:some_function(...) they then do something similar to $this->value = 'blabla' and expect X to be changed magically. So the comparison falls short. The problem with doing what you do is two-folds. First, you make things happen implicitly. It's not obvious what happens exactly unless you know the language. some_function(..., X) is obvious to almost all programmers, X:some_function(...) isn't for the reasons stated above: you can't modify X and expect X to change. I think being explicit helps a lot more readability. Second, you actually make things more confusing. See this small snippet: X = {utf8_string, 1, 2, 3}, X:some_function() %% this calls utf8_string:some_function/1 Y = utf8_string, Y:some_function() %% this calls utf8_string:some_function/0 This means that you need to know what's in your variable to know the function that will be called. It's fine if your application is small and you know every bits of it. But for bigger systems where these variables might be set in different applications or different nodes, you never know what you'll get. This makes things incredibly hard to read and debug. -- Lo?c Hoguin Erlang Cowboy Nine Nines From essen@REDACTED Thu Jun 28 11:49:34 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 11:49:34 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: References: <4FEC0F38.7020700@ninenines.eu> Message-ID: <4FEC28AE.4010105@ninenines.eu> No actually this is interesting. I *think* I can have a small script in a get-deps post_hook to fetch the extra dependencies when you get the Erlang ones. This way you download everything at the same time. I'll try it out. On 06/28/2012 10:18 AM, Roberto Aloi wrote: > @dva: Having a bash script which gets and builds stuff assumes that > you have (Internet) access to the resources you have to build at > building time. Sometimes, that's not the case. > > R > > On Thu, Jun 28, 2012 at 10:00 AM, Lo?c Hoguin wrote: >> Hey, >> >> I was wondering how you guys currently handle non-Erlang project >> dependencies, with regards to simple development but also releases handling. >> >> For example if you are writing a web application you are certainly going to >> depend on something like jquery+bootstrap. Do you simply put them in your >> own repos? Can't use rebar for this (yet) but maybe you use a nifty trick? >> >> Another example would be a C application that you communicate with through >> ports. >> >> Any feedback welcome, thanks. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From bchesneau@REDACTED Thu Jun 28 11:56:26 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 28 Jun 2012 11:56:26 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <4FEC28AE.4010105@ninenines.eu> References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> Message-ID: On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: > No actually this is interesting. I *think* I can have a small script in a > get-deps post_hook to fetch the extra dependencies when you get the Erlang > ones. This way you download everything at the same time. > > I'll try it out. > > This is what we do in refuge for now. Though it would be easier to have all included in rebar. Or maybe as a plugin. The only problem with that approach is when you need to distribute the sources with all the dependencies. - beno?t From ulf@REDACTED Thu Jun 28 12:53:36 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 28 Jun 2012 12:53:36 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEC2280.7010509@ninenines.eu> References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> <4FEC2280.7010509@ninenines.eu> Message-ID: <1F3CB7E0-C71A-4612-920B-7D7E9D6A0725@feuerlabs.com> On 28 Jun 2012, at 11:23, Lo?c Hoguin wrote: > Second, you actually make things more confusing. See this small snippet: > > X = {utf8_string, 1, 2, 3}, > X:some_function() %% this calls utf8_string:some_function/1 > > Y = utf8_string, > Y:some_function() %% this calls utf8_string:some_function/0 > > This means that you need to know what's in your variable to know the function that will be called. It's fine if your application is small and you know every bits of it. But for bigger systems where these variables might be set in different applications or different nodes, you never know what you'll get. This makes things incredibly hard to read and debug. Well, this is just as true for funs, and basically also for gen_servers, where the results of your function calls depend on what state the server has - which is actually worse in many ways, since the code is no longer referentially transparent, and thus much, much more difficult to test. As it happens, we consider this a feature, since interacting with the (ever changing) outside world is part of the problem domain that we want to attack. I have for many years fought many programmers' urge to abuse apply/3 in all kinds of settings. It can truly lead to code that is impossible to read. Even so, I love the meta-programming capabilities of Erlang, and wouldn't want to live without them. You just have to learn when to use them in a way that *improves* readability rather than the other way around. The question here should be, IMHO, given that we already have ways to create closures and behaviors, do we really need another similar feature? I found the technique described by Joe immensely useful in ErlHive, where I also dynamically created these tuples in order to achieve function dispatch with access control. For one thing, it allowed me to do things like: blogs_for_which_i_am_admin() -> Caller = erlhive.user:auth(), [B || {B,_} <- erlhive.user:via_index(blogs, is_admin, Caller)]. (http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/examples/blog/src/flexiblog.erl?revision=69&view=markup) (I.e. dynamically creating an access control environment through parameterized modules) and, on the caller side: fetch() -> erlhive:with_user( <<"user1">>, fun(M) -> Id = M:apply(erlhive.blog.flexiblog, string_to_id, ["ba/user1/1/1"]), [_|_] = M:apply(erlhive.blog.flexiblog, read_article, [Id]) end). (http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/examples/blog/src/blog_test.erl?revision=69&view=markup) Where I set up access by calling with_user(), which returned a parameterized API module, which allowed me to control which functions could be called, and the library code (above) to inspect static variables (actually, virtual functions, so they couldn't be subverted) and learn about the access profile. The code under the covers was not easy to understand, but I do think that the user-level code became quite clean. BR, Ulf W PS Elsewhere, I have done similar things with funs, but I don't think that they are as fit for this particular purpose. Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From watson.timothy@REDACTED Thu Jun 28 13:07:26 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 28 Jun 2012 12:07:26 +0100 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> Message-ID: <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> On 28 Jun 2012, at 10:56, Benoit Chesneau wrote: > On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: >> No actually this is interesting. I *think* I can have a small script in a >> get-deps post_hook to fetch the extra dependencies when you get the Erlang >> ones. This way you download everything at the same time. >> >> I'll try it out. >> That will work fine, just don't *declare* them as dependencies, otherwise rebar will puke when it doesn't find an application resource file. There is a patch in the rebar pull request queue which adds support for non-otp (raw, as the author puts it) dependencies, which just get downloaded but not checked for OTP compliance. If you like this feature, please go vote for it as @dizzyd and @tuncer are deliberating at the moment. I'm going to up-vote it (after playing devil's advocate for a while) as I'd like to use deps for both things without messing around with hooks. >> > This is what we do in refuge for now. Though it would be easier to > have all included in rebar. Or maybe as a plugin. The only problem > with that approach is when you need to distribute the sources with all > the dependencies. > Plugin distribution is a bit of a PITA. It works fine as long as you are happy to 'get-deps compile ' and have the plugin(s) kick-in *only* after the 'compile' command completes successfully. This is fine for most cases, but for dependency handling it doesn't work at all, making like quite onerous if you're trying to do custom dependency management as a plugin. I have no less than 3 custom dependency handling plugins, and none of them really come up to scratch because of this. Again, I think the rebar maintainers/developers are actively looking at this area at the moment so it's worth engaging with them (on the mailing list) if you have things you'd like them to take into consideration. Cheers, Tim From essen@REDACTED Thu Jun 28 13:17:58 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 13:17:58 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <1F3CB7E0-C71A-4612-920B-7D7E9D6A0725@feuerlabs.com> References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> <4FEC2280.7010509@ninenines.eu> <1F3CB7E0-C71A-4612-920B-7D7E9D6A0725@feuerlabs.com> Message-ID: <4FEC3D66.2060002@ninenines.eu> On 06/28/2012 12:53 PM, Ulf Wiger wrote: > On 28 Jun 2012, at 11:23, Lo?c Hoguin wrote: > >> Second, you actually make things more confusing. See this small snippet: >> >> X = {utf8_string, 1, 2, 3}, >> X:some_function() %% this calls utf8_string:some_function/1 >> >> Y = utf8_string, >> Y:some_function() %% this calls utf8_string:some_function/0 >> >> This means that you need to know what's in your variable to know the function that will be called. It's fine if your application is small and you know every bits of it. But for bigger systems where these variables might be set in different applications or different nodes, you never know what you'll get. This makes things incredibly hard to read and debug. > > Well, this is just as true for funs, and basically also for gen_servers, > where the results of your function calls depend on what state the > server has - which is actually worse in many ways, since the code > is no longer referentially transparent, and thus much, much more > difficult to test. The result changes depending on state, sure. But gen_server always call the same functions. Plus, gen_server calls are abstracted through an API. So you always call the same function and you may get the same or different result. But I'm not talking about the result. I'm talking about how the same code can call different arities depending on the contents of a variable. Var:func() doesn't always call a function of arity 0, as you'd read it instinctively, you need to look elsewhere to be sure of what is really happening. The syntax Var:func() has two meanings instead of just the one we expect. -- Lo?c Hoguin Erlang Cowboy Nine Nines From essen@REDACTED Thu Jun 28 13:20:16 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 13:20:16 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> Message-ID: <4FEC3DF0.3010908@ninenines.eu> On 06/28/2012 01:07 PM, Tim Watson wrote: > > On 28 Jun 2012, at 10:56, Benoit Chesneau wrote: > >> On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: >>> No actually this is interesting. I *think* I can have a small script in a >>> get-deps post_hook to fetch the extra dependencies when you get the Erlang >>> ones. This way you download everything at the same time. >>> >>> I'll try it out. >>> > > That will work fine, just don't *declare* them as dependencies, otherwise rebar will puke when it doesn't find an application resource file. There is a patch in the rebar pull request queue which adds support for non-otp (raw, as the author puts it) dependencies, which just get downloaded but not checked for OTP compliance. If you like this feature, please go vote for it as @dizzyd and @tuncer are deliberating at the moment. I'm going to up-vote it (after playing devil's advocate for a while) as I'd like to use deps for both things without messing around with hooks. Already did! But that works for downloading them, there's no automated integration with releases AFAIK. A post_hook can download into an application's priv which will be properly copied into the release. -- Lo?c Hoguin Erlang Cowboy Nine Nines From clist@REDACTED Thu Jun 28 13:43:58 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Thu, 28 Jun 2012 13:43:58 +0200 Subject: [erlang-questions] error in gen_fsm:reply/2 docs?? Message-ID: <201206281343.58671.clist@uah.es> Hi again.. gen_fsm:reply/2 does not return true (as stdlib 1.18-1 docs say ) but returns second argument ... children_response(Pids,Data) -> lists:foreach(fun(Child) -> Status = gen_fsm:reply(Child,Data) ,io:format("Sending replies got status = ~p\n",[Status]) end,Pids). "Sending replies got status = {#Ref<0.0.0.765>,0.9999786930675343}" /Angel From ulf@REDACTED Thu Jun 28 14:00:33 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 28 Jun 2012 14:00:33 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <4FEC3D66.2060002@ninenines.eu> References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> <4FEC2280.7010509@ninenines.eu> <1F3CB7E0-C71A-4612-920B-7D7E9D6A0725@feuerlabs.com> <4FEC3D66.2060002@ninenines.eu> Message-ID: <594DA0A0-41EC-4B9E-9F2D-CA0EB3356002@feuerlabs.com> On 28 Jun 2012, at 13:17, Lo?c Hoguin wrote: > But I'm not talking about the result. I'm talking about how the same code can call different arities depending on the contents of a variable. Var:func() doesn't always call a function of arity 0, as you'd read it instinctively, you need to look elsewhere to be sure of what is really happening. The syntax Var:func() has two meanings instead of just the one we expect. Sure, but with the gen_server, it's just a layer of indirection or two, and then you have the same problem. In the first step, it's easy enough: I call an API function, which (normally) immediately maps to a call to gen_server:call(Server, Req). How do I know which code Server will end up calling? Well, by convention, we keep the corresponding handle_call() in the same module. Otherwise, I'd have to try to resolve it, perhaps at runtime, or through a wider code search, to figure out which callback Server actually uses. Once I've found the right handle_call(), I still have to pattern-match mentally on the arguments to find the right clause. *Then*, I know which code actually gets called as a result of my initial operation. The fact that you can't easily pair sends with receives in Erlang code without resorting to convention is a problem, but convention is not to be knocked, of course. That we *can* easily map function calls is a feature, and a very good one. It isn't obvious that we should add language features that start eroding this feature. (In that sense I agree with you, of course. I think this dialogue is mainly an exercise in drilling down to the core arguments.) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From robert.virding@REDACTED Thu Jun 28 14:49:23 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Jun 2012 13:49:23 +0100 (BST) Subject: [erlang-questions] More on this: simple_one_by_one trouble and rare start_link/2 In-Reply-To: <201206281010.35303.clist@uah.es> Message-ID: <68f6b710-5f1b-4796-9d7f-51effd3cee8a@knuth> I agree on the documentation. It is also tricky to teach. :-) I honestly have no idea how often simple_one_for_one is actually used. Or if people use the other restart types and use start_child/2 with a child spec. It would be interesting to find out. Robert ----- Original Message ----- > > How much is used this feature? How many people use it ? > > I dont think so many, after a couple of greps along my installed OTP > packages i would say i dont think anyone is using > this simple_one_for_one feaure as all MFA's in child specs seem to > have the A part as [ ] so the args count in fixed by what you > put on start_child/2. The best i cant say in my limited knowledge is > that this feature is nullified buy this fact so apparently almost > nobody is using it, hence the dumb patch i did. > > However i didnt meant having all callbacks as init/1 but in this case > provided that start_child can have arbitrary args list > this would make your posible callbacks arity any, at any time, upon > what you put on the args. > > this is error prone at least > > In the best case scenario you can pass both args surrounded in list > so you get a Module:init/2 callback to be called.... better > than nothing. > > Do in the end maybe it is worth check if spec args are [] (the usual > case) and keep doing a simple ++ but in other case do a better list > joint > and keep your callback arity sane as documented... > > I think it deserve at least be documented so other guys dont get > stuck with this on the future . > > > /Angel > > On Mi?rcoles, 27 de Junio de 2012 14:41:58 usted escribi?: > > I don't really think that there is an inconsistency here. > > > > In all the cases where something is to be started, supervisor, > > application > > and proc_lib, the specifier for this is the Mod,Fun,Args of apply > > and > > spawn. Though I will admit that in the application case the > > function name > > is predefined, almost like a call back function. And for callbacks > > all the > > arguments, number and meaning, are predefined. > > > > I personally think the main problems are: > > > > - Inconsistency in the documentation caused by using "Args" to mean > > different things. So in supervisor. application and proc_lib "Args" > > really > > is a list of arguments in apply/spawn style where the call is built > > using > > the length of the list to determine the arity of the called > > function. In > > gen_XXX the ONE argument to Mod:init/1 is also called "Args" where > > it can > > be anything and even if it is a list it will only become one > > argument. > > Calling it "Data" would have made it easier to understand. > > > > - The second argument supervisor:start_child/2 to completely > > different, > > both type and meaning, depending on whether the supervisor is a > > simple_one_for_one or anything else. Yes, I know that it is > > documented but > > it IS inconsistent. > > > > I think that in general making all the start functions arity 1 and > > calling > > them with a list of arguments is of no real benefit. Apart from > > simple_one_for_one supervisors the arguments to the start function > > are > > anyway fixed so there would be no real gain here anyway. > > > > Also it would be a completely backwards incompatible change which > > would > > cause a lot of rewriting, so I don't see it being applied. > > > > We will just have to live with it. If it is of any consolation > > there are > > changes I would like to make which would also never be made. :-) > > > > Robert > > Ok ok, at least im not completely wrong, ill try to overcome this > with other > approach like passing a pid of something else so children can ask > some adult > whatever they know about their new playground. > > Maybe things like this deserve a place in a TODO list for a future > rewrite or > a OTP next-generation saga.... > > As always the good side of the history is a better understanding on > the inner > working of the OTP beats. > > Thanks Angel > > > > ----- Original Message ----- > > > > > i did'nt find any example of mixing args from the supervirsor > > > childspec > > > and start_child(...) so im going to make my own digging in this > > > issue: > > > > > > Let see How the simple_one_by_one works... > > > > > > handle_call({start_child, EArgs}, _From, State) when > > > ?is_simple(State) -> > > > > > > Child = hd(State#state.children), > > > #child{mfargs = {M, F, A}} = Child, > > > Args = A ++ EArgs, > > > case do_start_child_i(M, F, Args) of > > > > > > <------>{ok, undefined} when Child#child.restart_type =:= > > > temporary > > > -> > > > <------> {reply, {ok, undefined}, State}; > > > <------>{ok, Pid} -> > > > <------> NState = save_dynamic_child(Child#child.restart_type, > > > Pid, Args, State), > > > <------> {reply, {ok, Pid}, NState}; > > > <------>{ok, Pid, Extra} -> > > > <------> NState = save_dynamic_child(Child#child.restart_type, > > > Pid, Args, State), > > > <------> {reply, {ok, Pid, Extra}, NState}; > > > <------>What -> > > > <------> {reply, What, State} > > > > > > end; > > > > > > Shouldnt be do_start_child_i(M,F,EArgs) of the form : > > > > > > do_start_child(M,F,[EArgs]) ?? > > > > > > or the inner call apply(M,F,[A])? > > > > > > Provided the docs state that your unique exported funcion must > > > be of > > > arity one you should > > > expect that code will take measures in order to enforce arity > > > one, > > > enclosing whatever you pass it in a List on the apply phase. > > > > > > The semanctics of apply use a list to KNOW how many args the > > > target > > > call has, while the > > > semantics of sup childspec + sup start_child use a list ++ op to > > > know > > > how many args will be passed > > > to your callback of arity ONE. > > > > > > So for the apply part all your functions are arity ONE > > > > > > Without the ability to join Args from the ChildSpec and from the > > > start child call i would concur that you should > > > enclose in a list whatever you want to be as a sole argument and > > > this > > > code be ok, but as soon as you can put > > > two or more args one on the ChildSpec and one on the call Sup > > > code > > > must enforce that only one arg APPLY > > > call is made so you end calling start_link/1 whatever you pass on > > > the > > > args... > > > > > > Even the DOCS state "[ term() ]" as the MFA on the ChildSpec > > > > > > > > > My stdlib-1.18.1.pdf says (Pag 369) > > > > > > "start_child(SupRef, ChildSpec) -> startchild_ret() > > > Types: > > > SupRef = sup_ref() > > > ChildSpec = child_spec() | (List :: [term()]) > > > ... > > > ... > > > > > > ...If the case of a simple_one_for_one supervisor, the child > > > specification defined in Module:init/1 will be > > > used and ChildSpec should instead be an arbitrary list of terms > > > List. > > > The child process will then be started by > > > appending List to the existing start function arguments, i.e. by > > > calling apply(M, F, A++List) where {M,F,A} > > > is the start function defined in the child specification." > > > > > > Here IMHO lies the error, as A is a list we have [any()...] ++ > > > [any()...] is a list [any()...,any()...] and thats get us an > > > apply > > > call of arity > 1 ALWAYS even when you chilkdspec has a {M,F,[]} > > > and > > > you try provide a [..] in your start_child it will fail > > > unless both argslists are void. > > > > > > So a combination of: > > > > > > init(Opts) -> > > > > > > io:format("[GROUP SUP] Ready to manage MuC with opts: > > > ~p\'s\n",[Opts]), > > > > > > {ok, { > > > > > > {simple_one_for_one, 1, 60}, > > > [ > > > > > > {mucfsm, {sim_group_fsm, start_link, > > > [arg1,arg2]}, transient, 60, worker, [sim_group]} > > > > > > ] > > > } > > > > > > }. > > > > > > and a client that makes a call like: > > > ... > > > > > > {ok,ChildPid} = supervisor:start_child(SupPid,[arg3,arg4]), > > > > > > ... > > > > > > Gets you a final apply call {M,F,A} where A is in the form > > > [arg1,arg2,arg3,arg4] > > > not [[arg1,arg2,argf3,arg4]] at is should be when you intent to > > > call > > > a start_link/1.... > > > > > > > > > > > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > > > [QuickCheck] [PropEr] [Erl0MQ] > > > Starting simple_one_by_one group supervisor... > > > [GROUP SUP] staring with argument: [{option1,40}] > > > Eshell V5.9.1 (abort with ^G) > > > 1> [GROUP SUP] Ready to manage MuC with opts: [{option1,40}]'s > > > Adding a child dynamically... > > > {"init terminating in > > > do_boot",{{badmatch,{error,{'EXIT',{undef,[{sim_group_fsm,start_link,[arg > > > 1,arg2,arg3,arg4],[]},{supervisor,do_start_child_i,3,[{file,"supervisor.e > > > rl"},{line,319}]},{supervisor,handle_call,3,[{file,"supervisor.erl"},{lin > > > e,344}]}, > > > {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},{proc_lib > > > ,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}}}},[{test,main,0 > > > ,[{file,"test.erl"},{line,11}]},{init,start_it,1,[]},{init,start_em,1,[]} > > > ]}} > > > > > > > > > /Angel > > > > > > On Jueves, 21 de Junio de 2012 09:30:15 Angel J. Alvarez Miguel > > > > > > escribi?: > > > > That's right... > > > > > > > > But start_link it is expected to be or arity 3 and Opts is a > > > > property list. > > > > > > > > but then on test:main/0 when i do > > > > supervisor:start_child(SupPid,Opts2) > > > > > > > > being Opts2 another property list, then suppervisor should > > > > call: > > > > > > > > apply(sim_group_fsm,start_link,Opts ++ Opts2) on its line 319 > > > > which > > > > in turn > > > > calls gen_fsm:start_link, thus providing a Pid in the end... > > > > > > > > but what it really does is apply(sim_group_fsm,start_link,Opts > > > > ++ > > > > Opts2,[ > > > > ]) and then crashes unless i declare a start_link/2 on my > > > > gen_fsm... > > > > > > > > { > > > > > > > > {badmatch, > > > > > > > > {error, > > > > > > > > {'EXIT', > > > > > > > > {undef, > > > > > > > > [ > > > > {sim_group_fsm,start_link,[[{option1,40}], > [{option29,100}]],[]}, > > > > {supervisor,do_start_child_i,3,[ > > > > > > > > {file,"supervisor.erl"}, > > > > {line,319} > > > > ]}...... > > > > > > > > i dont see why i finish having a apply/4 call in supervisor: > > > > 319 > > > > instead of > > > > an apply/3... > > > > > > > > I want the sup to propagate one property list to children specs > > > > while i > > > > will provide another during children spawns providing that sup > > > > combines > > > > two in any way children will manage to demunge ...both property > > > > lists > > > > > > > > So cant just see what im doing wrong.... > > > > > > > > /angel > > > > > > > > > > > > PS: i changed a bit the test code to force sup crash > > > > > > > > On Jueves, 21 de Junio de 2012 06:22:54 Vance Shipley escribi?: > > > > > On Thu, Jun 21, 2012 at 12:30:08AM +0200, Angel J. Alvarez > > > > > Miguel > > > > > wrote: > > > > > } While calling a simple_one_by_one supervisor to start a > > > > > gen > > > > > fsm I > > > > > found } it causes calling a start_link/2 function on the > > > > > callback > > > > > module of the } gen_fsm. > > > > > > > > > > You are specifying that function to be called to start the > > > > > worker > > > > > yourself. In your module group_supervisor.erl you provide a > > > > > child_spec() > > > > > in the > > > > > > > > > > return value of init/1: > > > > > {mucfsm, {sim_group_fsm, start_link, [Opts]}, transient, > > > > > 60, > > > > > worker, > > > > > > > > > > [sim_group]} > > > > > > > > > > The form of which is: > > > > > {Id, StartFunc, Restart, Shutdown, Type, Modules} > > > > > > > > > > The second argument, StartFunc, has the form: > > > > > {Module, Function, Arguments} > > > > > > > > > > You provided: > > > > > {sim_group_fsm, start_link, [Opts]} > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Thu Jun 28 15:16:29 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 28 Jun 2012 17:16:29 +0400 Subject: [erlang-questions] =?utf-8?q?supervisor_with_obstinate_restart_po?= =?utf-8?q?licy_=E2=80=94_are_there_any_implementations=3F?= In-Reply-To: <6F133B91-AA62-4C86-BDDF-E2F5FDEB3C53@gmail.com> References: <4FEB5C68.7040809@gmail.com> <6F133B91-AA62-4C86-BDDF-E2F5FDEB3C53@gmail.com> Message-ID: Seems, that supervisor2 is exactly what I need. From essen@REDACTED Thu Jun 28 15:32:54 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jun 2012 15:32:54 +0200 Subject: [erlang-questions] Parameterized module initialization In-Reply-To: <594DA0A0-41EC-4B9E-9F2D-CA0EB3356002@feuerlabs.com> References: <14e8a71e-7743-46e8-84f6-660e50e34d53@knuth> <4FEC2280.7010509@ninenines.eu> <1F3CB7E0-C71A-4612-920B-7D7E9D6A0725@feuerlabs.com> <4FEC3D66.2060002@ninenines.eu> <594DA0A0-41EC-4B9E-9F2D-CA0EB3356002@feuerlabs.com> Message-ID: <4FEC5D06.2000909@ninenines.eu> On 06/28/2012 02:00 PM, Ulf Wiger wrote: > > On 28 Jun 2012, at 13:17, Lo?c Hoguin wrote: > >> But I'm not talking about the result. I'm talking about how the same code can call different arities depending on the contents of a variable. Var:func() doesn't always call a function of arity 0, as you'd read it instinctively, you need to look elsewhere to be sure of what is really happening. The syntax Var:func() has two meanings instead of just the one we expect. > > Sure, but with the gen_server, it's just a layer of indirection or two, > and then you have the same problem. > > In the first step, it's easy enough: > > I call an API function, which (normally) immediately maps to > a call to gen_server:call(Server, Req). > > How do I know which code Server will end up calling? > > Well, by convention, we keep the corresponding handle_call() > in the same module. Otherwise, I'd have to try to resolve it, perhaps > at runtime, or through a wider code search, to figure out which > callback Server actually uses. > > Once I've found the right handle_call(), I still have to pattern-match > mentally on the arguments to find the right clause. *Then*, I know > which code actually gets called as a result of my initial operation. > > The fact that you can't easily pair sends with receives in Erlang code > without resorting to convention is a problem, but convention is not > to be knocked, of course. That we *can* easily map function calls > is a feature, and a very good one. It isn't obvious that we should > add language features that start eroding this feature. Right, actually more things can go wrong with gen_server, the process may not even be alive anymore when you do the call. But all these issues are abstracted by gen_server, and unless you did something really weird you always get a reply. So the API function call stays intuitive, you just have many different possible return values. If your module exports doit/0 and you call it unknowingly with {module,123}:doit(), you end up with a crash. Not a return value. Or worse it can be doit/1 that is called, which may not even crash and just store your value somewhere ready to crash your code much later down the road. What is less intuitive and requires knowledge is the gen_server code itself, but as you say convention helps. The big difference however is that gen_server isn't an element of the language itself, unlike tuple calls. You can always choose to not use gen_server and write gen_better instead, but you're stuck with the language. -- Lo?c Hoguin Erlang Cowboy Nine Nines From lists.nico.k@REDACTED Thu Jun 28 15:53:39 2012 From: lists.nico.k@REDACTED (Nico Kruber) Date: Thu, 28 Jun 2012 15:53:39 +0200 Subject: [erlang-questions] performance drop in R15A? In-Reply-To: References: Message-ID: <1475803.38StdrHr0q@csr-pc40.zib.de> I ran the git bisect with R14B04 as last good and R15A as last bad. Find the test script attached. It basically compiles and installs Erlang and then compiles Scalaris running its benchmarks and evaluating the quorum_read bench. On our machine, R14B04 is around 7000 reqs/s, R15A around 5200, R15B01 around 5000 and current master around 5500. What you need in order to run this on your machine: 1) checkout otp to $HOME/otp 2) checkout current Scalaris from http://scalaris.googlecode.com/svn/trunk/ to $HOME/scalaris-otp 3) store the attached otp-find_sched_bug.sh in $HOME/ 4) run the git bisect as follows: git bisect start OTP_R15A OTP_R14B04 git bisect run $HOME/otp-find_sched_bug.sh ... There are only 'skip'ped commits left to test. The first bad commit could be any of: 55358c54778ead444e51f565d00175ba887ef182 a67e91e658bdbba24fcc3c79b06fdf10ff830bc9 c12befbdc957f7f166598c6d5143ce27a0d10fa8 We cannot bisect more! bisect run cannot continue any more Results: still good up until 11/11/2011 # good: [4ed347dcaeeb8e8844be532414a11817da3920f5] Merge branch 'anders/diameter/capabilities_cb/OTP-9654' bad starting at 13/11/2011 # bad: [c12befbdc957f7f166598c6d5143ce27a0d10fa8] Merge branch 'rickard/alloc- opt/OTP-7775' the two commits in between don't seem to compile to give you an impression of the differences, here are the bench results of some of the tested commits: 4ed347dcaeeb8e8844be532414a11817da3920f5 (good) quorum_read: "wall clock throughput : 6979.394732929971" c12befbdc957f7f166598c6d5143ce27a0d10fa8 (bad) quorum_read: "wall clock throughput : 5754.395660564197" ca58731b5df58aa2a8b42c583d1ba7bb929e72b2 (bad) quorum_read: "wall clock throughput : 5372.322931190965" 6ff77abac7d54f23e5a5fc465c3746c9cd7ee57b (bad) quorum_read: "wall clock throughput : 5271.737230890335" 2ac3f65eeeac88cf2a9a8b339eab1fee3c94e42a (bad) quorum_read: "wall clock throughput : 5199.73995060559" OTP_R15A (bad) quorum_read: "wall clock throughput : 5265.768884666444" now as you can see, most of the performance is lost with commit c12befbdc9 though further degradations can be seen... Unfortunately, c12befbdc9 is a very big patch working on the core of otp and probably not easy to debug. So how can we continue from here? Nico On Thursday 12 Apr 2012 22:42:05 Gustav Simonsson wrote: > Hi Nico, > > Have you tried running the benchmarks between git bisects, > using R14B04 and R15A as endpoints? > > // Gustav > > Sent from my PC > > ----- Original Message ----- > > > From: "Nico Kruber" > > To: erlang-questions@REDACTED > > Cc: "Lukas Larsson" > > Sent: Thursday, 12 April, 2012 2:47:50 PM > > Subject: Re: [erlang-questions] performance drop in R15A? > > > > is there any news on this? > > Our internal test server has a new additional configuration which > > compiles the > > latest otp from github once per day and tests its performance with > > the > > mentioned benchmarks. > > The resulting graphs for R14B04, R15B and latest git checkout are > > attached. As > > you can see - nothing has changed since then so I suppose it is still > > present > > in R15B01 :( > > > > I don't know how to debug this any further. If there is anything I > > could do to > > help you identify the issue, please tell me. > > > > > > Regards > > Nico Kruber > > > > On Wednesday 14 December 2011 17:41:33 Nico Kruber wrote: > > > ok, I did some profiling and made some optimisations but the > > > differences > > > between R15B (git checkout from master just now) and R14B04 are > > > still the > > > same: > > > > > > First the results of the calls to bench:quorum_read(10, 50000) > > > (best of > > > three) ------------------------------------------------------------ > > > > > > | R14B04 | R15B | time surplus > > > > > > ------------------------------------------------------------ > > > timer:tc, smp8:8 | 22484619| 26641737| +18% > > > timer:tc, smp4:4 | 34009785| 35963489| + 6% > > > timer:tc, smp2:2 | 32863428| 34102038| + 4% > > > timer:tc, smpoff | 53805419| 54214795| + 1% > > > ------------------------------------------------------------ > > > > > > It seems there is some lock-contention or some other problem with > > > smp in our > > > case which hasn't been there before. > > > I also observed that the results using R15B vary quite much, > > > especially with > > > smp8:8 (see below for an example). R14B04 is more stable among > > > different > > > test runs. > > > But still, even without smp, R15B is slower than R14B04 :( > > > > > > Lukas suggested to run fprof to a bit deeper and maybe find the > > > function > > > taking longer in R15 than in R14B04. > > > Although it was helpful in finding some hot spots, it did however > > > not help > > > with this problem. Following are the totals reported by fprof > > > (if you like, I can send the complete logs - didn't want to send > > > too much to > > > the mailing list though). > > > > > > ------------------------------------------------------------ > > > > > > | R14B04 | R15B | time surplus > > > > > > ------------------------------------------------------------ > > > fprof, CNT, smp8:8 | 7723796| 7722635| n/a > > > fprof, ACC, smp8:8 | 98239| 85477| -13% > > > fprof, OWN, smp8:8 | 300534| 264018| -12% > > > ------------------------------------------------------------ > > > > > > As you can see, according to fprof, R15B should be faster (but it > > > isn't!). > > > I assume that some function calls are not reported by fprof > > > (e.g. I did not see erlang:length/1). > > > Is there some way to include them again in order to analyse > > > further? > > > Maybe suspend or garbage_collect are not in ACC or OWN? > > > But even here, R15B wastes less time as you can see in the table > > > below. > > > ---------------------------------------------------------- > > > > > > | R14B04 | R15B | > > > > > > ---------------------------------------------------------- > > > fprof, suspend, smp8:8 | 9.8421e+06| 8.51855e+06| > > > fprof, garbage_collect, smp8:8 | 1315| 794| > > > ---------------------------------------------------------- > > > > > > We do rely quite much on the process dictionary (erlang:put and > > > erlang:get) > > > which you might see as an "non-typical usage pattern" for erlang. > > > However, if I switch to ets instead, performance of > > > bench:quorum_read(10, > > > 50000) for R15B does not change and for R14B04 decreases by about > > > 7%. > > > > > > Everything else I would consider "typical use" - or at least I'm > > > not aware > > > of anything untypical. > > > > > > I hope, someone can help to further analyse the problem. I did > > > everything I > > > could think of and are stuck now. > > > Is other real-world software also affected? E.g. testing couchdb > > > would be > > > nice. > > > > > > > > > Regards > > > Nico Kruber > > > > > > > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > how to re-produce the values > > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > > > > I'm using scalaris from http://code.google.com/p/scalaris/ > > > ./configure && make > > > ================ R15B =============== > > > -------------------------------- > > > ./bin/firstnode.sh > > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] > > > [async-threads:0] [hipe] > > > [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {33764932,ok} % 1st try > > > {26641737,ok} % 2nd try > > > {27575975,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "+S4:4" > > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] > > > [async-threads:0] [hipe] > > > [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {35963489,ok} % 1st try > > > {36332156,ok} % 2nd try > > > {37295819,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "+S2:2" > > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:2:2] > > > [async-threads:0] [hipe] > > > [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {34102038,ok} % 1st try > > > {34379052,ok} % 2nd try > > > {34416173,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "-smp disable" > > > Erlang R15B (erts-5.9) [source] [64-bit] [async-threads:0] [hipe] > > > [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {54214795,ok} % 1st try > > > {55210947,ok} % 2nd try > > > {54537194,ok} % 3rd try > > > > > > ================ R14B04 =============== > > > -------------------------------- > > > ./bin/firstnode.sh > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {22598819,ok} % 1st try > > > {22706185,ok} % 2nd try > > > {22484619,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "+S4:4" > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {35482368,ok} % 1st try > > > {34009785,ok} % 2nd try > > > {34527203,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "+S2:2" > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {33590356,ok} % 1st try > > > {33192033,ok} % 2nd try > > > {32863428,ok} % 3rd try > > > -------------------------------- > > > ./bin/firstnode.sh "-smp disable" > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [rq:1] > > > [async-threads:0] [hipe] > > > [kernel-poll:false] > > > > > > timer:tc(bench, quorum_read, [10, 50000]). > > > {53805419,ok} % 1st try > > > {54054538,ok} % 2nd try > > > {54066521,ok} % 3rd try > > > > > > > > > fprof runs: > > > ./bin/firstnode.sh > > > > > > fprof:trace([start, {procs, processes()}]). > > > bench:quorum_read(10, 1000). > > > fprof:trace(stop). > > > fprof:profile(). > > > fprof:analyse([{dest, "test.R15B.fprof.log"}]). % or R14B04 > > > respectively > > > > > > > cat test.R14B04.fprof.log |grep suspend|cut -d',' -f3 | awk > > > > '{s+=$1} END > > > > {print s}' > > > > > > 9.8421e+06 > > > > > > > cat test.R15B.fprof.log |grep suspend|cut -d',' -f3 | awk > > > > '{s+=$1} END > > > > {print s}' > > > > > > 8.51855e+06 > > > > > > > cat test.R14B04.fprof.log |grep garbage_collect|cut -d',' -f3 | > > > > awk > > > > '{s+=$1} END {print s}' > > > > > > 1315.33 > > > > > > > cat test.R15B.fprof.log |grep garbage_collect|cut -d',' -f3 | awk > > > > '{s+=$1} > > > > END {print s}' > > > > > > 794.956 > > > > > > On Friday 09 December 2011 12:09:56 Kenneth Lundin wrote: > > > > On 12/5/11, Nico Kruber wrote: > > > > > Is there any news on this? > > > > > > > > > > The performance drop of the recent R15 checkout compared to > > > > > R14B04 in > > > > > our > > > > > application is now at _30%_ compared to a drop of "only" 15% in > > > > > R15A! > > > > > > > > When working with R15B we are of course performing a number of > > > > bencmarks to keep track of the performance. > > > > > > > > Whe always strive to improve general performance in each release > > > > AND > > > > OUR BENCHMARKS INDICATE THAT R15B IS FASTER IN GENERAL THAN > > > > R14B04. > > > > > > > > It is however always possible that some specific operations or > > > > combinations of operations get slower while others are optimized. > > > > > > > > Regarding the findings for this particular application we think > > > > the > > > > reason for slowdown is a very frequent but non typical usage > > > > pattern > > > > that now becomes visible as a performance bottleneck. > > > > > > > > > > > > / Kenneth, Erlang/OTP Ericsson > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: bisect.log Type: text/x-log Size: 2662 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: otp-find_sched_bug.sh Type: application/x-shellscript Size: 1398 bytes Desc: not available URL: From dan@REDACTED Thu Jun 28 21:40:28 2012 From: dan@REDACTED (Daniel Dormont) Date: Thu, 28 Jun 2012 15:40:28 -0400 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema Message-ID: Here is the scenario that happened to me as best I can tell. I had two nodes in a cluster, let's call them A and B. B became unavailable for a while and got rebooted. When I tried to start it again, things seem to work except that certain tables seem not to exist any more. As far as I can tell, these tables used to be enabled only on B and not A, and are now in some sort of weird hybrid unavailable state. A is still running fine in production even with these tables missing, but I can't seem to get a clean start of my application (Ejabberd) on B. So what I figured I would do would be just start a fresh node on B, start Mnesia, add extra_db_nodes pointing to A and go from there. But the problem is A still thinks these certain tables exist only on B (they are listed as remote on A). Fortunately, Ejabberd is smart enough to create any tables it needs on startup, so I was thinking a clean start on B would do this. So I went into A and ran mnesia:del_table_copy(schema, B). thinking this would make the remote tables sort of go away. But instead it fails with {aborted,{no_exists,vcard_search}} And trying to delete the table directly yields the same result. Is there a way I can force Mnesia on A to completely forget about a set of remote tables (and, for that matter, the node that was supposed to store them) before I bring a new node online? Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Thu Jun 28 22:24:43 2012 From: heinz@REDACTED (Heinz N. Gies) Date: Thu, 28 Jun 2012 22:24:43 +0200 Subject: [erlang-questions] Erlang and Solaris SMF's Message-ID: I am sure someone has asked this before but I googled for the last hour and have found nothing but a discussion that didn't provide a answer. My problem is that I want to run a erlang otp release as a SMF service, now the monitoring does not work properly, what I discovered from the discussion mentioned is that it seems the problem that solaris things the release is still running since the epmd is still up and good even so the application might not. What would be the best solution for this? My first guess was to create a empd service but that kind of gets ugly once you've more then one erlang release running. Regards, Heinz -- Heinz N. Gies heinz@REDACTED http://licenser.net From ok@REDACTED Fri Jun 29 02:33:40 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 29 Jun 2012 12:33:40 +1200 Subject: [erlang-questions] Cryptography bugs Message-ID: <99A7FA71-DC32-4B0B-9941-B1387C248965@cs.otago.ac.nz> We had a thread just recently about people (ab)using Erlang's simple random number generator. If non-obvious but easily exploitable security flaws amuse you, you may enjoy http://gregorkopf.de/slides_berlinsides_2010.pdf which explains two bugs, one in Typo3 and one in Joomla. Me, I found this scary. I thought I was good at debugging, and PHP is similar enough to Perl that I should have been able to read the examples, but while I probably would have been wary enough to avoid making these mistakes in the first place, there's no way I would have been smart enough to see them. How do we write programs so that this kind of mistake is harder to make / easier to find? I don't think the programming language makes _much_ difference except to the degree that it lets you annotate values that are critical to security and track them somehow (like the way SPlint lets you track annotations in C). Perhaps having a list of things to look for in an inspection? From that paper, Typo3. What went wrong? * Shortening a MAC value without proper reasons We have enough bandwidth to submit full hash values ;) * Using a not-typesafe comparison operator * Further: forgetting about null bytes Joomla: * Using a weak PRNG 32 bit seed * No entropy accumulator * Frequently reseeding the PRNG * Using CRC32 for cryptographic purposes From Erlang: * Using anything other than a proper cryptographic PRNG Anyone who knows this stuff want to extend it so we can ask for it to be added to the Erlang documentation? From alavrik@REDACTED Fri Jun 29 07:43:51 2012 From: alavrik@REDACTED (Anton Lavrik) Date: Fri, 29 Jun 2012 00:43:51 -0500 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> Message-ID: On Thu, Jun 28, 2012 at 6:07 AM, Tim Watson wrote: > > On 28 Jun 2012, at 10:56, Benoit Chesneau wrote: > >> On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: >>> No actually this is interesting. I *think* I can have a small script in a >>> get-deps post_hook to fetch the extra dependencies when you get the Erlang >>> ones. This way you download everything at the same time. >>> >>> I'll try it out. >>> > > That will work fine, just don't *declare* them as dependencies, otherwise rebar will puke when it doesn't find an application resource file. There is a patch in the rebar pull request queue which adds support for non-otp (raw, as the author puts it) dependencies, which just get downloaded but not checked for OTP compliance. If you like this feature, please go vote for it as @dizzyd and @tuncer are deliberating at the moment. I'm going to up-vote it (after playing devil's advocate for a while) as I'd like to use deps for both things without messing around with hooks. It would be great if you up-voted it. Otherwise, your comments to the pull request could be interpreted as if you were uncertain about it. In fact, @tuncer told me that he didn't want to make that decision leaving it to @dizzyd. One of the reasons was because @dizzyd was going to do some significant internals rework and @tuncer wasn't sure how this change would fit in. Unfortunately, @dyzzyd is not being responsive. Does anybody know who else might be able to make the decision of accepting and merging this change? From watson.timothy@REDACTED Fri Jun 29 10:27:19 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 29 Jun 2012 09:27:19 +0100 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> Message-ID: <3DEDD10A-4373-42E8-8239-7E413C90AEB8@gmail.com> On 29 Jun 2012, at 06:43, Anton Lavrik wrote: > On Thu, Jun 28, 2012 at 6:07 AM, Tim Watson wrote: >> >> On 28 Jun 2012, at 10:56, Benoit Chesneau wrote: >> >>> On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: >>>> No actually this is interesting. I *think* I can have a small script in a >>>> get-deps post_hook to fetch the extra dependencies when you get the Erlang >>>> ones. This way you download everything at the same time. >>>> >>>> I'll try it out. >>>> >> >> That will work fine, just don't *declare* them as dependencies, otherwise rebar will puke when it doesn't find an application resource file. There is a patch in the rebar pull request queue which adds support for non-otp (raw, as the author puts it) dependencies, which just get downloaded but not checked for OTP compliance. If you like this feature, please go vote for it as @dizzyd and @tuncer are deliberating at the moment. I'm going to up-vote it (after playing devil's advocate for a while) as I'd like to use deps for both things without messing around with hooks. > > It would be great if you up-voted it. Otherwise, your comments to the > pull request could be interpreted as if you were uncertain about it. > > In fact, @tuncer told me that he didn't want to make that decision > leaving it to @dizzyd. One of the reasons was because @dizzyd was > going to do some significant internals rework and @tuncer wasn't sure > how this change would fit in. > > Unfortunately, @dyzzyd is not being responsive. Does anybody know who > else might be able to make the decision of accepting and merging this > change? You should be aware that some pull requests can take a long time to get through. I have some that are *really* old but only because I haven't closed them having agreed with the rebar developers that they won't merge them but pending some alternative resolution. I'm going to tidy those up this weekend. I *will* go and up-vote this, but anything to do with deps is tricky to decide on and don't be surprised if it takes a while. Anyway I wouldn't worry too much about this at all! Firstly you can of course use your own rebar fork in your own projects, but more significantly (IMHO) there is a soon to be merged branch ta-state which, when it does get merged, will allow you to completely customise the way anything (including deps) works from a locally (or globally) configured plugin and your [raw] feature can easily be supported for every rebar project that wants it without actually changing rebar at all. I package and distribute a number of global/general-purpose plugins which are used across 80% of my rebar based projects and this process is already very slick and easy and when ta-state gets merged, it will be simple to make features that even change core functionality such as adding support for non-otp deps and bundle this as a plugin without changing even the way you configure your projects or invoke rebar or anything. It's going to be awesome (can you tell I'm excited about it!?) because instead of sitting in a pull request queue debating whether or not what you want to do makes sense, you can just do it and distribute it via github and use it wherever you want to. And if it *does* make sense probably a bunch of other people will start using it and you'll be managing your own pull request queue instead of sitting in someone else's! Cheers, Tim From mahesh@REDACTED Fri Jun 29 16:06:53 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Fri, 29 Jun 2012 10:06:53 -0400 Subject: [erlang-questions] Why are records not 'export'-able? Message-ID: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> If I have a record I use in just one module, I declare it in that module. If I have a record I use across a number of modules, I declare it in an included file. If I have a record that I use across a number of applications, I end up declaring it once in each application. The simple-minded me says "Hmf. If I could just have a '-export_record' declaration, then I could do some fun things like '-record(Name, Application:Record)' The more realistic me says "There is probably some reason why you can't do this. Probably has to do with compilers. It usually does". Anyone? Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Blog | Twitter | Google+ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Fri Jun 29 16:12:46 2012 From: dan@REDACTED (Daniel Dormont) Date: Fri, 29 Jun 2012 10:12:46 -0400 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema In-Reply-To: References: Message-ID: Just to give a little more information: I tried creating a new node from scratch to join my cluster. I said erl -name new_node@REDACTED -s mnesia -mnesia dir '"/var/lib/ejabberd"' -mnesia extra_db_nodes "['ejabberd@REDACTED']" and that seemed to load successfully. I then did the usual mnesia:add_table_copy(schema, node(), disc_copies). and got the error: {aborted,{already_exists,schema,'new_node@REDACTED'}} What would cause that? Does this mean the schema on A is permanently hosed in some way and I need to wipe it and recreate? dan On Thu, Jun 28, 2012 at 3:40 PM, Daniel Dormont wrote: > Here is the scenario that happened to me as best I can tell. I had two > nodes in a cluster, let's call them A and B. B became unavailable for a > while and got rebooted. When I tried to start it again, things seem to work > except that certain tables seem not to exist any more. As far as I can > tell, these tables used to be enabled only on B and not A, and are now in > some sort of weird hybrid unavailable state. > > A is still running fine in production even with these tables missing, but > I can't seem to get a clean start of my application (Ejabberd) on B. So > what I figured I would do would be just start a fresh node on B, start > Mnesia, add extra_db_nodes pointing to A and go from there. But the problem > is A still thinks these certain tables exist only on B (they are listed as > remote on A). Fortunately, Ejabberd is smart enough to create any tables it > needs on startup, so I was thinking a clean start on B would do this. So I > went into A and ran > > mnesia:del_table_copy(schema, B). > > thinking this would make the remote tables sort of go away. But instead it > fails with > > {aborted,{no_exists,vcard_search}} > > And trying to delete the table directly yields the same result. > > Is there a way I can force Mnesia on A to completely forget about a set of > remote tables (and, for that matter, the node that was supposed to store > them) before I bring a new node online? > > Dan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Jun 29 16:26:20 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 29 Jun 2012 10:26:20 -0400 Subject: [erlang-questions] [ANN] Spawnfest 2012 Message-ID: <4FEDBB0C.4020704@ferd.ca> Hello there, Erlang Mailing List. I'm writing you to announce and/or remind you that we're bringing back Spawnfest (http://spawnfest.com) again this year. Spawnfest is a 48 hours programming contest for Erlang, much like Rails Rumble or Node Knockout. It is scheduled for July 7-8, and the rules are available on the contest's website. We're excited to bring the contest back, as we feel it's a great opportunity to show the world what Erlang/OTP and its users are capable of. The contest is in no way limited to web applications or any specific domain; we embrace the diversity of entries, as shows last year's winners (http://2011.spawnfest.com/winners/). If you participate (and you should!), you'll have your applications judged by our high profile judges: Eric B. Merrit ( founding member of the Erlware, co-author of "Erlang and OTP in Action"), Joe Armstrong ( co-inventor of Erlang, no introduction required), Lo?c Hoguin (Founder of Nine Nines, writer of Cowboy), and Scott Lystig Fritchie (long-time Erlang user, senior software engineer at Basho, worked on D-Trace for Erlang, and so on). You can register your team at http://spawnfest.com/ Feel free to post any question on the mailing list, we will be happy to address those. If you're interested in becoming a sponsor, please email me directly at this address; it could be a tremendous opportunity to expose your business. While on the topic, I'd like to thank our current sponsors for the contest, many of them who were there last year too: Visit Basho Visit GreenQloud Visit Github Visit Cloudant Visit Inaka Visit Erlang Solutions Visit O'Reilly Visit Heroku vmware RabbitMQ Nine Nines Visit Spawngrid Visit Kodiak -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5202 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 14127 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 7264 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 4572 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 7605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 20812 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 9562 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 8166 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 9370 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 12992 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 4588 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 8133 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 10135 bytes Desc: not available URL: From f@REDACTED Fri Jun 29 16:27:23 2012 From: f@REDACTED (Francesco Mazzoli) Date: Fri, 29 Jun 2012 15:27:23 +0100 Subject: [erlang-questions] Removing mnesia cluster node while mnesia is offline Message-ID: <878vf6chx0.wl%f@mazzo.li> Hi list, I have the following situation: there is an offline mnesia node that is in a cluster. While this node was offline, some nodes left the cluster it is in. The problem is that the offline node still thinks it is clustered with the offline (or worse, not clustered anymore nodes). Starting mnesia on that node will either "pollute" the other nodes by propagating the schema with the not-in-the-cluster-anymore nodes, or worse hangs if the nodes that the node thinks are still clustered are online. So my question is: is it possible to remove some node from the cluster schema *while the node is offline*? I know how to do that while mnesia is online - `mnesia:del_table_copy(schema, Node)' - but I'm lost when I have to do that while mnesia is offline. Changing the `extra_db_nodes' won't help because that's only use as an addition to the schema. Any hints? Thanks, Francesco. From lenartlad@REDACTED Fri Jun 29 16:27:56 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Fri, 29 Jun 2012 16:27:56 +0200 Subject: [erlang-questions] Why are records not 'export'-able? In-Reply-To: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> References: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> Message-ID: <4FEDBB6C.8010902@volny.cz> Hello. You can create a library application with your include file (and perhaps other common modules) and then use -include_lib(...) from all other applications. HTH, Ladislav Lenart On 29.6.2012 16:06, Mahesh Paolini-Subramanya wrote: > If I have a record I use in just one module, I declare it in that module. > If I have a record I use across a number of modules, I declare it in an > included file. > If I have a record that I use across a number of applications, I end up > declaring it once in each application. > > The simple-minded me says "Hmf. If I could just have a '-export_record' > declaration, then I could do some fun things like '-record(Name, > Application:Record)' > > The more realistic me says "There is probably some reason why you can't do this. > Probably has to do with compilers. It usually does". > > Anyone? > > */ > */Mahesh Paolini-Subramanya/* > > That Tall Bald Indian Guy... > Blog | Twitter > | Google+ > > /* > __ > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rpettit@REDACTED Fri Jun 29 16:41:02 2012 From: rpettit@REDACTED (Rick Pettit) Date: Fri, 29 Jun 2012 09:41:02 -0500 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema In-Reply-To: References: Message-ID: <8E477290-4215-4223-9A5E-49928B37A08E@vailsys.com> On Jun 28, 2012, at 2:40 PM, Daniel Dormont wrote: > Here is the scenario that happened to me as best I can tell. I had two nodes in a cluster, let's call them A and B. B became unavailable for a while and got rebooted. When I tried to start it again, things seem to work except that certain tables seem not to exist any more. As far as I can tell, these tables used to be enabled only on B and not A, and are now in some sort of weird hybrid unavailable state. > > A is still running fine in production even with these tables missing, but I can't seem to get a clean start of my application (Ejabberd) on B. So what I figured I would do would be just start a fresh node on B, start Mnesia, add extra_db_nodes pointing to A and go from there. But the problem is A still thinks these certain tables exist only on B (they are listed as remote on A). Fortunately, Ejabberd is smart enough to create any tables it needs on startup, so I was thinking a clean start on B would do this. So I went into A and ran > > mnesia:del_table_copy(schema, B). > > thinking this would make the remote tables sort of go away. But instead it fails with > > {aborted,{no_exists,vcard_search}} > > And trying to delete the table directly yields the same result. > > Is there a way I can force Mnesia on A to completely forget about a set of remote tables (and, for that matter, the node that was supposed to store them) before I bring a new node online? You might want to take a look at the documentation for mnesia:set_master_nodes/1,2 and maybe mnesia:force_load_table/1. Just make sure you understand exactly how these work before using either in production--if used incorrectly, you could leave the database in an inconsistent state. Hope that helps, -Rick From robert.virding@REDACTED Fri Jun 29 17:27:10 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 29 Jun 2012 16:27:10 +0100 (BST) Subject: [erlang-questions] Why are records not 'export'-able? In-Reply-To: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> Message-ID: <495b5f26-3eee-4263-b9dd-d1d6456dc111@knuth> The main reason is that records are purely compile-time and record definitions only exist in the compiler. At runtime a record is just a normal tuple with nothing in it to show that it came from a record or what the record definition is. So there are no problems accessing record as tuples and tuples as records, though in most cases it is a stupid thing to do. So the problem is that you need to be able to access the record definition whenever a module using it is compiled. One solution is to put the record definition .hrl file in an applications include directory. It is then possible to access that file from the compiler using: -include_lib("app/include/recdef.hrl"). The compiler will extend the app name with version numbers when necessary. Robert ----- Original Message ----- > If I have a record I use in just one module, I declare it in that > module. > If I have a record I use across a number of modules, I declare it in > an included file. > If I have a record that I use across a number of applications, I end > up declaring it once in each application. > The simple-minded me says "Hmf. If I could just have a > '-export_record' declaration, then I could do some fun things like > '-record(Name, Application:Record)' > The more realistic me says "There is probably some reason why you > can't do this. Probably has to do with compilers. It usually does". > Anyone? > Mahesh Paolini-Subramanya > That Tall Bald Indian Guy... > Blog | Twitter | Google+ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Jun 29 17:27:21 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 29 Jun 2012 17:27:21 +0200 Subject: [erlang-questions] Why are records not 'export'-able? In-Reply-To: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> References: <8367670F-7092-4618-ADF7-EF0843BF65E8@dieswaytoofast.com> Message-ID: 2012/6/29 Mahesh Paolini-Subramanya : > If I have a record I use in just one module, I declare it in that module. > If I have a record I use across a number of modules, I declare it in an > included?file. > If I have a record that I use across a number of applications, I end up > declaring it once in each application. > > The simple-minded me says "Hmf. If I could just have a '-export_record' > declaration, then I could do some fun things like '-record(Name, > Application:Record)' > > The more realistic me says "There is probably some reason why you can't do > this. Probably has to do with compilers. It usually does". As the documentation says, "record is not a true data type. Instead record expressions are translated to tuple expressions during compilation.", so there's nothing to export. However, nothing stops you from including a header from an other application. From dan@REDACTED Fri Jun 29 17:46:01 2012 From: dan@REDACTED (Daniel Dormont) Date: Fri, 29 Jun 2012 11:46:01 -0400 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema In-Reply-To: <8E477290-4215-4223-9A5E-49928B37A08E@vailsys.com> References: <8E477290-4215-4223-9A5E-49928B37A08E@vailsys.com> Message-ID: Well, it appears that I may have done just that. How do I determine if the database is in an inconsistent state and what can be done about that? Again, I am ok with completely deleting certain tables and nodes from the schema, "brutally" if need be. Is there any way of doing such a thing short of wiping the entire schema and starting from scratch? On Fri, Jun 29, 2012 at 10:41 AM, Rick Pettit wrote: > > On Jun 28, 2012, at 2:40 PM, Daniel Dormont wrote: > > > Here is the scenario that happened to me as best I can tell. I had two > nodes in a cluster, let's call them A and B. B became unavailable for a > while and got rebooted. When I tried to start it again, things seem to work > except that certain tables seem not to exist any more. As far as I can > tell, these tables used to be enabled only on B and not A, and are now in > some sort of weird hybrid unavailable state. > > > > A is still running fine in production even with these tables missing, > but I can't seem to get a clean start of my application (Ejabberd) on B. So > what I figured I would do would be just start a fresh node on B, start > Mnesia, add extra_db_nodes pointing to A and go from there. But the problem > is A still thinks these certain tables exist only on B (they are listed as > remote on A). Fortunately, Ejabberd is smart enough to create any tables it > needs on startup, so I was thinking a clean start on B would do this. So I > went into A and ran > > > > mnesia:del_table_copy(schema, B). > > > > thinking this would make the remote tables sort of go away. But instead > it fails with > > > > {aborted,{no_exists,vcard_search}} > > > > And trying to delete the table directly yields the same result. > > > > Is there a way I can force Mnesia on A to completely forget about a set > of remote tables (and, for that matter, the node that was supposed to store > them) before I bring a new node online? > > You might want to take a look at the documentation for > mnesia:set_master_nodes/1,2 and maybe mnesia:force_load_table/1. > > Just make sure you understand exactly how these work before using either > in production--if used incorrectly, you could leave the database in an > inconsistent state. > > Hope that helps, > > -Rick > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Fri Jun 29 18:03:39 2012 From: rpettit@REDACTED (Rick Pettit) Date: Fri, 29 Jun 2012 11:03:39 -0500 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema In-Reply-To: References: <8E477290-4215-4223-9A5E-49928B37A08E@vailsys.com> Message-ID: On Jun 29, 2012, at 10:46 AM, Daniel Dormont wrote: > Well, it appears that I may have done just that. How do I determine if the database is in an inconsistent state and what can be done about that? Again, I am ok with completely deleting certain tables and nodes from the schema, "brutally" if need be. Is there any way of doing such a thing short of wiping the entire schema and starting from scratch? Daniel, I think you might be in a situation similar (though perhaps not exactly like) one which I believe was solved on the Trap Exit forums: > >>> On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > >>> > >>>> I seem to be stuck in a state where I can't create a table because it > >>>> exists, but I can't delete the table because it doesn't exist! You can view the thread @ http://forum.trapexit.org/viewtopic.php?p=62092&sid=7a78bf70100c90aadea4267c921e662d Take a quick look and see if that sounds like the problem you are having. If so, I would pay particular attention to comments from Ulf W. -Rick > On Fri, Jun 29, 2012 at 10:41 AM, Rick Pettit wrote: > > On Jun 28, 2012, at 2:40 PM, Daniel Dormont wrote: > > > Here is the scenario that happened to me as best I can tell. I had two nodes in a cluster, let's call them A and B. B became unavailable for a while and got rebooted. When I tried to start it again, things seem to work except that certain tables seem not to exist any more. As far as I can tell, these tables used to be enabled only on B and not A, and are now in some sort of weird hybrid unavailable state. > > > > A is still running fine in production even with these tables missing, but I can't seem to get a clean start of my application (Ejabberd) on B. So what I figured I would do would be just start a fresh node on B, start Mnesia, add extra_db_nodes pointing to A and go from there. But the problem is A still thinks these certain tables exist only on B (they are listed as remote on A). Fortunately, Ejabberd is smart enough to create any tables it needs on startup, so I was thinking a clean start on B would do this. So I went into A and ran > > > > mnesia:del_table_copy(schema, B). > > > > thinking this would make the remote tables sort of go away. But instead it fails with > > > > {aborted,{no_exists,vcard_search}} > > > > And trying to delete the table directly yields the same result. > > > > Is there a way I can force Mnesia on A to completely forget about a set of remote tables (and, for that matter, the node that was supposed to store them) before I bring a new node online? > > You might want to take a look at the documentation for mnesia:set_master_nodes/1,2 and maybe mnesia:force_load_table/1. > > Just make sure you understand exactly how these work before using either in production--if used incorrectly, you could leave the database in an inconsistent state. > > Hope that helps, > > -Rick > > From mahesh@REDACTED Fri Jun 29 18:08:56 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Fri, 29 Jun 2012 12:08:56 -0400 Subject: [erlang-questions] Why are records not 'export'-able? In-Reply-To: <495b5f26-3eee-4263-b9dd-d1d6456dc111@knuth> References: <495b5f26-3eee-4263-b9dd-d1d6456dc111@knuth> Message-ID: Sadly, I *knew* that records don't survive compilation - just didn't put the pieces together. -include_lib is pretty much the most logical option here. The only question remaining is "How much refactoring do I want to do?" :-) Thanks all... Cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Blog | Twitter | Google+ On Jun 29, 2012, at 11:27 AM, Robert Virding wrote: > The main reason is that records are purely compile-time and record definitions only exist in the compiler. At runtime a record is just a normal tuple with nothing in it to show that it came from a record or what the record definition is. So there are no problems accessing record as tuples and tuples as records, though in most cases it is a stupid thing to do. > > So the problem is that you need to be able to access the record definition whenever a module using it is compiled. One solution is to put the record definition .hrl file in an applications include directory. It is then possible to access that file from the compiler using: > > -include_lib("app/include/recdef.hrl"). > > The compiler will extend the app name with version numbers when necessary. > > Robert > > If I have a record I use in just one module, I declare it in that module. > If I have a record I use across a number of modules, I declare it in an included file. > If I have a record that I use across a number of applications, I end up declaring it once in each application. > > The simple-minded me says "Hmf. If I could just have a '-export_record' declaration, then I could do some fun things like '-record(Name, Application:Record)' > > The more realistic me says "There is probably some reason why you can't do this. Probably has to do with compilers. It usually does". > > Anyone? > > Mahesh Paolini-Subramanya > That Tall Bald Indian Guy... > Blog | Twitter | Google+ > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Fri Jun 29 18:31:24 2012 From: dan@REDACTED (Daniel Dormont) Date: Fri, 29 Jun 2012 12:31:24 -0400 Subject: [erlang-questions] mnesia recover from netsplit, can't delete node from schema In-Reply-To: References: <8E477290-4215-4223-9A5E-49928B37A08E@vailsys.com> Message-ID: I tried following Ulf's suggestion of starting a fresh node with the same node name (ejabberd@REDACTED) as the node that, according to mnesia:table_info(tab, all) is supposed to be holding a copy of this table, and then adding it to the cluster using extra_db_nodes. At that moment, the new node takes about 15 seconds to complete the mnesia:change_config(extra_db_nodes, ['ejabberd@REDACTED']). operation, but it does seem to succeed. At that point on the new node it shows every table, including the ones I'm interested in, as remote. At that point, on the new node calling mnesia:del_table_copy(vcard, node()). returns {aborted,{badarg,vcard,unknown}} and calling mnesia:add_table_copy(vcard, node(), ram_copies). returns aborted,{system_limit,vcard, {'ejabberd@REDACTED',none_active}}} I'm not able to find any documentation on what 'none_active' means. Any idea? dan On Fri, Jun 29, 2012 at 12:03 PM, Rick Pettit wrote: > > On Jun 29, 2012, at 10:46 AM, Daniel Dormont wrote: > > > Well, it appears that I may have done just that. How do I determine if > the database is in an inconsistent state and what can be done about that? > Again, I am ok with completely deleting certain tables and nodes from the > schema, "brutally" if need be. Is there any way of doing such a thing short > of wiping the entire schema and starting from scratch? > > Daniel, > > I think you might be in a situation similar (though perhaps not exactly > like) one which I believe was solved on the Trap Exit forums: > > > >>> On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > > >>> > > >>>> I seem to be stuck in a state where I can't create a table > because it > > >>>> exists, but I can't delete the table because it doesn't exist! > > You can view the thread @ > http://forum.trapexit.org/viewtopic.php?p=62092&sid=7a78bf70100c90aadea4267c921e662d > > Take a quick look and see if that sounds like the problem you are having. > > If so, I would pay particular attention to comments from Ulf W. > > -Rick > > > > On Fri, Jun 29, 2012 at 10:41 AM, Rick Pettit > wrote: > > > > On Jun 28, 2012, at 2:40 PM, Daniel Dormont wrote: > > > > > Here is the scenario that happened to me as best I can tell. I had two > nodes in a cluster, let's call them A and B. B became unavailable for a > while and got rebooted. When I tried to start it again, things seem to work > except that certain tables seem not to exist any more. As far as I can > tell, these tables used to be enabled only on B and not A, and are now in > some sort of weird hybrid unavailable state. > > > > > > A is still running fine in production even with these tables missing, > but I can't seem to get a clean start of my application (Ejabberd) on B. So > what I figured I would do would be just start a fresh node on B, start > Mnesia, add extra_db_nodes pointing to A and go from there. But the problem > is A still thinks these certain tables exist only on B (they are listed as > remote on A). Fortunately, Ejabberd is smart enough to create any tables it > needs on startup, so I was thinking a clean start on B would do this. So I > went into A and ran > > > > > > mnesia:del_table_copy(schema, B). > > > > > > thinking this would make the remote tables sort of go away. But > instead it fails with > > > > > > {aborted,{no_exists,vcard_search}} > > > > > > And trying to delete the table directly yields the same result. > > > > > > Is there a way I can force Mnesia on A to completely forget about a > set of remote tables (and, for that matter, the node that was supposed to > store them) before I bring a new node online? > > > > You might want to take a look at the documentation for > mnesia:set_master_nodes/1,2 and maybe mnesia:force_load_table/1. > > > > Just make sure you understand exactly how these work before using either > in production--if used incorrectly, you could leave the database in an > inconsistent state. > > > > Hope that helps, > > > > -Rick > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Fri Jun 29 23:54:56 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 29 Jun 2012 23:54:56 +0200 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: <3DEDD10A-4373-42E8-8239-7E413C90AEB8@gmail.com> References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> <3DEDD10A-4373-42E8-8239-7E413C90AEB8@gmail.com> Message-ID: On Fri, Jun 29, 2012 at 10:27 AM, Tim Watson wrote: > On 29 Jun 2012, at 06:43, Anton Lavrik wrote: > > > On Thu, Jun 28, 2012 at 6:07 AM, Tim Watson wrote: > > > > > > On 28 Jun 2012, at 10:56, Benoit Chesneau wrote: > > > > > > > On Thu, Jun 28, 2012 at 11:49 AM, Lo?c Hoguin wrote: > > > > No actually this is interesting. I *think* I can have a small > > > > script in a get-deps post_hook to fetch the extra dependencies > > > > when you get the Erlang ones. This way you download everything > > > > at the same time. > > > > > > > > I'll try it out. > > > > > > That will work fine, just don't *declare* them as dependencies, > > > otherwise rebar will puke when it doesn't find an application > > > resource file. There is a patch in the rebar pull request queue > > > which adds support for non-otp (raw, as the author puts it) > > > dependencies, which just get downloaded but not checked for OTP > > > compliance. If you like this feature, please go vote for it as > > > @dizzyd and @tuncer are deliberating at the moment. I'm going to > > > up-vote it (after playing devil's advocate for a while) as I'd > > > like to use deps for both things without messing around with > > > hooks. > > > > It would be great if you up-voted it. Otherwise, your comments to the > > pull request could be interpreted as if you were uncertain about it. > > > > In fact, @tuncer told me that he didn't want to make that decision > > leaving it to @dizzyd. One of the reasons was because @dizzyd was > > going to do some significant internals rework and @tuncer wasn't sure > > how this change would fit in. > > > > Unfortunately, @dyzzyd is not being responsive. Does anybody know who > > else might be able to make the decision of accepting and merging this > > change? > > You should be aware that some pull requests can take a long time to > get through. I have some that are *really* old but only because I > haven't closed them having agreed with the rebar developers that > they won't merge them but pending some alternative resolution. I'm > going to tidy those up this weekend. I *will* go and up-vote this, > but anything to do with deps is tricky to decide on and don't be > surprised if it takes a while. Commented in the pull request after realizing a couple more issues. Let's continue the patch discussion over there. > Anyway I wouldn't worry too much about this at all! Firstly you can > of course use your own rebar fork in your own projects, but more > significantly (IMHO) there is a soon to be merged branch ta-state > which, when it does get merged, will allow you to completely > customise the way anything (including deps) works from a locally (or > globally) configured plugin and your [raw] feature can easily be > supported for every rebar project that wants it without actually > changing rebar at all. I package and distribute a number of > global/general-purpose plugins which are used across 80% of my rebar > based projects and this process is already very slick and easy and > when ta-state gets merged, it will be simple to make features that > even change core functionality such as adding support for non-otp > deps and bundle this as a plugin without changing even the way you > configure your projects or invoke rebar or anything. It's going to > be awesome (can you tell I'm excited about it!?) because instead of > sitting in a pull request queue debating whether or not what you > want to do makes sense, you can just do it and distribute it via > github and use it wherever you want to. And if it *does* make sense > probably a bunch of other people will start using it and you'll be > managing your own pull request queue instead of sitting in someone > else's! Tim, let me say I'm glad that you push the limits of the existing plugin system :). Some patches are not universal enough or an easy fit to be applied to rebar master as is. Therefore it's usually a good idea to let such patches exist as plugins first. If it proves to be popular and the implementation has been tested/refined, it will be much easier to consider a merge. Also, compiler plugins for alternative languages are best distributed with their compiler. Finally, having a tried and tested external plugin gives us great insight to consider core changes making proper solutions possible. Interested rebar users might want to consider helping with ta-state by giving it a spin and reporting all issues. For details see http://lists.basho.com/pipermail/rebar_lists.basho.com/2012-June/001623.html There's a known issue with list-deps when run on https://github.com/hyperthunk/eunit_cover_example. Should be fixed this weekend. From watson.timothy@REDACTED Sat Jun 30 01:21:17 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 30 Jun 2012 00:21:17 +0100 Subject: [erlang-questions] Non-Erlang dependencies In-Reply-To: References: <4FEC0F38.7020700@ninenines.eu> <4FEC28AE.4010105@ninenines.eu> <1DD85A1D-6191-4987-A67B-3F2152EB5C90@gmail.com> <3DEDD10A-4373-42E8-8239-7E413C90AEB8@gmail.com> Message-ID: <5431B355-BDD6-4ECC-807A-157B768DF675@gmail.com> On 29 Jun 2012, at 22:54, Tuncer Ayaz wrote: > > Tim, let me say I'm glad that you push the limits of the existing > plugin system :). > Thanks lol! :) From watson.timothy@REDACTED Sat Jun 30 13:31:04 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 30 Jun 2012 12:31:04 +0100 Subject: [erlang-questions] Leex scanners and default token matching Message-ID: <3B8537A4-E310-4963-A1DF-4002895D3A42@gmail.com> Hi all, I've got a simple Leex scanner, which appears to go into a non-terminating state for certain inputs, consuming 100% CPU and quickly eating up all available memory. I found this *very* surprising - should the generated scanner really be able to get itself into this state? Is there some way for me to provide a default rule that will execute if no other regex matches, so I can return {error, Reason} for this??? Below is a copy of the xrl file - running scanner:string("aa = !") will cause it to hang. Am I missing some obvious way of preventing this? Definitions. COMMA = [,] PARENS = [\(\)] L = [A-Za-z_\$] D = [0-9-] F = (\+|-)?[0-9]+\.[0-9]+((E|e)(\+|-)?[0-9]+)? HEX = 0x[0-9]+ WS = ([\000-\s]|%.*) S = ({COMMA}|{PARENS}) CMP = (=|>|>=|<|<=|<>) AOP = (\\+|-|\\*|/) Rules. LIKE : {token, {op_like, TokenLine, like}}. IN : {token, {op_in, TokenLine, in}}. AND : {token, {op_and, TokenLine, conjunction}}. OR : {token, {op_or, TokenLine, disjunction}}. NOT : {token, {op_not, TokenLine, negation}}. IS{WS}NULL : {token, {op_null, TokenLine, is_null}}. IS{WS}NOT{WS}NULL : {token, {op_null, TokenLine, not_null}}. BETWEEN : {token, {op_between, TokenLine, range}}. ESCAPE : {token, {escape, TokenLine, escape}}. {CMP} : {token, {op_cmp, TokenLine, atomize(TokenChars)}}. {AOP} : {token, {op_arith, TokenLine, atomize(TokenChars)}}. {L}({L}|{D})* : {token, {ident, TokenLine, TokenChars}}. '([^''])*' : {token, {lit_string, TokenLine, strip(TokenChars,TokenLen)}}. {S} : {token, {list_to_atom(TokenChars),TokenLine}}. {D}+ : {token, {lit_int, TokenLine, list_to_integer(TokenChars)}}. {F} : {token, {lit_flt, TokenLine, list_to_float(TokenChars)}}. {HEX} : {token, {lit_hex, TokenLine, hex_to_int(TokenChars)}}. {WS}+ : skip_token. Erlang code. strip(TokenChars,TokenLen) -> lists:sublist(TokenChars, 2, TokenLen - 2). hex_to_int([_,_|R]) -> {ok,[Int],[]} = io_lib:fread("~16u", R), Int. atomize(TokenChars) -> list_to_atom(TokenChars). From watson.timothy@REDACTED Sat Jun 30 13:47:00 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 30 Jun 2012 12:47:00 +0100 Subject: [erlang-questions] Leex scanners and default token matching In-Reply-To: <3B8537A4-E310-4963-A1DF-4002895D3A42@gmail.com> References: <3B8537A4-E310-4963-A1DF-4002895D3A42@gmail.com> Message-ID: <0B878C56-EC7A-4DC2-B02A-A4D5866BCF63@gmail.com> Well the 'solution' is probably as simple as filtering them out - not quite as onerous as I'd imagined once I realised that any character that isn't single quoted falls into a pretty small range. An extra macro: ILLEGAL = ([^\s\w=><*/]) and then an extra Rule: {ILLEGAL}+ : {error, "Illegal Character(s): " ++ TokenChars}. Still, it should be documented that Leex *can* produce a scanner that is non-terminating. Even though there's nothing to say that it shouldn't, whilst the Yecc documentation explains something about the implementation that helps you understand how it works (LALR-1) there is nothing in the leex docs to suggest that I should *beware* of this. Luckily I decided to randomly generate input strings using PropEr and quickly came across the issue in my implementation. What do other think about this? Should there be a caution in the documentation explaining this potential situation? On 30 Jun 2012, at 12:31, Tim Watson wrote: > Hi all, > > I've got a simple Leex scanner, which appears to go into a non-terminating state for certain inputs, consuming 100% CPU and quickly eating up all available memory. I found this *very* surprising - should the generated scanner really be able to get itself into this state? Is there some way for me to provide a default rule that will execute if no other regex matches, so I can return {error, Reason} for this??? > > Below is a copy of the xrl file - running scanner:string("aa = !") will cause it to hang. Am I missing some obvious way of preventing this? > > Definitions. > COMMA = [,] > PARENS = [\(\)] > L = [A-Za-z_\$] > D = [0-9-] > F = (\+|-)?[0-9]+\.[0-9]+((E|e)(\+|-)?[0-9]+)? > HEX = 0x[0-9]+ > WS = ([\000-\s]|%.*) > S = ({COMMA}|{PARENS}) > CMP = (=|>|>=|<|<=|<>) > AOP = (\\+|-|\\*|/) > > Rules. > > LIKE : {token, {op_like, TokenLine, like}}. > IN : {token, {op_in, TokenLine, in}}. > AND : {token, {op_and, TokenLine, conjunction}}. > OR : {token, {op_or, TokenLine, disjunction}}. > NOT : {token, {op_not, TokenLine, negation}}. > IS{WS}NULL : {token, {op_null, TokenLine, is_null}}. > IS{WS}NOT{WS}NULL : {token, {op_null, TokenLine, not_null}}. > BETWEEN : {token, {op_between, TokenLine, range}}. > ESCAPE : {token, {escape, TokenLine, escape}}. > {CMP} : {token, {op_cmp, TokenLine, atomize(TokenChars)}}. > {AOP} : {token, {op_arith, TokenLine, atomize(TokenChars)}}. > {L}({L}|{D})* : {token, {ident, TokenLine, TokenChars}}. > '([^''])*' : {token, {lit_string, TokenLine, strip(TokenChars,TokenLen)}}. > {S} : {token, {list_to_atom(TokenChars),TokenLine}}. > {D}+ : {token, {lit_int, TokenLine, list_to_integer(TokenChars)}}. > {F} : {token, {lit_flt, TokenLine, list_to_float(TokenChars)}}. > {HEX} : {token, {lit_hex, TokenLine, hex_to_int(TokenChars)}}. > {WS}+ : skip_token. > > Erlang code. > > strip(TokenChars,TokenLen) -> > lists:sublist(TokenChars, 2, TokenLen - 2). > > hex_to_int([_,_|R]) -> > {ok,[Int],[]} = io_lib:fread("~16u", R), > Int. > > atomize(TokenChars) -> > list_to_atom(TokenChars). > From romanshestakov@REDACTED Sat Jun 23 04:23:56 2012 From: romanshestakov@REDACTED (Roman Shestakov) Date: Sat, 23 Jun 2012 02:23:56 -0000 Subject: [erlang-questions] (no subject) Message-ID: <1340418235.59023.YahooMailNeo@web171302.mail.ir2.yahoo.com> http://emissionszero.com/videos/classes/com/adobe/favideo/managers/_notes/google.html?ehrv=ezs.jhj&vcbn=cym.ww&ghb=nyac -------------- next part -------------- An HTML attachment was scrubbed... URL:
, , and > so on. ?When I'm told to use instead of