From ericbmerritt@REDACTED Wed May 1 00:09:53 2013 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 30 Apr 2013 15:09:53 -0700 Subject: [erlang-questions] Gittip In-Reply-To: References: Message-ID: I just saw this for some reason. I agree. It seems pretty intensive though. I wonder if I can just make a 'fund' with proportions and just don't a certain amount to my 'fund' each week. On Apr 12, 2013, at 9:49 AM, Tristan Sloughter wrote: > I just learned about https://www.gittip.com/ > > It would be awesome if as many of us as possible started funding, even just a $1 a week, to help our favorite Erlang projects. > > I know Loic is on, https://www.gittip.com/essen/ -- so if you use Cowboy, Ranch or Bullet, that would be a good place to start :) > > Would be great if people also gave suggestions of whom people should donate to, even if they themselves can't afford it or if it is their project and they know they could do more with some donations. > > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayson.barley@REDACTED Wed May 1 00:25:22 2013 From: jayson.barley@REDACTED (Jayson Barley) Date: Tue, 30 Apr 2013 15:25:22 -0700 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: <52ADF12A-AF3E-43E6-BCC5-B02CB891F0C4@cs.otago.ac.nz> Message-ID: This may be a really dumb question, but is there a reason you can't keep the numbers that have been removed in a separate file? On Tue, Apr 30, 2013 at 4:40 AM, Salvador Tamarit wrote: > 2013/4/30 Richard A. O'Keefe : > > > > On 30/04/2013, at 1:23 PM, Salvador Tamarit wrote: > > > >> Suppose that we have a huge binary file (8GB or so) containing all the > >> integers from 0 to a given (very big) number. The numbers are unsorted > >> and some of them (70 numbers for instance) has been removed. > > > > This sounds like a well known problem with an answer in > > > http://books.google.co.nz/books?id=415loiMd_c0C&lpg=PP1&dq=muthukrishnan+data+stream+algorithms&hl=el&pg=PA1&redir_esc=y > > > > That's great. I was suspicious that it existed, but I didn't know how > to find it :P Thank you very much Richard. > > > with the twist that you are asked to find a specific missing number > > (in value order) rather than all of them. > > > > When you talk about reading the numbers 4 bytes at a time, that tells > > me the numbers are not >that< "huge". > > > > One very simple step would be to set up > > number-present : map (0..6535) -> nat := (all -> 0)_ > > for each number in the input > > number-present(high 16 bits of that number) +:= 1 > > Now you can tell which block of numbers must contain the > > number you are looking for, and you can make a second pass > > through the data counting numbers in that block, and then > > it's a simple scan through the block. > > > It would solve the problem of being able to load the data into memory, > but I still having problems reading the whole file. It takes too much > time. I know my function is very naive, however I'm not sure how could > I improve it. Only reading integers from file, without further > processing, takes a lot of time (minutes) in a fast laptop. > > > read_integers(FDevice) -> > case file:read(FDevice, 4) of > {ok,<<_:2/little-unit:8,Readed:2/little-unit:8>>} -> > read_integers(FDevice); > eof -> > ok > end. > > > > > So if the numbers are 0..N-1, the problem can be solved in > > O(N**1/2) space in two passes, or in O(N**1/3) space in > > three passes, &c. > > > > This is _not_ mathematically optimal, but it's quite easy to > > program, and if you really had this problem, I'd rather have > > the data read twice by a simple program I trusted than once > > by a clever program that was almost certainly wrong. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Wed May 1 01:14:38 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 30 Apr 2013 18:14:38 -0500 Subject: [erlang-questions] basho_bench, was Re: [ANN] Silly benchmarking In-Reply-To: Message of "Tue, 30 Apr 2013 07:35:28 PDT." <517FD6B0.3070505@gmail.com> Message-ID: <87073.1367363678@snookles.snookles.com> Michael Truog wrote: mt> The other option is trying to use basho_bench here mt> https://github.com/basho/basho_bench, if you are testing key/value mt> storage. Actually, you can use it to measure whatever you want, if you write a callback module with a new/1 and run/4 function. Tracking throughput is always good, but also tracking min, mean, median, 95th, 99th, 99_9th, and maximum latency stats plus error rates is An Even BetterThing(tm). Nice R and Gnuplot graphs is also very convenient. For an example graph, see http://www.snookles.com/scotttmp/basho_bench_null_summary.png ... It used the bundled examples/null_test.config config, which tests approximately nothing but basho_bench's internal overhead. :-) That graph was generated on an 8 HT core MacBook Pro (with a fair amount of other unrelated stuff running in parallel, CPU frequency changing due to heat, etc etc.), using today's 'master' branch: git clone git://github.com/basho/basho_bench.git cd basho_bench make ./basho_bench examples/null_test.config make results ... and the graph is at tests/current/summary.png. -Scott From steven.charles.davis@REDACTED Wed May 1 02:53:27 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 30 Apr 2013 19:53:27 -0500 Subject: [erlang-questions] Static type checking... In-Reply-To: References: <6AB510F1-B600-47E3-98B6-108230D507D2@gmail.com> Message-ID: <25786410-47A1-44AB-927B-0ABCCFF2BD14@gmail.com> Thanks, Jesper, for the considered answer... I fully agree and support that you recognize the importance of protocol. And given the problem, the interesting part is defining a boundary (which defines state, not just types) beyond which you should be able to assume good data or blow up. Defining that boundary at every possible library point is onerous. Static typing (compared to protocol contracts) does not account for state (which any process could potentially maintain)... What to do? I wish I had the answer, i.e. an answer that was complete and beyond just an intent of solving that issue. /s On Apr 30, 2013, at 5:38 AM, Jesper Louis Andersen wrote: > > > > On Apr 30, 2013, at 2:40 AM, Steve Davis wrote: > >> It's my experience that attempts at static type checks ends up in "turtles all the way down" and enormously onerous code. > > Static typing, preferably with inference of the types, is a really nice tool for finding and killing bugs early. But interestingly, there is not that much formality when it comes to checking the *protocol* between two processes. Since Erlang processes are often simple internally, it becomes way more interesting to have a formal protocol checker to verify adherence. I don't care what you are doing internally in your process. You could write it in any language you desired, as long as you adhere to the protocol we laid out on top. > > There is a similarity to having a signature on a module specifying a skeleton of how that code will execute type-wise. > > There are two things needed with a validation layer: > > It has to be effective. It needs to catch bugs. It needs to remove bugs before they wreak havoc on the system. > > It has to be simple. We can't afford the layer to take up considerable human resources to specify a protocol contract. Otherwise, we end up wasting time. If you are going to pay time up-front, you better know that you find errors later on in the process. Otherwise, that time was just wasted. > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > From jeremy@REDACTED Wed May 1 03:56:24 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 30 Apr 2013 18:56:24 -0700 Subject: [erlang-questions] [ANN] new erlang protocol buffer project Message-ID: https://github.com/jeremyong/eprotoc At Quark Games, we leverage protocol buffers to transfer state between server and client and back and also use it to encode data when storing it on the database. The main advantages it gives are an easy to use, contractual way to send and receive data between many different languages and domains. For more on protocol buffers: https://developers.google.com/protocol-buffers/ The problem with many existing libraries is the use of records, which make it very difficult to use protocol buffers in this way (not easily shared between modules, not resilient to code upgrades). Protocol buffers themselves are designed to fluidly transition to new versions of the same message. Other alternatives do much more than just handle protocol buffers so they weren't a great fit either. So, in light of this, I wrote my own library. The README will hopefully give an idea of where I am going with this. Given a message like package Foo; message Bar { required uint32 zap = 1; } eprotoc will generate functions like fun g_zap/1, fun s_zap/1, fun encode/1, fun decode/1 in a separate foo__bar.erl module. It will do the right thing for nested messages and enums. Make a new Bar message with zap set to 100 is as simple as Bar = foo__bar:s_zap([], 100). There are still a number of things to be done and this is mentioned in the roadmap but I am announcing it now as it is useable (I believe) and hopefully people may benefit from it. Also (shameless plug), if you like working on things like this, consider working with us at Quark Games and send me an email :). We're always looking for great people to work with. Best, Jeremy From ok@REDACTED Wed May 1 05:20:24 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 1 May 2013 15:20:24 +1200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: <517FADD6.8030006@gmail.com> References: <517FADD6.8030006@gmail.com> Message-ID: <3552C853-C634-4E5A-BE00-C6E97D9AC7FD@cs.otago.ac.nz> Some numbers: I wrote a C program to generate the numbers 0..2**31-1 with 256 of them missing, in a more-or-less random order. m% ctime a.out > gen8g.dat 89.760 user + 25.320 system = 115.080 total in 228.005 real seconds. This was on a 2.66 GHz desktop Mac with 8GB of 1.333 GHz DDR3 memory and an NFS file system. It was generating lots of random numbers and the RNG was not particularly fast. You'll notice that the CPU time was about half the real time: the difference is waiting for the file system. m% ctime wc gen8g.dat 41943037 268107752 8589933636 gen8g.dat 79.400 user + 11.530 system = 90.930 total in 283.981 real seconds. Here you see that the CPU time was less than a third of the real time. Again, the difference is waiting for the file system. I wrote a little Erlang module. It counted 8589933636 bytes taking 4261.280 user + 26.740 system = 4288.020 total in 4566.779 real seconds. One problem here is that this particular Erlang build doesn't support native compilation. By the way, there is one special quirk of the problem as given to us which makes (or might make) the "arithmetic" schemes useless: we are *not* told that there are no duplicated numbers. The trick of determining which 1 number is missing from a collection of numbers relies on there being no duplicates. From jesper.louis.andersen@REDACTED Wed May 1 11:17:48 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 1 May 2013 11:17:48 +0200 Subject: [erlang-questions] basho_bench, was Re: [ANN] Silly benchmarking In-Reply-To: <87073.1367363678@snookles.snookles.com> References: <87073.1367363678@snookles.snookles.com> Message-ID: On May 1, 2013, at 1:14 AM, Scott Lystig Fritchie wrote: > > Actually, you can use it to measure whatever you want, if you write a > callback module with a new/1 and run/4 function. Tracking throughput is > always good, but also tracking min, mean, median, 95th, 99th, 99_9th, > and maximum latency stats plus error rates is An Even BetterThing(tm). > Nice R and Gnuplot graphs is also very convenient. > Also, bootstrapping the samples to track how much your median and mean vary inside 100000 bootstrapping runs could tell you how stable your measurement seems to be. R has a really nice multi-core bootstrapping module call I intended to do for this. But that project is dormant at the moment. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From careersdublin@REDACTED Wed May 1 16:34:46 2013 From: careersdublin@REDACTED (Sumup Careers Dublin) Date: Wed, 1 May 2013 15:34:46 +0100 Subject: [erlang-questions] SumUp Limited are seeking a Senior Erlang Developer-Dublin Message-ID: Hi Everyone, I hope you don't mind me posting to your mailing list. We are currently looking for a Senior Erlang Developer to join an exciting new Start up company based in Dublin. I have attached a copy of the job description for review so if you are interested please send your CV and cover letter to careersdublin@REDACTED Many Thanks Lisa -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Software Developer-Senior Erlang.doc Type: application/msword Size: 67072 bytes Desc: not available URL: From jwilberding@REDACTED Thu May 2 19:07:16 2013 From: jwilberding@REDACTED (Jordan Wilberding) Date: Thu, 2 May 2013 13:07:16 -0400 Subject: [erlang-questions] Gittip In-Reply-To: References: Message-ID: Is there any easy way to filter by Erlang projects? On Fri, Apr 12, 2013 at 12:49 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > I just learned about https://www.gittip.com/ > > It would be awesome if as many of us as possible started funding, even > just a $1 a week, to help our favorite Erlang projects. > > I know Loic is on, https://www.gittip.com/essen/ -- so if you use Cowboy, > Ranch or Bullet, that would be a good place to start :) > > Would be great if people also gave suggestions of whom people should > donate to, even if they themselves can't afford it or if it is their > project and they know they could do more with some donations. > > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Fri May 3 17:32:16 2013 From: dan@REDACTED (Daniel Dormont) Date: Fri, 3 May 2013 11:32:16 -0400 Subject: [erlang-questions] Mnesia: strategy for auto-recovery from netsplit Message-ID: Hi Erlangers, I'm running ejabberd with a two-node cluster in my production environment. Today that system encountered a netsplit. It was properly recorded and logged. But I need to work on some way to automate a solution for this. I'm aware that the problem can't be solved in general, but there are two mitigating factors in my case: 1 - Almost all of my tables are RAM-only. 2 - None of the data are truly critical for me. That is, loss of some portion of the data isn't critical because my application can recover. So in this case, I just picked a node, restarted ejabberd on it, and all is well. But what I'd like to do is write some actual Erlang code that can subscribe to the Mnesia partitioned network event and do something about it. What are my options there? thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri May 3 18:02:34 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 3 May 2013 21:32:34 +0530 Subject: [erlang-questions] Mnesia: strategy for auto-recovery from netsplit In-Reply-To: References: Message-ID: You seem to know what to do, restart one of the nodes. Or at least restart mnesia. To restart a node programmatically you may use init:restart/0. On May 3, 2013 9:02 PM, "Daniel Dormont" wrote: > Hi Erlangers, > > I'm running ejabberd with a two-node cluster in my production environment. > Today that system encountered a netsplit. It was properly recorded and > logged. But I need to work on some way to automate a solution for this. I'm > aware that the problem can't be solved in general, but there are two > mitigating factors in my case: > > 1 - Almost all of my tables are RAM-only. > 2 - None of the data are truly critical for me. That is, loss of some > portion of the data isn't critical because my application can recover. > > So in this case, I just picked a node, restarted ejabberd on it, and all > is well. But what I'd like to do is write some actual Erlang code that can > subscribe to the Mnesia partitioned network event and do something about > it. What are my options there? > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri May 3 18:02:56 2013 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 03 May 2013 09:02:56 -0700 Subject: [erlang-questions] Mnesia: strategy for auto-recovery from netsplit In-Reply-To: References: Message-ID: <5183DFB0.7090406@gmail.com> The only solution seems to be https://github.com/uwiger/unsplit usage where you manually resolve any conflicts. Someone may already have integration with ejabberd that is available, but the problem of which side of the nodesplit to take should be error-prone, difficult, and sometimes impossible (depending on the data stored). I think it is simpler to just hookup ejabberd to postgres or mysql instead of the mnesia usage. There still remains mnesia usage internally, but I don't think the internal mnesia usage that doesn't go to postgres or mysql is distributed (would be good to check). On 05/03/2013 08:32 AM, Daniel Dormont wrote: > Hi Erlangers, > > I'm running ejabberd with a two-node cluster in my production environment. Today that system encountered a netsplit. It was properly recorded and logged. But I need to work on some way to automate a solution for this. I'm aware that the problem can't be solved in general, but there are two mitigating factors in my case: > > 1 - Almost all of my tables are RAM-only. > 2 - None of the data are truly critical for me. That is, loss of some portion of the data isn't critical because my application can recover. > > So in this case, I just picked a node, restarted ejabberd on it, and all is well. But what I'd like to do is write some actual Erlang code that can subscribe to the Mnesia partitioned network event and do something about it. What are my options there? > > thanks, > Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Fri May 3 21:47:07 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Fri, 3 May 2013 15:47:07 -0400 Subject: [erlang-questions] erlang-mode: follow more Emacs idioms Message-ID: In modern Emacs, it's recommended to install MELPAand use M-x install-package RET erlang-mode RET to get Erlang mode up and running. With most major modes, this is all that is required, however erlang-mode needs some manual configuration that we should probably work on automating during its installation. Currently, Emacs users must modify several lines in ~/.emacs in order to setup erlang-mode: (autoload 'erlang-mode "erlang" "" t) (add-to-list 'auto-mode-alist '("\\.erl\\'" . erlang-mode)) (add-to-list 'auto-mode-alist '("\\.escript\\'" . erlang-mode)) (add-hook 'erlang-mode-hook (lambda () (setq erlang-indent-level tab-width))) The first three collectively register the ".erl" and ".escript" file extensions with erlang-mode. In modern Emacs, the idiom is to register these autoloads as part of installation. For example, go-mode successfully does this. https://code.google.com/p/go/source/browse/#hg%2Fmisc%2Femacs Also, it's more idiomatic for Emacs users to set a global tab-width, and modify it by file type with hooks. As we see above, erlang-mode in its current form does not respect the conventional variable tab-width, using erlang-indent-level instead. We can improve this a bit, while maintaining backwards compatibility, by defaulting erlang-indent-level to tab-width when a buffer enters erlang-mode. What do you think? -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From johanclaesson@REDACTED Sat May 4 00:38:14 2013 From: johanclaesson@REDACTED (Johan Claesson) Date: Sat, 04 May 2013 00:38:14 +0200 Subject: [erlang-questions] erlang-mode: follow more Emacs idioms In-Reply-To: (Andrew Pennebaker's message of "Fri, 3 May 2013 15:47:07 -0400") References: Message-ID: <87txmjpu3t.fsf@bredband.net> The MELPA archive can be added to GNU Emacs 24 with: (require 'package) (add-to-list 'package-archives '("melpa" . "http://melpa.milkbox.net/packages/")) After that the following will install the erlang package: M-x package-install RET erlang RET (Or do M-x list-packages RET, go to erlang package and hit `i' and `x') The two first lisp lines in your post is already in this package. Note that they are in erlang-autoloads.el and not in erlang.el. The ".escript" extension is not added to auto-mode-list but instead an entry to interpreter-mode-alist is added. It will make any file starting with for example the line "#!/usr/bin/escript" enter erlang mode. To initialize erlang-indent-level to tab-width would break backwards compatibility since the default value of erlang-indent-level is 4 and the default value of tab-width is 8. Regards, /Johan Andrew Pennebaker writes: > In modern Emacs, it's recommended to install > MELPAand use M-x install-package RET > erlang-mode RET to get Erlang mode up and > running. With most major modes, this is all that is required, however > erlang-mode needs some manual configuration that we should probably work on > automating during its installation. Currently, Emacs users must modify > several lines in ~/.emacs in order to setup erlang-mode: > > (autoload 'erlang-mode "erlang" "" t) > (add-to-list 'auto-mode-alist '("\\.erl\\'" . erlang-mode)) > (add-to-list 'auto-mode-alist '("\\.escript\\'" . erlang-mode)) > (add-hook 'erlang-mode-hook > (lambda () > (setq erlang-indent-level tab-width))) > > The first three collectively register the ".erl" and ".escript" file > extensions with erlang-mode. In modern Emacs, the idiom is to register > these autoloads as part of installation. For example, go-mode successfully > does this. https://code.google.com/p/go/source/browse/#hg%2Fmisc%2Femacs > > Also, it's more idiomatic for Emacs users to set a global tab-width, and > modify it by file type with hooks. As we see above, erlang-mode in its > current form does not respect the conventional variable tab-width, using > erlang-indent-level instead. We can improve this a bit, while maintaining > backwards compatibility, by defaulting erlang-indent-level to tab-width > when a buffer enters erlang-mode. > > What do you think? From jon@REDACTED Sat May 4 07:10:57 2013 From: jon@REDACTED (Jon Schneider) Date: Sat, 04 May 2013 06:10:57 +0100 Subject: [erlang-questions] Quick diversion into Lisp - Emacs mode problems Message-ID: Does anybody else out there use Emacs Erlang mode with Xemacs ? I seem to get a lot of trouble with it. The most common error is Wrong type argument: stringp, nil The stacktrace shows looking-at called with nil but it isn't clear why. Though getting a stacktrace isn't difficult it doesn't really explain what is going wrong even with a little experience in Lisp. Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon May 6 08:00:02 2013 From: sean@REDACTED (Functional Jobs) Date: Mon, 6 May 2013 02:00:02 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <518746f19e6a8@functionaljobs.com> Here are some functional programming job opportunities that were posted recently:Senior Erlang Developer at SumUp Limitedhttp://functionaljobs.com/jobs/138-senior-erlang-developer-at-sumup-limitedCheers,Sean MurphyFunctionalJobs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 6 15:37:45 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 6 May 2013 17:37:45 +0400 Subject: [erlang-questions] error, case_clause 19 when connecting to SSL website Message-ID: http://pastie.org/7808529 lhttpc:request("https://www.bitstamp.net/api/order_book/", get, [], 1000). ** exception exit: {{{badmatch,{error,{asn1,{{case_clause,19}, [{'OTP-PUB-KEY',check_and_convert_restricted_string,5, [{file,"OTP-PUB-KEY.erl"},{line,14114}]}, .... What can it be? This error doesn't reproduce on my laptop. From rajmanoj.bottle@REDACTED Mon May 6 14:58:47 2013 From: rajmanoj.bottle@REDACTED (Manoj Raj) Date: Mon, 6 May 2013 18:28:47 +0530 Subject: [erlang-questions] ejabberd_listener supervisor Message-ID: A supervisor module should have the following declaration...right? -behaviour(supervisor). But in ejabberd application, ejabberd_listener supervisor module don't have the behavior thing declared...And how it works right?...Is it because it is started by another supervisor module (ejabberd_sup) as a supervisor process? Please someone clarify me... Thank you Cheers, Mandoo -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 6 15:55:03 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 6 May 2013 17:55:03 +0400 Subject: [erlang-questions] error, case_clause 19 when connecting to SSL website In-Reply-To: References: Message-ID: Looks, like the problem is in erlang version: R15B03, R15B02 works, R15B01 doesn't work. Was ssl somehow broken in B01? From lee.sylvester@REDACTED Mon May 6 20:05:56 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 6 May 2013 19:05:56 +0100 Subject: [erlang-questions] Finding node PID Message-ID: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> Hey guys, So, I'm hoping to integrate CloudI into my development. Starting CloudI involves starting it as a service. I need to get my existing Erlang apps linked to the CloudI node so I can send messages to it, but I can't get the CloudI node address. Since CloudI is started as a service, it doesn't give it to me. Is there a way to find all running Erlang nodes on the current machine? Or, perhaps, a way to start CloudI and specify a node name? Thanks, Lee From desired.mta@REDACTED Mon May 6 20:11:38 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 6 May 2013 21:11:38 +0300 Subject: [erlang-questions] Finding node PID In-Reply-To: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> Message-ID: On Mon, May 6, 2013 at 9:05 PM, Lee Sylvester wrote: > Hey guys, > > So, I'm hoping to integrate CloudI into my development. Starting CloudI involves starting it as a service. I need to get my existing Erlang apps linked to the CloudI node so I can send messages to it, but I can't get the CloudI node address. Since CloudI is started as a service, it doesn't give it to me. Is there a way to find all running Erlang nodes on the current machine? Or, perhaps, a way to start CloudI and specify a node name? `epmd -names` will give you locally running nodes in distributed mode (that is, having a node name). -- Motiejus Jak?tys From lee.sylvester@REDACTED Mon May 6 20:19:00 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 6 May 2013 19:19:00 +0100 Subject: [erlang-questions] Finding node PID In-Reply-To: References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> Message-ID: Thank you. I see this in the output: name cloudi at port 4374 However, doing any of the following returns pang: net_adm:ping(cloudi). net_adm:ping('cloudi@REDACTED'). net_adm:ping('cloudi@REDACTED'). Am I doing something silly? Thanks, Lee On 6 May 2013, at 19:11, Motiejus Jak?tys wrote: > epmd -names -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Mon May 6 20:28:51 2013 From: pablo.platt@REDACTED (pablo platt) Date: Mon, 6 May 2013 21:28:51 +0300 Subject: [erlang-questions] uncatched cowboy websocket handler termination Message-ID: Hi, I'm using a cowboy websocket handler as a simple chat transport. The websocket handler pass messages to a session gen_server. The session gen_server traps exits and links to the websocket pid so if the websocket pid dies, the session gen_server should terminate as well. I sometimes see sessions that thinks that the websocket pid is still alive although checking with is_process_alive/1 shows it is not. That leads to zombie sessions that accumulate over time. Is there a race condition in my code or something else I'm missing in catching the termination of the websocket pid? Please see a simplified version of the websocket handler and the session gen_server below. Thanks -module(my_session). -behaviour(gen_server). -export([handle_message/2]). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). -record(state, {ws}). handle_message(undefined, Msg) -> case supervisor:start_child(my_session_sup, [Msg, self()]) of {error, Reason} -> {close, 1000, <<"">>}; {ok, Pid} -> gen_server:call(Pid, {websocket, self()}), self() ! {session, Pid}, {text, <<"ok">>} end; handle_message(Pid, Msg) -> gen_server:call(Pid, {msg, Msg}). init([Msg, WS]) -> case auth(Msg) of false -> {stop, normal}; true -> process_flag(trap_exit,true), {ok, #state{}) end. handle_call({websocket, Pid}, _, State) -> link(Pid), {reply, ok, State#state{ws=Pid}}; handle_call({msg, Msg}, _, State) -> Resp = msg(Msg, State), {reply, Resp, State}. handle_info({'EXIT', _Pid, _Reason}, State) -> {stop, normal, State}. ... -module(ws_handler). -behaviour(cowboy_websocket_handler). -export([init/3]). -export([websocket_init/3]). -export([websocket_handle/3]). -export([websocket_info/3]). -export([websocket_terminate/3]). -record(state, {tref, session}). init({tcp, http}, _Req, _Opts) -> {upgrade, protocol, cowboy_websocket}. websocket_init(_TransportName, Req, _Opts) -> Tref = erlang:send_after(10000, self(), timeout), {ok, Req, #state{tref=Tref}}. websocket_handle({text, ""}, Req, State) -> erlang:cancel_timer(State#state.tref), Tref = erlang:send_after(10000, self(), timeout), {ok, Req, State#state{tref=Tref}}; websocket_handle({text, Msg}, Req, State) -> Tref = erlang:send_after(10000, self(), timeout), Resp = my_session:handle_message(State#state.session, Msg), {reply, Resp, Req, State#state{tref=Tref}}; websocket_handle(_Data, Req, State) -> {ok, Req, State}. websocket_info({session, Pid}, Req, State) -> {ok, Req, State#state{session=Pid}}; websocket_info({terminate}, Req, State) -> {reply, {close, 1000, <<"">>}, Req, State}; websocket_info({timeout, _Ref, Msg}, Req, State) -> {shutdown, Req, State}; websocket_info(_Info, Req, State) -> {ok, Req, State}. websocket_terminate(_Reason, _Req, _State) -> ok. -------------- next part -------------- An HTML attachment was scrubbed... URL: From f355@REDACTED Mon May 6 20:31:18 2013 From: f355@REDACTED (Konstantin Tcepliaev) Date: Mon, 06 May 2013 22:31:18 +0400 Subject: [erlang-questions] Finding node PID In-Reply-To: References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> Message-ID: <999381367865078@web25e.yandex.ru> An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Mon May 6 21:06:35 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 6 May 2013 20:06:35 +0100 Subject: [erlang-questions] Finding node PID In-Reply-To: <999381367865078@web25e.yandex.ru> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> <999381367865078@web25e.yandex.ru> Message-ID: <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> Hi Konstantin, How do you run them in the same VM? Sorry, still an Erlang noob. Thanks, Lee On 6 May 2013, at 19:31, Konstantin Tcepliaev wrote: > Hello, > > From what I see here: https://github.com/okeuday/CloudI/blob/master/src/rel/files/vm.args#L5 the problem is that cookies on your node and CloudI are different. > As for me, I'd choose to run both applications within the same VM, given that documentation clearly suggests this: https://github.com/okeuday/CloudI/blob/master/src/api/README.markdown > > Cheers, > -- > Konstantin > > 06.05.2013, 22:19, "Lee Sylvester" : >> Thank you. I see this in the output: >> >> name cloudi at port 4374 >> >> However, doing any of the following returns pang: >> >> net_adm:ping(cloudi). >> net_adm:ping('cloudi@REDACTED'). >> net_adm:ping('cloudi@REDACTED'). >> >> Am I doing something silly? >> >> Thanks, >> Lee >> >> >> On 6 May 2013, at 19:11, Motiejus Jak?tys wrote: >> >>> epmd -names >> , >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From f355@REDACTED Mon May 6 21:25:26 2013 From: f355@REDACTED (Konstantin Tcepliaev) Date: Mon, 06 May 2013 23:25:26 +0400 Subject: [erlang-questions] Finding node PID In-Reply-To: <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> <999381367865078@web25e.yandex.ru> <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> Message-ID: <1453831367868326@web20f.yandex.ru> An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Mon May 6 21:29:17 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 6 May 2013 20:29:17 +0100 Subject: [erlang-questions] Finding node PID In-Reply-To: <1453831367868326@web20f.yandex.ru> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> <999381367865078@web25e.yandex.ru> <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> <1453831367868326@web20f.yandex.ru> Message-ID: Oh, I know how to start an app under a supervisor, if that's what you're saying. I'm simply not sure what Cloudi start Is doing under the covers. Lee On 6 May 2013, at 20:25, Konstantin Tcepliaev wrote: > Hello Lee, > > I'm not even sure where to start. > Possible keywords to search for are the following: "applications", "releases", "reltool", "rebar dependencies". > > Cheers, > -- > Konstantin > > 06.05.2013, 23:06, "Lee Sylvester" : >> Hi Konstantin, >> >> How do you run them in the same VM? >> >> Sorry, still an Erlang noob. >> >> Thanks, >> Lee >> >> >> On 6 May 2013, at 19:31, Konstantin Tcepliaev wrote: >> >>> Hello, >>> >>> From what I see here: https://github.com/okeuday/CloudI/blob/master/src/rel/files/vm.args#L5 the problem is that cookies on your node and CloudI are different. >>> As for me, I'd choose to run both applications within the same VM, given that documentation clearly suggests this: https://github.com/okeuday/CloudI/blob/master/src/api/README.markdown >>> >>> Cheers, >>> -- >>> Konstantin >>> >>> 06.05.2013, 22:19, "Lee Sylvester" : >>>> Thank you. I see this in the output: >>>> >>>> name cloudi at port 4374 >>>> >>>> However, doing any of the following returns pang: >>>> >>>> net_adm:ping(cloudi). >>>> net_adm:ping('cloudi@REDACTED'). >>>> net_adm:ping('cloudi@REDACTED'). >>>> >>>> Am I doing something silly? >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> On 6 May 2013, at 19:11, Motiejus Jak?tys wrote: >>>> >>>>> epmd -names >>>> , >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon May 6 21:35:26 2013 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 06 May 2013 12:35:26 -0700 Subject: [erlang-questions] Finding node PID In-Reply-To: <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> <999381367865078@web25e.yandex.ru> <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> Message-ID: <518805FE.5040500@gmail.com> Hi Lee, If you build a release that depends on the cloudi_core application, it will find all the dependencies (as you can see, they are listed in the application file input here: https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi_core/src/cloudi_core.app.src.in). You could use the reltool.config file as a way of understanding how you can modify your own reltool.config file. The nodefinder applications are the exception, since they are not listed as dependencies within the application file, but are included within the release, since their usage is based on configuration values. So look at https://github.com/okeuday/CloudI/blob/master/src/reltool.config.in for more details about how the CloudI dependencies are included. That particular reltool.config is a bit explicit, but if you were using "{incl_cond, derived}, {mod_cond, derived}" within your reltool.config file, providing the cloudi_core dependency to include would be enough for reltool to find all the other dependencies, except the nodefinder applications (so the nodefinder applications can be listed separately, as shown). I will create a sample application that includes CloudI for the next release to help make this clearer as an example. It is simpler if you run both within the same Erlang VM and that helps to minimize latency. You do have the option of using the cpg module in a separate Erlang VM connected with distributed Erlang using the same cookie to communicate with CloudI services, but that approach doesn't utilize the CloudI API, so it is a bit awkward and lacks the benefits of using CloudI. Best Regards, Michael On 05/06/2013 12:06 PM, Lee Sylvester wrote: > Hi Konstantin, > > How do you run them in the same VM? > > Sorry, still an Erlang noob. > > Thanks, > Lee > > > On 6 May 2013, at 19:31, Konstantin Tcepliaev > wrote: > >> Hello, >> >> From what I see here: https://github.com/okeuday/CloudI/blob/master/src/rel/files/vm.args#L5 the problem is that cookies on your node and CloudI are different. >> As for me, I'd choose to run both applications within the same VM, given that documentation clearly suggests this: https://github.com/okeuday/CloudI/blob/master/src/api/README.markdown >> >> Cheers, >> -- >> Konstantin >> >> 06.05.2013, 22:19, "Lee Sylvester" >: >>> Thank you. I see this in the output: >>> >>> name cloudi at port 4374 >>> >>> However, doing any of the following returns pang: >>> >>> net_adm:ping(cloudi). >>> net_adm:ping('cloudi@REDACTED'). >>> net_adm:ping('cloudi@REDACTED '). >>> >>> Am I doing something silly? >>> >>> Thanks, >>> Lee >>> >>> >>> On 6 May 2013, at 19:11, Motiejus Jaks(tys > wrote: >>> >>>> epmd -names >>> , >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmacklin10@REDACTED Mon May 6 20:41:31 2013 From: danmacklin10@REDACTED (Dan Macklin) Date: Mon, 6 May 2013 19:41:31 +0100 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library Message-ID: Hello, I'd like to announce the first beta release of erlang_cep. erlang_cep is a basic CEP (Complex Event Processing) library that is inspired by esper and couchdb. It allows users to create size or time based windows, and then use erlang_js from Basho to write CEP functions. You can find the project at the following link :- https://github.com/danmacklin/erlang_cep I've written a fair amount of documentation and there are some simple examples checked into the project. I'm quite new to Erlang so any comments or suggestions would be gratefully received. Many thanks Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Mon May 6 21:53:51 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 6 May 2013 20:53:51 +0100 Subject: [erlang-questions] Finding node PID In-Reply-To: <518805FE.5040500@gmail.com> References: <4AFD6083-2052-49D1-9E56-4A17B6CF6535@gmail.com> <999381367865078@web25e.yandex.ru> <274729D4-2FD7-4364-8E11-0F081524A7AC@gmail.com> <518805FE.5040500@gmail.com> Message-ID: <860CA389-9FF1-454C-85A3-DC4E0597B81B@gmail.com> Thank you, Michael, I'll give that a try. Regards, Lee On 6 May 2013, at 20:35, Michael Truog wrote: > Hi Lee, > > If you build a release that depends on the cloudi_core application, it will find all the dependencies (as you can see, they are listed in the application file input here: https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi_core/src/cloudi_core.app.src.in). You could use the reltool.config file as a way of understanding how you can modify your own reltool.config file. The nodefinder applications are the exception, since they are not listed as dependencies within the application file, but are included within the release, since their usage is based on configuration values. So look at https://github.com/okeuday/CloudI/blob/master/src/reltool.config.in for more details about how the CloudI dependencies are included. That particular reltool.config is a bit explicit, but if you were using "{incl_cond, derived}, {mod_cond, derived}" within your reltool.config file, providing the cloudi_core dependency to include would be enough for reltool to find all the other dependencies, except the nodefinder applications (so the nodefinder applications can be listed separately, as shown). I will create a sample application that includes CloudI for the next release to help make this clearer as an example. > > It is simpler if you run both within the same Erlang VM and that helps to minimize latency. You do have the option of using the cpg module in a separate Erlang VM connected with distributed Erlang using the same cookie to communicate with CloudI services, but that approach doesn't utilize the CloudI API, so it is a bit awkward and lacks the benefits of using CloudI. > > Best Regards, > Michael > > On 05/06/2013 12:06 PM, Lee Sylvester wrote: >> Hi Konstantin, >> >> How do you run them in the same VM? >> >> Sorry, still an Erlang noob. >> >> Thanks, >> Lee >> >> >> On 6 May 2013, at 19:31, Konstantin Tcepliaev wrote: >> >>> Hello, >>> >>> From what I see here: https://github.com/okeuday/CloudI/blob/master/src/rel/files/vm.args#L5 the problem is that cookies on your node and CloudI are different. >>> As for me, I'd choose to run both applications within the same VM, given that documentation clearly suggests this: https://github.com/okeuday/CloudI/blob/master/src/api/README.markdown >>> >>> Cheers, >>> -- >>> Konstantin >>> >>> 06.05.2013, 22:19, "Lee Sylvester" : >>>> Thank you. I see this in the output: >>>> >>>> name cloudi at port 4374 >>>> >>>> However, doing any of the following returns pang: >>>> >>>> net_adm:ping(cloudi). >>>> net_adm:ping('cloudi@REDACTED'). >>>> net_adm:ping('cloudi@REDACTED'). >>>> >>>> Am I doing something silly? >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> On 6 May 2013, at 19:11, Motiejus Jak?tys wrote: >>>> >>>>> epmd -names >>>> , >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon May 6 23:54:34 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 7 May 2013 09:54:34 +1200 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: References: Message-ID: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> On 7/05/2013, at 6:41 AM, Dan Macklin wrote: > I'd like to announce the first beta release of erlang_cep. Looks interesting. I have just begun to read the documentation, but I don't understand why Erlang code should be specifying what to do by stuffing Javascript code into binaries instead of using Erlang functions. From zerthurd@REDACTED Tue May 7 05:00:20 2013 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 7 May 2013 10:00:20 +0700 Subject: [erlang-questions] PG2 accidentally restarted Message-ID: Hello I used R16B (erts-5.10.1). There is one pg2 group in my code used only on single node. Today I saw that after hours of working without high load main process of pg2 accidentally stopped and restarted by kernel_safe_sup: 04:20:38.633 [debug] Supervisor kernel_safe_sup started pg2:start_link() at pid <0.118.0> And there is no any another errors before which can be cause of this stopping. After this pg2 restart my own group was lost and error appeared: pg2:get_closest_pid(my_group) returns {error,{no_such_group,my_group}} During work of my code, some number of processes joins to my_group, then takes one task to handle and leaves this group. After task completed process returns reply and joins to my_group again. Is this bug? Thank you -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Tue May 7 05:09:43 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 7 May 2013 06:09:43 +0300 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> References: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> Message-ID: On Tue, May 7, 2013 at 12:54 AM, Richard A. O'Keefe wrote: > > On 7/05/2013, at 6:41 AM, Dan Macklin wrote: > >> I'd like to announce the first beta release of erlang_cep. > > Looks interesting. I have just begun to read the documentation, > but I don't understand why Erlang code should be specifying > what to do by stuffing Javascript code into binaries instead of > using Erlang functions. The idea is the same in Riak (like mentioned in the announcement), they provide an option to use javascript for map/reduce implementation. Same here: implement "reduce" logic in javascript. However, I fail to see the reason why this might be useful in this case, as I cannot see any REST API, and reduce functions are quite expensive (thus I imply there is no notion of "request", where you can implement your match/reduce functions). -- Motiejus Jak?tys From yendris.cruz@REDACTED Mon May 6 22:22:56 2013 From: yendris.cruz@REDACTED (Yendris Cruz Mendoza) Date: Mon, 06 May 2013 16:22:56 -0400 Subject: [erlang-questions] ChicagoBoss in Yaws Message-ID: Hi everyone, is possible run ChicagoBoss in yaws server? From danmacklin10@REDACTED Tue May 7 07:50:26 2013 From: danmacklin10@REDACTED (Dan Macklin) Date: Tue, 7 May 2013 06:50:26 +0100 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> References: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> Message-ID: Hi Richard, When I first wrote the system I envisaged that users would input queries via a web front end, which would be stored in a database and loaded in dynamically. I thought that JavaScript would be a more accessible language for this use case. However as I have learned more I have also realised that I can do the same in erlang or indeed a simple dsl. I think I'll add a config option that allows the use of both. Shouldn't take long to do. Thanks for the feedback. Dan On Monday, May 6, 2013, Richard A. O'Keefe wrote: > > On 7/05/2013, at 6:41 AM, Dan Macklin wrote: > > > I'd like to announce the first beta release of erlang_cep. > > Looks interesting. I have just begun to read the documentation, > but I don't understand why Erlang code should be specifying > what to do by stuffing Javascript code into binaries instead of > using Erlang functions. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmacklin10@REDACTED Tue May 7 08:05:08 2013 From: danmacklin10@REDACTED (Dan Macklin) Date: Tue, 7 May 2013 07:05:08 +0100 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: References: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> Message-ID: > However, I fail to see the reason why this might be useful in this > case, as I cannot see any REST API, and reduce functions are quite > expensive (thus I imply there is no notion of "request", where you can > implement your match/reduce functions). Hi, You're right the reduce is similar to Riak or CouchDB. I wanted to give users the ability to iterate over matches within a window and perform logic over them before the results are passed onto a subscribed process (this could just be output formatting or simple averaging logic). The difference here is that the reduce function only runs when the CEP system finds a match, and you can control the size of the matched data. This should mean that the amount of data that the reduce runs over is quite small. However a rest interface seems a great idea, why didn't I think of that! So I'll add it to my todo list. Many thanks Dan On Tuesday, May 7, 2013, Motiejus Jak?tys wrote: > On Tue, May 7, 2013 at 12:54 AM, Richard A. O'Keefe > > wrote: > > > > On 7/05/2013, at 6:41 AM, Dan Macklin wrote: > > > >> I'd like to announce the first beta release of erlang_cep. > > > > Looks interesting. I have just begun to read the documentation, > > but I don't understand why Erlang code should be specifying > > what to do by stuffing Javascript code into binaries instead of > > using Erlang functions. > > The idea is the same in Riak (like mentioned in the announcement), > they provide an option to use javascript for map/reduce > implementation. Same here: implement "reduce" logic in javascript. > > However, I fail to see the reason why this might be useful in this > case, as I cannot see any REST API, and reduce functions are quite > expensive (thus I imply there is no notion of "request", where you can > implement your match/reduce functions). > > -- > Motiejus Jak?tys > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Tue May 7 10:30:36 2013 From: davidnwelton@REDACTED (David Welton) Date: Tue, 7 May 2013 10:30:36 +0200 Subject: [erlang-questions] ChicagoBoss in Yaws In-Reply-To: References: Message-ID: At the moment, it's not one of the supported servers - you might ask on the Chicago Boss mailing list to see if anyone's had a look at integrating it. On Mon, May 6, 2013 at 10:22 PM, Yendris Cruz Mendoza wrote: > Hi everyone, is possible run ChicagoBoss in yaws server? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From davidnwelton@REDACTED Tue May 7 10:34:59 2013 From: davidnwelton@REDACTED (David Welton) Date: Tue, 7 May 2013 10:34:59 +0200 Subject: [erlang-questions] erlexec and its environment Message-ID: Hi, We're doing some work to fix up a few things we perceive as deficiencies in erlexec, and thought about asking for a community opinion. Currently, this: exec:run_link("exec /tmp/print_env.sh", []) Launches the child process with no environment, which was quite a surprise to us - the default on Unix systems is to have child processes inherit the parent process' environment. We're hacking the system so that it'll behave like we expect. When an environment is specified, we were thinking to use that to say "ok, clear out the environment and don't inherit it": exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) So that if you really needed to pass it in, you could do: exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) Thoughts? -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From desired.mta@REDACTED Tue May 7 10:42:20 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 7 May 2013 10:42:20 +0200 Subject: [erlang-questions] Fwd: [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: References: <972259A9-2915-4E44-8F43-EF9C76D0B4CB@cs.otago.ac.nz> Message-ID: On Tue, May 7, 2013 at 7:50 AM, Dan Macklin wrote: > Hi Richard, > > When I first wrote the system I envisaged that users would input queries via > a web front end, which would be stored in a database and loaded in > dynamically. I thought that JavaScript would be a more accessible language > for this use case. However as I have learned more I have also realised that > I can do the same in erlang or indeed a simple dsl. I would advise against a "simple dsl". Pick erlang, javascript or lua. Or something similar, a.k.a. a language. It makes life easier for everyone... -- Motiejus Jak?tys From watson.timothy@REDACTED Tue May 7 10:50:27 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 7 May 2013 09:50:27 +0100 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: References: Message-ID: <67C359DF-E334-4C8C-AB0C-1C6B197EA191@gmail.com> Got any benchmarks? If/when I find time (and that's a fairly big if/when), I'd like to hook up RabbitMQ as an input source. Does the feeds API make that easy (ish) to do? Cheers Tim On 6 May 2013, at 19:41, Dan Macklin wrote: > Hello, > > I'd like to announce the first beta release of erlang_cep. > > erlang_cep is a basic CEP (Complex Event Processing) library that is inspired by esper and couchdb. It allows users to create size or time based windows, and then use erlang_js from Basho to write CEP functions. > > You can find the project at the following link :- > > https://github.com/danmacklin/erlang_cep > > I've written a fair amount of documentation and there are some simple examples checked into the project. > > I'm quite new to Erlang so any comments or suggestions would be gratefully received. > > Many thanks > > Dan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Tue May 7 10:55:21 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 7 May 2013 10:55:21 +0200 Subject: [erlang-questions] erlexec and its environment In-Reply-To: References: Message-ID: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, There are some Erlang functions that have {env, [...]} argument. Ex: erlang:open_port/2. I think it would be good to have the same syntax ("a list of tuples {Name, Val}, where Name is the name of an environment variable, and Val is the value it is to have in the spawned port process." and the same semantics for exec:run_link/2's {env, [...]}. bengt On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: > Hi, > > We're doing some work to fix up a few things we perceive as > deficiencies in erlexec, and thought about asking for a community > opinion. > > Currently, this: > > exec:run_link("exec /tmp/print_env.sh", []) > > Launches the child process with no environment, which was quite a > surprise to us - the default on Unix systems is to have child > processes inherit the parent process' environment. We're hacking the > system so that it'll behave like we expect. > > When an environment is specified, we were thinking to use that to say > "ok, clear out the environment and don't inherit it": > > exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) > > So that if you really needed to pass it in, you could do: > > exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) > > Thoughts? > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From davidnwelton@REDACTED Tue May 7 11:02:38 2013 From: davidnwelton@REDACTED (David Welton) Date: Tue, 7 May 2013 11:02:38 +0200 Subject: [erlang-questions] erlexec and its environment In-Reply-To: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: > There are some Erlang functions that have {env, [...]} argument. Ex: > erlang:open_port/2. > > I think it would be good to have the same syntax ("a list of tuples > {Name, Val}, where Name is the name of an environment variable, and Val > is the value it is to have in the spawned port process." and the same > semantics for exec:run_link/2's {env, [...]}. That's a good point: {env, Env}: This is only valid for {spawn, Command} and {spawn_executable, FileName}. The environment of the started process is extended using the environment specifications in Env. This means that it does not clear out the child process' environment, ever - if env is not specified, it inherits it like you'd expect it would. That actually works fine for us, I can't think of a reason offhand why we'd want to wipe the environment and feed the child process an empty one. (Also: why oh why does Erlang not have the capability of handling stdout and stderr as separate things, as well as a kill function .... argh! ) -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From jesper.louis.andersen@REDACTED Tue May 7 11:24:09 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 7 May 2013 11:24:09 +0200 Subject: [erlang-questions] ejabberd_listener supervisor In-Reply-To: References: Message-ID: <18AF6A1F-0FDD-45B4-A0CC-42F89FF7F7F8@erlang-solutions.com> In principle it should have a -behaviour declaration, yes. But in Erlang it is enough if the module exports the right functions so the supervisor module can make the correct callbacks. It is not checked. The reason for the -behaviour line is two-fold: * It serves as documentation for the reader. The reader will know the general structure of the module given its behaviour. * It provides the compiler an opportunity of warnings if the wrong functions are defined. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On May 6, 2013, at 2:58 PM, Manoj Raj wrote: > A supervisor module should have the following declaration...right? > > -behaviour(supervisor). > > But in ejabberd application, ejabberd_listener supervisor module don't have the behavior thing declared...And how it works right?...Is it because it is started by another supervisor module (ejabberd_sup) as a supervisor process? > > Please someone clarify me... > > Thank you > > Cheers, > Mandoo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Tue May 7 11:54:07 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 7 May 2013 11:54:07 +0200 Subject: [erlang-questions] error, case_clause 19 when connecting to SSL website In-Reply-To: References: Message-ID: In R15B02 we introduced the following workaround (from release notes ssl-5.1) "Workaround for handling certificates that wrongly encode X509countryname in utf-8 when the actual value is a valid ASCCI value of length 2. Such certificates are accepted by many browsers such as Chrome and Firefox so for interoperability reasons we will too. Own Id: OTP-10222" So that is your explanation! Regards Ingela Erlang/OTP team - Ericsson AB 2013/5/6 Max Lapshin > Looks, like the problem is in erlang version: R15B03, R15B02 works, > R15B01 doesn't work. > > Was ssl somehow broken in B01? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue May 7 11:55:35 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 7 May 2013 13:55:35 +0400 Subject: [erlang-questions] error, case_clause 19 when connecting to SSL website In-Reply-To: References: Message-ID: Thank you a lot, Ingela. From wckbluesky@REDACTED Tue May 7 12:00:57 2013 From: wckbluesky@REDACTED (=?gb2312?B?zqSz559q?=) Date: Tue, 7 May 2013 18:00:57 +0800 Subject: [erlang-questions] nif problem in windows Message-ID: <201305071800568105382@yahoo.com.cn> hi I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows Everything is ok before I running "c(complex6)" in erlang VM, in windows, it reports =ERROR REPORT==== 7-May-2013::17:36:19 === Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi le,"code_server.erl"},... =ERROR REPORT==== 7-May-2013::17:36:19 === The on_load function for module complex6 returned {{badmatch, {error, {load_failed, "Failed to load NIF library ./complex6_nif: 'Unspecified error'"}}}, [{complex6,init,0, [{file,...},{...}]}, {code_server, '-handle_on_load/4-fun-0-', 1, [{...}|...]}]} {error,on_load_failure} It is hard to find some useful information in google. Can anyone tell how to fix it? ckwei Best Regards From magnus@REDACTED Tue May 7 12:15:21 2013 From: magnus@REDACTED (Magnus Henoch) Date: Tue, 07 May 2013 11:15:21 +0100 Subject: [erlang-questions] erlang-mode: follow more Emacs idioms In-Reply-To: (Andrew Pennebaker's message of "Fri, 3 May 2013 15:47:07 -0400") References: Message-ID: Andrew Pennebaker writes: > In modern Emacs, it's recommended to install > MELPAand use M-x install-package RET > erlang-mode RET to get Erlang mode up and > running. With most major modes, this is all that is required, however > erlang-mode needs some manual configuration that we should probably work on > automating during its installation. Currently, Emacs users must modify > several lines in ~/.emacs in order to setup erlang-mode: > > (autoload 'erlang-mode "erlang" "" t) > (add-to-list 'auto-mode-alist '("\\.erl\\'" . erlang-mode)) > (add-to-list 'auto-mode-alist '("\\.escript\\'" . erlang-mode)) > (add-hook 'erlang-mode-hook > (lambda () > (setq erlang-indent-level tab-width))) > > The first three collectively register the ".erl" and ".escript" file > extensions with erlang-mode. In modern Emacs, the idiom is to register > these autoloads as part of installation. For example, go-mode successfully > does this. https://code.google.com/p/go/source/browse/#hg%2Fmisc%2Femacs The first two of those lines shouldn't be necessary since 22nd March. Have a look at ~/.emacs.d/elpa/erlang-20130322.1917/erlang-autoloads.el, and you should see that corresponding forms have been generated already. There is also a line that enables erlang-mode for files that start with "#!/usr/bin/env escript" or similar - that covers most of my escript files. I wouldn't mind adding the rule for "*.escript" files, though. > Also, it's more idiomatic for Emacs users to set a global tab-width, and > modify it by file type with hooks. As we see above, erlang-mode in its > current form does not respect the conventional variable tab-width, using > erlang-indent-level instead. We can improve this a bit, while maintaining > backwards compatibility, by defaulting erlang-indent-level to tab-width > when a buffer enters erlang-mode. > > What do you think? While this might be a good idea, I disagree that it's conventional :) Most programming language modes have a separate variable for setting the indentation step (c-basic-offset, sh-basic-offset, ruby-indent-level), independent of tab-width. (Python and makefile modes are exceptions, for obvious reasons, and it seems Go mode exclusively uses tab-width.) Erlang mode indentation depends on the combination erlang-indent-level and tab-width, so this change wouldn't be backwards compatible. You can try it by making the change and running "make test_indentation" in lib/tools/emacs. So this might be a good policy for a new project, but not as a default value. I've been working on changes to erlang-mode to make it more friendly to the "one indentation level, one tab" indentation style (in particular deactivating lining up lines with specific elements in a previous line), but it feels like I keep adding special cases; it would need some cleanup before I could submit a patch for it. For the curious, here it is: https://github.com/legoscia/otp/commits/dogfood-r15b03/lib/tools/emacs/erlang.el Regards, Magnus From dch@REDACTED Tue May 7 13:37:14 2013 From: dch@REDACTED (Dave Cottlehuber) Date: Tue, 7 May 2013 13:37:14 +0200 Subject: [erlang-questions] nif problem in windows In-Reply-To: References: <201305071800568105382@yahoo.com.cn> Message-ID: On 7 May 2013 12:00, ??? wrote: > hi > > I am testing nif in windows right now, the nif c code is very simple and > it is copyed directly from > http://www.erlang.org/doc/tutorial/nif.html#id64988 > > All the code is running well in Linux, and c code is compiled to .so in > linux, .dll in windows > > Everything is ok before I running "c(complex6)" in erlang VM, in windows, > > it reports > > =ERROR REPORT==== 7-May-2013::17:36:19 === > Error in process <0.70.0> with exit value: > {{badmatch,{error,{load_failed,"Faile > d to load NIF library ./complex6_nif: 'Unspecified > error'"}}},[{complex6,init,0, > > [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi > le,"code_server.erl"},... > > > =ERROR REPORT==== 7-May-2013::17:36:19 === > The on_load function for module complex6 returned {{badmatch, > {error, > {load_failed, > "Failed to load NIF > library ./complex6_nif: 'Unspecified error'"}}}, > [{complex6,init,0, > [{file,...},{...}]}, > {code_server, > > '-handle_on_load/4-fun-0-', > 1, > [{...}|...]}]} > {error,on_load_failure} > > It is hard to find some useful information in google. > Can anyone tell how to fix it? > > Hi ckwei, It might be useful to paste some of the source code that isn't working for you. I don't have windows handy, but here's a NIF example that works on both Windows using command shell and SDK 7.1, and LInux, that Volker & I did last year. The erlang side NIF loader was poached from one of Paul Davis' projects IIRC. https://github.com/vmx/hellonif A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngreco@REDACTED Tue May 7 14:43:24 2013 From: ngreco@REDACTED (Nahuel Greco) Date: Tue, 7 May 2013 09:43:24 -0300 Subject: [erlang-questions] Quick diversion into Lisp - Emacs mode problems In-Reply-To: References: Message-ID: I was an XEmacs user until a couple of years ago. I think XEmacs development stagnated and currently there is no reason to prefer it over Emacs 23/24. Give a try to Emacs, you will avoid this and many other problems. Saludos, Nahuel Greco. -- Sent from my mobile device. El 04/05/2013 02:11, "Jon Schneider" escribi?: > Does anybody else out there use Emacs Erlang mode with Xemacs ? I seem to > get a lot of trouble with it. > > The most common error is > > Wrong type argument: stringp, nil > The stacktrace shows looking-at called with nil but it isn't clear why. > > Though getting a stacktrace isn't difficult it doesn't really explain what > is going wrong even with a little experience in Lisp. > > Jon > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Tue May 7 15:16:36 2013 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 7 May 2013 09:16:36 -0400 Subject: [erlang-questions] nif problem in windows In-Reply-To: <201305071800568105382@yahoo.com.cn> References: <201305071800568105382@yahoo.com.cn> Message-ID: <20130507131636.GA3340@ioctl> On Tue, May 07, 2013 at 06:00:57PM +0800, ??? wrote: > hi > > I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 > > All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows > > Everything is ok before I running "c(complex6)" in erlang VM, in windows, > > it reports > > =ERROR REPORT==== 7-May-2013::17:36:19 === > Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile > d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, Under Windows, does specifiying the path as "./complex6_nif" work? If so, is complex6_nif.dll in the current working directory? From dch@REDACTED Tue May 7 15:31:06 2013 From: dch@REDACTED (Dave Cottlehuber) Date: Tue, 7 May 2013 15:31:06 +0200 Subject: [erlang-questions] nif problem in windows In-Reply-To: <201305072115176044184@yahoo.com.cn> References: <201305071800568105382@yahoo.com.cn> <201305072115176044184@yahoo.com.cn> Message-ID: On 7 May 2013 15:15, ?????j wrote: > ** > hi, Dave, > > Thanks for you replying, I try your example and it is return the same > error I wrote before, here is the output: > > E:\firefox_download\hellonif-master>rebar.cmd compile > ==> hellonif-master (compile) > Compiled src/hello.erl > > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 > Copyright (C) Microsoft Corporation. All rights reserved. > > This is SDK 7.1 IIRC. That's good. > hello.c > > D:\Program Files (x86)\Microsoft Visual Studio 8\VC\INCLUDE\stdlib.h(215) : warn > I think you have an SDK that doesn't match the header files you're providing it here. No idea if that is supposed to work, but it smells bad to me. ing C4255: '_get_purecall_handler' : no function prototype given: converting '() > ' to '(void)' > > d:\PROGRA~2\ERL510~1.1\erts-5.10.1\include\erl_nif.h(132) : warning C4820: ' The above NIF was done with R14 only at the time. I'm pretty sure R15 also works fine, as we use this same approach in CouchDB today, but I've not tested it in R16B yet. > amed-tag>' : '4' bytes padding added after data member 'ref_bin' > c_src/hello.c(3) : warning C4100: 'argv' : unreferenced formal parameter > c_src/hello.c(3) : warning C4100: 'argc' : unreferenced formal parameter > > c_src/hello.c(13) : warning C4013: 'memcpy' undefined; assuming extern returning > int > Microsoft (R) Incremental Linker Version 8.00.50727.42 > Copyright (C) Microsoft Corporation. All rights reserved. > > This linker doesn't match the SDK. > ?????????? priv/hello.lib ?????? priv/hello.exp > > > E:\firefox_download\hellonif-master>erl -pa ebin > Eshell V5.10.1 (abort with ^G) > 1> hello:init(). > > {error,{load_failed,"Failed to load NIF library priv/hello: 'Unspecified error'" > }} > 2> hello:world(). > "NIF library not loaded" > 3> > > > Notice that, I change the code hello.erl slightly, it catch all the error > when erlang:load_nif, I delete it, so it can output the error message > I using operating system win7, is it matters? > > OS shouldn't matter, no. But the mismatch of compiler + linker + header files likely does matter. Can you try this with a clean environment (path set to minimum windows specific ones, and libs, include variables set to empty) and then run setsdk.cmd /x86 /release before adding erlang headers and path? That should be enough to keep a clean compile using only the SDK components, and skipping Visual C++ 8. > ------------------------------ > ckwei > > *From:* Dave Cottlehuber > *Date:* 2013-05-07 19:37 > *To:* erlang-questions > *Subject:* Re: [erlang-questions] nif problem in windows > On 7 May 2013 12:00, ?????j wrote: > >> hi >> >> I am testing nif in windows right now, the nif c code is very simple and >> it is copyed directly from >> http://www.erlang.org/doc/tutorial/nif.html#id64988 >> >> All the code is running well in Linux, and c code is compiled to .so in >> linux, .dll in windows >> >> Everything is ok before I running "c(complex6)" in erlang VM, in windows, >> >> it reports >> >> =ERROR REPORT==== 7-May-2013::17:36:19 === >> Error in process <0.70.0> with exit value: >> {{badmatch,{error,{load_failed,"Faile >> d to load NIF library ./complex6_nif: 'Unspecified >> error'"}}},[{complex6,init,0, >> >> [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi >> le,"code_server.erl"},... >> >> >> =ERROR REPORT==== 7-May-2013::17:36:19 === >> The on_load function for module complex6 returned {{badmatch, >> {error, >> {load_failed, >> "Failed to load NIF >> library ./complex6_nif: 'Unspecified error'"}}}, >> [{complex6,init,0, >> [{file,...},{...}]}, >> {code_server, >> >> '-handle_on_load/4-fun-0-', >> 1, >> [{...}|...]}]} >> {error,on_load_failure} >> >> It is hard to find some useful information in google. >> Can anyone tell how to fix it? >> >> > Hi ckwei, > > It might be useful to paste some of the source code that isn't working for > you. I don't have windows handy, but here's a NIF example that works on > both Windows using command shell and SDK 7.1, and LInux, that Volker & I > did last year. The erlang side NIF loader was poached from one of Paul > Davis' projects IIRC. > > https://github.com/vmx/hellonif > > A+ > Dave > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Tue May 7 15:43:57 2013 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 7 May 2013 09:43:57 -0400 Subject: [erlang-questions] ChicagoBoss in Yaws In-Reply-To: References: Message-ID: If anyone wants help getting ChicagoBoss running on Yaws, just let me know. I see that the source code contains a file include/yaws_api.hrl, so somebody must have been working on it it at some point. I'm sure it would be pretty easy to get it running on Yaws. --steve On Tue, May 7, 2013 at 4:30 AM, David Welton wrote: > At the moment, it's not one of the supported servers - you might ask > on the Chicago Boss mailing list to see if anyone's had a look at > integrating it. > > On Mon, May 6, 2013 at 10:22 PM, Yendris Cruz Mendoza > wrote: > > Hi everyone, is possible run ChicagoBoss in yaws server? > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wckbluesky@REDACTED Tue May 7 15:15:18 2013 From: wckbluesky@REDACTED (=?utf-8?B?6Z+m5bSH54Sc?=) Date: Tue, 7 May 2013 21:15:18 +0800 Subject: [erlang-questions] nif problem in windows References: <201305071800568105382@yahoo.com.cn> , Message-ID: <201305072115176044184@yahoo.com.cn> hi, Dave, Thanks for you replying, I try your example and it is return the same error I wrote before, here is the output: E:\firefox_download\hellonif-master>rebar.cmd compile ==> hellonif-master (compile) Compiled src/hello.erl Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. hello.c D:\Program Files (x86)\Microsoft Visual Studio 8\VC\INCLUDE\stdlib.h(215) : warn ing C4255: '_get_purecall_handler' : no function prototype given: converting '() ' to '(void)' d:\PROGRA~2\ERL510~1.1\erts-5.10.1\include\erl_nif.h(132) : warning C4820: '' : '4' bytes padding added after data member 'ref_bin' c_src/hello.c(3) : warning C4100: 'argv' : unreferenced formal parameter c_src/hello.c(3) : warning C4100: 'argc' : unreferenced formal parameter c_src/hello.c(13) : warning C4013: 'memcpy' undefined; assuming extern returning int Microsoft (R) Incremental Linker Version 8.00.50727.42 Copyright (C) Microsoft Corporation. All rights reserved. ????? priv/hello.lib ??? priv/hello.exp E:\firefox_download\hellonif-master>erl -pa ebin Eshell V5.10.1 (abort with ^G) 1> hello:init(). {error,{load_failed,"Failed to load NIF library priv/hello: 'Unspecified error'" }} 2> hello:world(). "NIF library not loaded" 3> Notice that, I change the code hello.erl slightly, it catch all the error when erlang:load_nif, I delete it, so it can output the error message I using operating system win7, is it matters? ckwei From: Dave Cottlehuber Date: 2013-05-07 19:37 To: erlang-questions Subject: Re: [erlang-questions] nif problem in windows On 7 May 2013 12:00, ??? wrote: hi I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows Everything is ok before I running "c(complex6)" in erlang VM, in windows, it reports =ERROR REPORT==== 7-May-2013::17:36:19 === Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi le,"code_server.erl"},... =ERROR REPORT==== 7-May-2013::17:36:19 === The on_load function for module complex6 returned {{badmatch, {error, {load_failed, "Failed to load NIF library ./complex6_nif: 'Unspecified error'"}}}, [{complex6,init,0, [{file,...},{...}]}, {code_server, '-handle_on_load/4-fun-0-', 1, [{...}|...]}]} {error,on_load_failure} It is hard to find some useful information in google. Can anyone tell how to fix it? Hi ckwei, It might be useful to paste some of the source code that isn't working for you. I don't have windows handy, but here's a NIF example that works on both Windows using command shell and SDK 7.1, and LInux, that Volker & I did last year. The erlang side NIF loader was poached from one of Paul Davis' projects IIRC. https://github.com/vmx/hellonif A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Tue May 7 16:55:23 2013 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 7 May 2013 09:55:23 -0500 Subject: [erlang-questions] ChicagoBoss in Yaws In-Reply-To: References: Message-ID: Without me knowing much of anything of the internals of CB, my guess is that's possibly a reference to the fact that Chicago Boss used to have a copy of simple_bridge rather than using it as a dependency. Seeing as ChicagoBoss uses simple_bridge, it should be pretty easy getting it going with Yaws. You could check out the yaws-specific code that's used by Nitrogen for using simple_bridge with Yaws: https://github.com/nitrogen/nitrogen/tree/master/rel/overlay/yaws (site/src/ is the code, and etc/ are the config files) Yaws is definitely one of the simplest servers to interface with using simple_bridge. -Jesse On May 7, 2013 8:44 AM, "Steve Vinoski" wrote: > If anyone wants help getting ChicagoBoss running on Yaws, just let me > know. I see that the source code contains a file include/yaws_api.hrl, so > somebody must have been working on it it at some point. I'm sure it would > be pretty easy to get it running on Yaws. > > --steve > > > On Tue, May 7, 2013 at 4:30 AM, David Welton wrote: > >> At the moment, it's not one of the supported servers - you might ask >> on the Chicago Boss mailing list to see if anyone's had a look at >> integrating it. >> >> On Mon, May 6, 2013 at 10:22 PM, Yendris Cruz Mendoza >> wrote: >> > Hi everyone, is possible run ChicagoBoss in yaws server? >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> David N. Welton >> >> http://www.welton.it/davidw/ >> >> http://www.dedasys.com/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Tue May 7 17:51:30 2013 From: dan@REDACTED (Daniel Dormont) Date: Tue, 7 May 2013 11:51:30 -0400 Subject: [erlang-questions] Mnesia: strategy for auto-recovery from netsplit In-Reply-To: <5183DFB0.7090406@gmail.com> References: <5183DFB0.7090406@gmail.com> Message-ID: It is. For example the mappings between Jabber IDs of various kinds (user, chatroom, etc) and process IDs are kept in Mnesia tables which are distributed - in fact this is really the core of how clustered ejabberd works. So I will really need to do something here. A brief past experiment suggested that ejabberd did not take kindly to a Mnesia restart on a live node - I think I will have to restart the node. A related question while I'm thinking of it - are there any modules out there that can hook into the error logger (or configuration options in the error logger) and do something different with certain log messages - for example send them by email? Dan On Fri, May 3, 2013 at 12:02 PM, Michael Truog wrote: > The only solution seems to be https://github.com/uwiger/unsplit usage > where you manually resolve any conflicts. Someone may already have > integration with ejabberd that is available, but the problem of which side > of the nodesplit to take should be error-prone, difficult, and sometimes > impossible (depending on the data stored). I think it is simpler to just > hookup ejabberd to postgres or mysql instead of the mnesia usage. There > still remains mnesia usage internally, but I don't think the internal > mnesia usage that doesn't go to postgres or mysql is distributed (would be > good to check). > > > On 05/03/2013 08:32 AM, Daniel Dormont wrote: > > Hi Erlangers, > > I'm running ejabberd with a two-node cluster in my production > environment. Today that system encountered a netsplit. It was properly > recorded and logged. But I need to work on some way to automate a solution > for this. I'm aware that the problem can't be solved in general, but there > are two mitigating factors in my case: > > 1 - Almost all of my tables are RAM-only. > 2 - None of the data are truly critical for me. That is, loss of some > portion of the data isn't critical because my application can recover. > > So in this case, I just picked a node, restarted ejabberd on it, and all > is well. But what I'd like to do is write some actual Erlang code that can > subscribe to the Mnesia partitioned network event and do something about > it. What are my options there? > > thanks, > Dan > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Tue May 7 18:42:37 2013 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 07 May 2013 12:42:37 -0400 Subject: [erlang-questions] erlexec and its environment In-Reply-To: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <51892EFD.7090106@aleynikov.org> The master branch of erlexec now has support for both {Name, Val} and "VAR=VALUE" environment options. On 5/7/2013 4:55 AM, Bengt Kleberg wrote: > Greetings, > > There are some Erlang functions that have {env, [...]} argument. Ex: > erlang:open_port/2. > > I think it would be good to have the same syntax ("a list of tuples > {Name, Val}, where Name is the name of an environment variable, and Val > is the value it is to have in the spawned port process." and the same > semantics for exec:run_link/2's {env, [...]}. > > > bengt > > On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: >> Hi, >> >> We're doing some work to fix up a few things we perceive as >> deficiencies in erlexec, and thought about asking for a community >> opinion. >> >> Currently, this: >> >> exec:run_link("exec /tmp/print_env.sh", []) >> >> Launches the child process with no environment, which was quite a >> surprise to us - the default on Unix systems is to have child >> processes inherit the parent process' environment. We're hacking the >> system so that it'll behave like we expect. >> >> When an environment is specified, we were thinking to use that to say >> "ok, clear out the environment and don't inherit it": >> >> exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) >> >> So that if you really needed to pass it in, you could do: >> >> exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) >> >> Thoughts? >> -- >> David N. Welton >> >> http://www.welton.it/davidw/ >> >> http://www.dedasys.com/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From janos.n.hary@REDACTED Tue May 7 19:06:01 2013 From: janos.n.hary@REDACTED (Janos Hary) Date: Tue, 7 May 2013 19:06:01 +0200 Subject: [erlang-questions] nif problem in windows In-Reply-To: <201305072115176044184@yahoo.com.cn> References: <201305071800568105382@yahoo.com.cn> , <201305072115176044184@yahoo.com.cn> Message-ID: <001e01ce4b45$273e6430$75bb2c90$@gmail.com> Your error message says ?priv/hello?. Check the path in your erlang:load_nif. Janos From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of ??? Sent: Tuesday, May 07, 2013 3:15 PM To: Dave Cottlehuber Cc: erlang-questions Subject: Re: [erlang-questions] nif problem in windows hi, Dave, Thanks for you replying, I try your example and it is return the same error I wrote before, here is the output: E:\firefox_download\hellonif-master>rebar.cmd compile ==> hellonif-master (compile) Compiled src/hello.erl Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. hello.c D:\Program Files (x86)\Microsoft Visual Studio 8\VC\INCLUDE\stdlib.h(215) : warn ing C4255: '_get_purecall_handler' : no function prototype given: converting '() ' to '(void)' d:\PROGRA~2\ERL510~1.1\erts-5.10.1\include\erl_nif.h(132) : warning C4820: '' : '4' bytes padding added after data member 'ref_bin' c_src/hello.c(3) : warning C4100: 'argv' : unreferenced formal parameter c_src/hello.c(3) : warning C4100: 'argc' : unreferenced formal parameter c_src/hello.c(13) : warning C4013: 'memcpy' undefined; assuming extern returning int Microsoft (R) Incremental Linker Version 8.00.50727.42 Copyright (C) Microsoft Corporation. All rights reserved. ????? priv/hello.lib ??? priv/hello.exp E:\firefox_download\hellonif-master>erl -pa ebin Eshell V5.10.1 (abort with ^G) 1> hello:init(). {error,{load_failed,"Failed to load NIF library priv/hello: 'Unspecified error'" }} 2> hello:world(). "NIF library not loaded" 3> Notice that, I change the code hello.erl slightly, it catch all the error when erlang:load_nif, I delete it, so it can output the error message I using operating system win7, is it matters? _____ ckwei From: Dave Cottlehuber Date: 2013-05-07 19:37 To: erlang-questions Subject: Re: [erlang-questions] nif problem in windows On 7 May 2013 12:00, ??? wrote: hi I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows Everything is ok before I running "c(complex6)" in erlang VM, in windows, it reports =ERROR REPORT==== 7-May-2013::17:36:19 === Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi le,"code_server.erl"},... =ERROR REPORT==== 7-May-2013::17:36:19 === The on_load function for module complex6 returned {{badmatch, {error, {load_failed, "Failed to load NIF library ./complex6_nif: 'Unspecified error'"}}}, [{complex6,init,0, [{file,...},{...}]}, {code_server, '-handle_on_load/4-fun-0-', 1, [{...}|...]}]} {error,on_load_failure} It is hard to find some useful information in google. Can anyone tell how to fix it? Hi ckwei, It might be useful to paste some of the source code that isn't working for you. I don't have windows handy, but here's a NIF example that works on both Windows using command shell and SDK 7.1, and LInux, that Volker & I did last year. The erlang side NIF loader was poached from one of Paul Davis' projects IIRC. https://github.com/vmx/hellonif A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmacklin10@REDACTED Tue May 7 20:20:36 2013 From: danmacklin10@REDACTED (Dan Macklin) Date: Tue, 7 May 2013 19:20:36 +0100 Subject: [erlang-questions] [ANN:] erlang_cep a new Complex Event Processing Library In-Reply-To: <67C359DF-E334-4C8C-AB0C-1C6B197EA191@gmail.com> References: <67C359DF-E334-4C8C-AB0C-1C6B197EA191@gmail.com> Message-ID: Hi Tim, I've only done a couple of stress tests on a single core VM running on my home laptop. For a very simple row filter I was able to do 10,000 transactions in 5.3 seconds. If you are able to build the code using rebar you can run the same test as follows: stress_test:generate_test_data(). stress_test:run_match_stress(). Please be aware though this was a very simple use case. The feed_api should be fairly easy to use, but let me know if it isn't! I look forward to hearing how you get on if / when you get the time to give it a try. Best Regards Dan On Tue, May 7, 2013 at 9:50 AM, Tim Watson wrote: > Got any benchmarks? If/when I find time (and that's a fairly big if/when), > I'd like to hook up RabbitMQ as an input source. Does the feeds API make > that easy (ish) to do? > > Cheers > Tim > > On 6 May 2013, at 19:41, Dan Macklin wrote: > > Hello, > > I'd like to announce the first beta release of erlang_cep. > > erlang_cep is a basic CEP (Complex Event Processing) library that is > inspired by esper and couchdb. It allows users to create size or time > based windows, and then use erlang_js from Basho to write CEP functions. > > You can find the project at the following link :- > > https://github.com/danmacklin/erlang_cep > > I've written a fair amount of documentation and there are some simple > examples checked into the project. > > I'm quite new to Erlang so any comments or suggestions would be gratefully > received. > > Many thanks > > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue May 7 20:41:42 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 7 May 2013 19:41:42 +0100 Subject: [erlang-questions] Conflict issue Message-ID: Hi guys, So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. Thanks, Lee From arthurbeall@REDACTED Tue May 7 21:23:33 2013 From: arthurbeall@REDACTED (Art Beall) Date: Tue, 7 May 2013 12:23:33 -0700 (PDT) Subject: [erlang-questions] re. nif problem in windows Message-ID: <1367954613.49369.YahooMailNeo@web160502.mail.bf1.yahoo.com> Hi ckwei, We're working on a NIF project now in Windows. I tried various combinations of the NIF dll path, current directory, self-relative to two levels up, self relative to current (./), and full explicit path. They all work in our situation. ? Couple of suggestions: ? 1) Make sure you know where the erl's current directory actually is. If you don't directly set it, try setting it to the correct location explicitly. 2) Since it works on Linux, this may be mute, but check that each NIF function matches in name and arity in the windows version of code. ? Cheers, Art -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue May 7 21:37:34 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 7 May 2013 20:37:34 +0100 Subject: [erlang-questions] Conflict issue References: Message-ID: <4B73A6E1-A0D9-4BB6-A7A6-EA6E6D1EABF8@gmail.com> It seems I've run into the problem again, with both RabbitMQ and CloudI both using pqueue. This seems like a problem that should plague a lot of people. Isn't there a preferred way of dealing with this? Thanks, Lee On 7 May 2013, at 19:49, Alex Shneyderman wrote: > the safest thing to do is to use jsx for json in your own > applications. Obviously you will be back to square one if two of your > deps will depend on mochijson2 but you did not state that this is your > current problem, so switching to jsx (which is actually a lot more > enjoyable interface-wise) will take you a long way. > > another solution is to rename mochijson2 to say something > leesylvester_mochijson2 in your application. > > cheers. > > On Tue, May 7, 2013 at 2:41 PM, Lee Sylvester wrote: >> Hi guys, >> >> So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. >> >> Thanks, >> Lee >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue May 7 21:40:47 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 07 May 2013 12:40:47 -0700 Subject: [erlang-questions] Conflict issue In-Reply-To: References: Message-ID: <518958BF.2050402@gmail.com> Hi Lee, Yes, unfortunately this is a common problem with Erlang/OTP development. The normal way people deal with it, is to manually rename each module within a conflicting application, then rename the application, so that the conflict is avoided. There is no namespace feature that is part of Erlang/OTP release generation that automatically would resolve conflicting applications that are possibly different versions. I will look at making the database dependencies optional (probably with manual application start calls) so that future versions of CloudI can avoid the problems within Erlang/OTP release generation. With the 1.2.1 release of CloudI, I think you will need to remove the ecouchdb dependency from the cloudi_services_databases application (here https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi_services_databases/src/cloudi_services_databases.app.src.in), remove it from the release (here https://github.com/okeuday/CloudI/blob/master/src/reltool.config.in), and then reinstall (rm -rf /usr/local/lib/cloudi-1.2.1 and then do the install). Best Regards, Michael On 05/07/2013 11:41 AM, Lee Sylvester wrote: > Hi guys, > > So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. > > Thanks, > Lee > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lee.sylvester@REDACTED Tue May 7 21:55:51 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 7 May 2013 20:55:51 +0100 Subject: [erlang-questions] Conflict issue In-Reply-To: <518958BF.2050402@gmail.com> References: <518958BF.2050402@gmail.com> Message-ID: <8E50883A-F21E-41EE-915D-3DD9BA023C1A@gmail.com> Thanks, Michael, that got rid of the ecouchdb issue, but the pqueue issue will be a lot harder to get rid of. I'm loathe to rename it for CloudI as it will be a royal pain to keep doing so every time I update the CloudI dependency. I think I may need to rethink my approach :-) Regards, Lee On 7 May 2013, at 20:40, Michael Truog wrote: > Hi Lee, > > Yes, unfortunately this is a common problem with Erlang/OTP development. The normal way people deal with it, is to manually rename each module within a conflicting application, then rename the application, so that the conflict is avoided. There is no namespace feature that is part of Erlang/OTP release generation that automatically would resolve conflicting applications that are possibly different versions. I will look at making the database dependencies optional (probably with manual application start calls) so that future versions of CloudI can avoid the problems within Erlang/OTP release generation. With the 1.2.1 release of CloudI, I think you will need to remove the ecouchdb dependency from the cloudi_services_databases application (here https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi_services_databases/src/cloudi_services_databases.app.src.in), remove it from the release (here https://github.com/okeuday/CloudI/blob/master/src/reltool.config.in), and > then reinstall (rm -rf /usr/local/lib/cloudi-1.2.1 and then do the install). > > Best Regards, > Michael > > On 05/07/2013 11:41 AM, Lee Sylvester wrote: >> Hi guys, >> >> So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. >> >> Thanks, >> Lee >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > From mjtruog@REDACTED Tue May 7 22:04:21 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 07 May 2013 13:04:21 -0700 Subject: [erlang-questions] Conflict issue In-Reply-To: <4B73A6E1-A0D9-4BB6-A7A6-EA6E6D1EABF8@gmail.com> References: <4B73A6E1-A0D9-4BB6-A7A6-EA6E6D1EABF8@gmail.com> Message-ID: <51895E45.7020506@gmail.com> The CloudI release should be modified to strip the application of unused modules, so that would help minimize the problem. I will also remove the priority_queue.erl module from the pqueue application, since that should be the only module conflict with RabbitMQ. Otherwise, you just end up renaming everything. Best Regards, Michael On 05/07/2013 12:37 PM, Lee Sylvester wrote: > > It seems I've run into the problem again, with both RabbitMQ and CloudI both using pqueue. This seems like a problem that should plague a lot of people. Isn't there a preferred way of dealing with this? > > Thanks, > Lee > > > > On 7 May 2013, at 19:49, Alex Shneyderman > wrote: > > >> the safest thing to do is to use jsx for json in your own >> applications. Obviously you will be back to square one if two of your >> deps will depend on mochijson2 but you did not state that this is your >> current problem, so switching to jsx (which is actually a lot more >> enjoyable interface-wise) will take you a long way. >> >> another solution is to rename mochijson2 to say something >> leesylvester_mochijson2 in your application. >> >> cheers. >> >> On Tue, May 7, 2013 at 2:41 PM, Lee Sylvester > wrote: >>> Hi guys, >>> >>> So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. >>> >>> Thanks, >>> Lee >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue May 7 23:01:44 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 7 May 2013 22:01:44 +0100 Subject: [erlang-questions] Conflict issue In-Reply-To: <51895E45.7020506@gmail.com> References: <4B73A6E1-A0D9-4BB6-A7A6-EA6E6D1EABF8@gmail.com> <51895E45.7020506@gmail.com> Message-ID: Hi Michael, I've removed priority_queue, which fixed the issue. The app now compiles, but when run, I get an error at line 78 of cloudi_app.erl, with the call to get_env. Can you tell me where the configuration value in that call is pulled from CloudI, so I can ensure the data is passed? Thanks, Lee On 7 May 2013, at 21:04, Michael Truog wrote: > The CloudI release should be modified to strip the application of unused modules, so that would help minimize the problem. I will also remove the priority_queue.erl module from the pqueue application, since that should be the only module conflict with RabbitMQ. Otherwise, you just end up renaming everything. > > Best Regards, > Michael > > On 05/07/2013 12:37 PM, Lee Sylvester wrote: >> >> It seems I've run into the problem again, with both RabbitMQ and CloudI both using pqueue. This seems like a problem that should plague a lot of people. Isn't there a preferred way of dealing with this? >> >> Thanks, >> Lee >> >> >> >> On 7 May 2013, at 19:49, Alex Shneyderman wrote: >> >> >>> the safest thing to do is to use jsx for json in your own >>> applications. Obviously you will be back to square one if two of your >>> deps will depend on mochijson2 but you did not state that this is your >>> current problem, so switching to jsx (which is actually a lot more >>> enjoyable interface-wise) will take you a long way. >>> >>> another solution is to rename mochijson2 to say something >>> leesylvester_mochijson2 in your application. >>> >>> cheers. >>> >>> On Tue, May 7, 2013 at 2:41 PM, Lee Sylvester wrote: >>>> Hi guys, >>>> >>>> So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue May 7 23:21:44 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 07 May 2013 14:21:44 -0700 Subject: [erlang-questions] Conflict issue In-Reply-To: References: <4B73A6E1-A0D9-4BB6-A7A6-EA6E6D1EABF8@gmail.com> <51895E45.7020506@gmail.com> Message-ID: <51897068.4060803@gmail.com> It is trying to load the file app.config which is installed into /usr/local/etc/cloudi/app.config (if the configured prefix is /usr/local/), which is generated from https://github.com/okeuday/CloudI/blob/master/src/rel/files/app.config.in . Within that file, you should see the only value used for the cloudi_core application is the configuration entry which provides the path to the CloudI configuration (installed at /usr/local/etc/cloudi/cloudi.conf), however, the other values help provide some appropriate defaults for a few other applications. The error you received means that it couldn't find the configuration file path for the cloudi_core application, so adding that entry should fix the problem. Best Regards, Michael On 05/07/2013 02:01 PM, Lee Sylvester wrote: > Hi Michael, > > I've removed priority_queue, which fixed the issue. The app now compiles, but when run, I get an error at line 78 of cloudi_app.erl, with the call to get_env. Can you tell me where the configuration value in that call is pulled from CloudI, so I can ensure the data is passed? > > Thanks, > Lee > > > On 7 May 2013, at 21:04, Michael Truog > wrote: > >> The CloudI release should be modified to strip the application of unused modules, so that would help minimize the problem. I will also remove the priority_queue.erl module from the pqueue application, since that should be the only module conflict with RabbitMQ. Otherwise, you just end up renaming everything. >> >> Best Regards, >> Michael >> >> On 05/07/2013 12:37 PM, Lee Sylvester wrote: >>> >>> It seems I've run into the problem again, with both RabbitMQ and CloudI both using pqueue. This seems like a problem that should plague a lot of people. Isn't there a preferred way of dealing with this? >>> >>> Thanks, >>> Lee >>> >>> >>> >>> On 7 May 2013, at 19:49, Alex Shneyderman > wrote: >>> >>> >>>> the safest thing to do is to use jsx for json in your own >>>> applications. Obviously you will be back to square one if two of your >>>> deps will depend on mochijson2 but you did not state that this is your >>>> current problem, so switching to jsx (which is actually a lot more >>>> enjoyable interface-wise) will take you a long way. >>>> >>>> another solution is to rename mochijson2 to say something >>>> leesylvester_mochijson2 in your application. >>>> >>>> cheers. >>>> >>>> On Tue, May 7, 2013 at 2:41 PM, Lee Sylvester > wrote: >>>>> Hi guys, >>>>> >>>>> So, I'm compiling CloudI into my app (safest option), but I'm getting a conflict when I compile. The app I have using Mochijson2, but so does ecouchdb which is part of CloudI. Thus, when I compile my app (well, actually, it's when I generate a release), I now get the error "{'EXIT',{{badmatch,{error,"Module mochijson2 potentially included by two different applications: ecouchdb and mochijson2."}},". I've removed the mochijson2.erl file in the ecouchdb app, hoping that would force the compiler to make ecouchdb use the app version of Mochijson2, but that did nothing. Is there something I can add to my reltool.config to fix this? I've Googled, but nothing that made sense came up. >>>>> >>>>> Thanks, >>>>> Lee >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangprogram@REDACTED Wed May 8 00:19:42 2013 From: erlangprogram@REDACTED (S X) Date: Tue, 7 May 2013 18:19:42 -0400 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Hi Anders, Thanks for your quick responses and detailed comments. I am learning erlang during my spare time and sometimes progress slowly since no enough time to engage into it. Yes, like you mentioned, I started the diameter client configuration (capability exchange) in a separate process and wait for peer up callback (which I think should be same as diameter:subscribe, though I haven't try it yet). Meanwhile I started workers which wait for notifications from the peer up callback via ets table approach to proceed with sending diameter messages. The problem is that if there is a lot of workers, after peer up notifications being received, some of them don't receive diameter answer from the server, instead, they still get no_connection errors from diameter_traffic:send function, and others can get proper diameter answers. Sometimes, all the workers can get the answers after sending requests upon getting notifications from the sole peer up callback. The confusing part is that once the client receives the peer up, it should mean the server is ready to process diameter requests, but why still get no_connection errors. As you mentioned in the previous emails "If you call before the relevant peer_up callback then the result is {error, no_connection}." The other question is not diameter related. It is about the erlang BIF function register. I tried to register every process in a loop. However I found through debugger or using whereis function that not all process names can be registered properly. As I check the erlang document, it says register(Name, Pid) -> true. It supposes to be always successful unless the process name exists already then get the badarg exception. util:foreach_index( fun(N) -> ProcessName = list_to_atom(atom_to_list(test) ++ integer_to_list(N)), register(ProcessName, spawn(executer, execute, [])) end, MyList) Do you have any suggestions why register could fail without any warning/error reported? Thanks! Samuel On Tue, Apr 30, 2013 at 5:04 AM, Anders Svensson wrote: > Hi Samuel. > > On Tue, Apr 30, 2013 at 4:35 AM, S X wrote: > > Hi Anders, > > > > Thanks for your explanation. Setting additional options in > > diameter:add_transport and retrieving them with diameter:service_info > from > > the peer_up callback work well. And the descriptions about the peer > states > > according to RFC3539 and 6733 make a lot of sense. > > > > The case I am trying to resolve is: > > Start multiple processes on one pc, each process tries to connect to one > > same server (obviously I will get already_connected warning, but it can > keep > > going), in each process, then I try to send a diameter message to the > > server, I used to insert a certain amount of sleep time before I send the > > message after connecting to the server and it works ok. But I wanted to > use > > the peer up callback to behave like a handshaking procedure among the > client > > and the server, so the server is supposed to be ready when the client > > process sends the message. > > > > Now, the fact is that only one peer up callback is invoked for multiple > > processes. Then I tried to notify all the processes in the peer up > callback > > when invoked and the processes send out the diameter messages. However, > the > > processes don't receive any responses. Or sometimes only part of the > > processes get responses. Why inserting an amount of sleep time always > works. > > > > This is a little bit confusing me, how peer up should be used properly to > > get the notification done right for multiple processes. > > It's difficult to say what your problem is without seeing code but > here are a few points. > > It's not your client processes that connect to the server. A transport > connection results from someone calling diameter:add_transport/2. Once > diameter establishes the connection, successfully performs > capabilities exchange, and makes peer_up callbacks, that transport > connection is available to any process that wants to send to the peer > in question. In your case, you have multiple processes that want to do > so, but if you're calling diameter:add_transport/2 with the same > config from each of those processes then you're effectively telling > diameter to establish multiple transport connections, only one of > which will succeed by default. If you want to allow multiple transport > connections per peer then you need to configure your service with > {restrict_connections, false}, but the peer need not support this (see > 2.1 in RFC 6733) so may reject all but one CER. Without > restrict_connections it's diameter that disallows multiple > connections. > > Given that you want multiple client processes to send from (not wrong, > but why?), I would treat these as workers that have nothing to do with > configuration. That is, call diameter:start_service/2 and > diameter:add_transport/2 from elsewhere and have your workers kick > into action after capabilities exchange. > > Like you said, your workers want to know when the peer becomes > available. There are two ways to get notification: a peer_up callback > or a diameter event. The latter are subscribed to with > diameter:subscribe/1 so one way is for each worker to subscribe to > events and react to them. If you want to use peer_up as a trigger then > you need to code some way for the callback to notify your worker > processes. There are plenty of ways to do this, none of which are > diameter-specific. For example: maintain your worker pids in an ets > table that the callback examines; have workers register with a > registered gen_server that finds out about callbacks; spawn workers as > a result of peer_up. Sleep is never a reliable solution. > > Anders > > > > > > Thanks for any suggestions! > > > > Samuel > > > > > > > > > > On Thu, Apr 25, 2013 at 5:40 AM, Anders Svensson > > wrote: > >> > >> On Wed, Apr 24, 2013 at 5:23 PM, S X wrote: > >> > Thanks a lot for your comments. > >> > > >> > The diameter:call function allows to send extra arguments. It works > fine > >> > after changing the callback function signatures accordingly. This is > >> > very > >> > useful when I want to notify the other process to do the next thing. > >> > > >> > Before sending a diameter message, we call diameter:add_transport to > >> > connect > >> > to a peer and peform CER/CEA capability information exchange. On the > >> > caller > >> > side, let's say the client callback module, the peer_up will be > invoked > >> > when > >> > CER/CEA is completed. Why it doesn't have the similar mechanism like > >> > diameter:call to allow insert additional arguments so we can utilize > >> > them, > >> > for example, notify the others to send diameter messages? > >> > >> No particular reason aside from history and that the need hasn't come > >> up. As it is, you can pass arbitrary options to > >> diameter:add_transport/2 (history again) and retrieve these in a > >> callback with diameter:service_info(PeerRef), so that can be used a > >> substitute for extra arguments in this case. > >> > >> > diameter:add_transport is a sync call, but it doesn't mean you can > send > >> > >> add_transport *doesn't* wait for the config it's passed to result in a > >> connection before returning. (It might never happen for one.) > >> > >> > diameter messages successfully when the function returns, i.e. it > >> > usually > >> > gets {error, no_connection} or { error, timeout }(this might because > of > >> > the > >> > server side) if you call diameter:call right after > >> > diameter:add_transport. > >> > >> This is because the peer won't be ready to respond to requests until > >> capabilities exchange has completed (at least). If you call before the > >> relevant peer_up callback then the result is {error, no_connection}. > >> After peer_up the result will be {error, timeout} if the peer doesn't > >> answer. One case in which this is expected is after a connection has > >> been reestablished following an exchange of 3 x DWR/DWA. (ie. RFC 3539 > >> DOWN -> REOPEN -> OKAY.) Since both ends of the connection do this, > >> the client can consider the connection to be reestablished before the > >> server. If the client sends a request before the server is done with > >> it's exchange (ie. reached OKAY) then RFC 6733 says it should discard > >> the request, resulting in a timeout on the client end. > >> > >> > So is there any reason why not allowing to add additional arguments > and > >> > use > >> > them in peer_up callback function? Since it means capability exchange > is > >> > done and the peer is ready, at this point it should be safe to send > >> > diameter > >> > messages to the peer. > >> > >> See above. > >> > >> Anders > >> > >> > >> > > >> > Any suggestions? > >> > > >> > Thanks! > >> > > >> > Samuel > >> > > >> > > >> > On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson < > anders.otp@REDACTED> > >> > wrote: > >> >> > >> >> On Sun, Apr 14, 2013 at 7:01 PM, S X > wrote: > >> >> > Hello, > >> >> > > >> >> > Based on the erlang diameter library and the sample code, I want to > >> >> > start > >> >> > multiple diameter client processes in one erlang node(one client > IP), > >> >> > and > >> >> > the client needs to define a diameter_app callback module for > certain > >> >> > application, for example: > >> >> > > >> >> > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com > "}, > >> >> > {'Origin-Realm', "example.com"}, > >> >> > {'Vendor-Id', 193}, > >> >> > {'Product-Name', "Client"}, > >> >> > {'Auth-Application-Id', > >> >> > [?DIAMETER_APP_ID_COMMON, > >> >> > ?DIAMETER_APP_ID_CCRA]}, > >> >> > {application, [{alias, ?APP_CCR_ALIAS}, > >> >> > {dictionary, > >> >> > ?DIAMETER_DICT_CCRA}, > >> >> > {module, > client_cb_ccra}]}]). > >> >> > > >> >> > > >> >> > First question: > >> >> > how the diameter library handles this situation? Will all diameter > >> >> > client > >> >> > processes share one single diameter_app callback module > >> >> > "client_cb_ccra" > >> >> > or > >> >> > it will automatically attach different instance of the callback > >> >> > module > >> >> > (process) to the different client process by using spawn_monitor? > So > >> >> > from > >> >> > the callback handle_answer in "client_cb_ccra" I can notify the > >> >> > proper > >> >> > client proce by just calling client:notify() something. > >> >> > >> >> diameter doesn't know anything about your client process. If you want > >> >> a callback to be able to contact the client process associated with > >> >> the service in question (ie. the service whose name the callback gets > >> >> as an argument) then you need to give the callback the means to do > so. > >> >> One way would be to map the service name to your process (eg. it's > the > >> >> registered name of your client process), another would be to pass > some > >> >> identification as extra arguments to the callbacks. (Eg. {module, > >> >> [client_cb_ccra, X]} in the config above.) > >> >> > >> >> > Second question: > >> >> > Notice that diameter:call allows to set extra arguments, so I was > >> >> > wondering > >> >> > I could set some data like: > >> >> > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). > >> >> > which sets the client process ID and I hope to deliver to the > >> >> > callback > >> >> > module "client_cb_ccra" and when the callback module knows which > >> >> > client > >> >> > process sends request and response accordingly, for example using > the > >> >> > client > >> >> > process ID in the callback function handle_answer. > >> >> > >> >> You can do that, but what is it you're trying to accomplish? if it's > >> >> handle_answer that's supposed to communicate something then it > already > >> >> does: the return value of handle_answer is returned by diameter:call. > >> >> > >> >> > However, I don't want pack the extra arguments into the diameter > >> >> > packet > >> >> > since the diameter server doesn't know what they are and the extra > >> >> > ones > >> >> > are > >> >> > not part of standard diameter packet. > >> >> > >> >> Not sure what you mean here. There's no way for your client to send > >> >> the server anything other than a Diameter message. > >> >> > >> >> > Now I changed the pick_peer callback signatures to allow extra > >> >> > arguments, > >> >> > pick_peer([Peer | _], _, _SvcName, _State, A) > >> >> > > >> >> > But I got encoding error in the callback prepare_request > >> >> > > >> >> > =ERROR REPORT==== 14-Apr-2013::11:40:41 === > >> >> > Error in process <0.175.0> with exit value: > >> >> > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, > >> >> > >> >> The arity of your callback probably doesn't agree with the number of > >> >> extra arguments you've specified: prepare_request will also get your > >> >> extra arguments. > >> >> > >> >> > >> >> > > >> >> > > > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... > >> >> > > >> >> > {error,encode} > >> >> > > >> >> > In overall, I read the online documents, which have limited > >> >> > information > >> >> > on > >> >> > how to use the extra arguments in the library and don't quite get > >> >> > how > >> >> > to > >> >> > utilize the extra arguments to do something tricky, > >> >> > > >> >> > Are there any suggestions on how to deal with multiple client > >> >> > processes? > >> >> > >> >> Not sure I understand what problem it is you're trying to solve. > >> >> Multiple processes invoking diameter:call is nothing strange. You > >> >> typically just return something useful (eg. the answer message from > >> >> the peer) from handle_answer, which diameter:call then returns for > the > >> >> caller to deal with. > >> >> > >> >> Anders > >> >> > >> >> > >> >> > Thanks a lot! > >> >> > > >> >> > Samuel > >> >> > > >> > > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wckbluesky@REDACTED Wed May 8 04:11:39 2013 From: wckbluesky@REDACTED (=?utf-8?B?6Z+m5bSH54Sc?=) Date: Wed, 8 May 2013 10:11:39 +0800 Subject: [erlang-questions] nif problem in windows References: <201305071800568105382@yahoo.com.cn> , <201305072115176044184@yahoo.com.cn>, <001e01ce4b45$273e6430$75bb2c90$@gmail.com> Message-ID: <201305081011389629782@yahoo.com.cn> hi, Janos I think the path is right, I also try to change the parameter "PathName" in erlang:load_nif(PathName, ....) to \\priv\\hello, ./priv/hello, but none of them works. And the file hello.dll does exists in path "./priv", if I delete it from this path, hello:init(). will return error message, : "can not find the specific module", so I definitely sure that erlang has read the hello.dll successfully, but the file format is error. I was thinking that, whether I chose the wrong compile parameter when compile the hello.dll, but in the other hand, my computer reports the same error when I run the same code Dave gave to me, and he said it is works fine to him, I have no idead now. From: Janos Hary Date: 2013-05-08 01:06 To: 'wckbluesky' CC: 'erlang-questions' Subject: RE: [erlang-questions] nif problem in windows Your error message says ?priv/hello?. Check the path in your erlang:load_nif. Janos From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of ??? Sent: Tuesday, May 07, 2013 3:15 PM To: Dave Cottlehuber Cc: erlang-questions Subject: Re: [erlang-questions] nif problem in windows hi, Dave, Thanks for you replying, I try your example and it is return the same error I wrote before, here is the output: E:\firefox_download\hellonif-master>rebar.cmd compile ==> hellonif-master (compile) Compiled src/hello.erl Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. hello.c D:\Program Files (x86)\Microsoft Visual Studio 8\VC\INCLUDE\stdlib.h(215) : warn ing C4255: '_get_purecall_handler' : no function prototype given: converting '() ' to '(void)' d:\PROGRA~2\ERL510~1.1\erts-5.10.1\include\erl_nif.h(132) : warning C4820: '' : '4' bytes padding added after data member 'ref_bin' c_src/hello.c(3) : warning C4100: 'argv' : unreferenced formal parameter c_src/hello.c(3) : warning C4100: 'argc' : unreferenced formal parameter c_src/hello.c(13) : warning C4013: 'memcpy' undefined; assuming extern returning int Microsoft (R) Incremental Linker Version 8.00.50727.42 Copyright (C) Microsoft Corporation. All rights reserved. ????? priv/hello.lib ??? priv/hello.exp E:\firefox_download\hellonif-master>erl -pa ebin Eshell V5.10.1 (abort with ^G) 1> hello:init(). {error,{load_failed,"Failed to load NIF library priv/hello: 'Unspecified error'" }} 2> hello:world(). "NIF library not loaded" 3> Notice that, I change the code hello.erl slightly, it catch all the error when erlang:load_nif, I delete it, so it can output the error message I using operating system win7, is it matters? ckwei From: Dave Cottlehuber Date: 2013-05-07 19:37 To: erlang-questions Subject: Re: [erlang-questions] nif problem in windows On 7 May 2013 12:00, ??? wrote: hi I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows Everything is ok before I running "c(complex6)" in erlang VM, in windows, it reports =ERROR REPORT==== 7-May-2013::17:36:19 === Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi le,"code_server.erl"},... =ERROR REPORT==== 7-May-2013::17:36:19 === The on_load function for module complex6 returned {{badmatch, {error, {load_failed, "Failed to load NIF library ./complex6_nif: 'Unspecified error'"}}}, [{complex6,init,0, [{file,...},{...}]}, {code_server, '-handle_on_load/4-fun-0-', 1, [{...}|...]}]} {error,on_load_failure} It is hard to find some useful information in google. Can anyone tell how to fix it? Hi ckwei, It might be useful to paste some of the source code that isn't working for you. I don't have windows handy, but here's a NIF example that works on both Windows using command shell and SDK 7.1, and LInux, that Volker & I did last year. The erlang side NIF loader was poached from one of Paul Davis' projects IIRC. https://github.com/vmx/hellonif A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From wckbluesky@REDACTED Wed May 8 04:31:16 2013 From: wckbluesky@REDACTED (=?gb2312?B?zqSz559q?=) Date: Wed, 8 May 2013 10:31:16 +0800 Subject: [erlang-questions] =?gb2312?b?u9i4tDogUmU6ICBuaWYgcHJvYmxlbSBp?= =?gb2312?b?biB3aW5kb3dz?= References: <201305071800568105382@yahoo.com.cn> <201305072115176044184@yahoo.com.cn>, Message-ID: <201305081031164382324@yahoo.com.cn> hi, Dave and all friends reply my e-mail, I have figure out this problem. My operating system is win7 64bits, so I use Visual Studio Tools "Visual Studio X64 Command Line" in vs2010, Run the same code, the same compile command, and it is works finally! I think the 64bits operating system is the key reason which cause this error, but I really don't know why. ckwei ???????? Dave Cottlehuber ?????????? 2013-05-07 21:31 ???????? wckbluesky; erlang-questions ?????? Re: Re: [erlang-questions] nif problem in windows On 7 May 2013 15:15, ?????j wrote: hi, Dave, Thanks for you replying, I try your example and it is return the same error I wrote before, here is the output: E:\firefox_download\hellonif-master>rebar.cmd compile ==> hellonif-master (compile) Compiled src/hello.erl Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. This is SDK 7.1 IIRC. That's good. hello.c D:\Program Files (x86)\Microsoft Visual Studio 8\VC\INCLUDE\stdlib.h(215) : warn I think you have an SDK that doesn't match the header files you're providing it here. No idea if that is supposed to work, but it smells bad to me. ing C4255: '_get_purecall_handler' : no function prototype given: converting '() ' to '(void)' d:\PROGRA~2\ERL510~1.1\erts-5.10.1\include\erl_nif.h(132) : warning C4820: '' : '4' bytes padding added after data member 'ref_bin' c_src/hello.c(3) : warning C4100: 'argv' : unreferenced formal parameter c_src/hello.c(3) : warning C4100: 'argc' : unreferenced formal parameter c_src/hello.c(13) : warning C4013: 'memcpy' undefined; assuming extern returning int Microsoft (R) Incremental Linker Version 8.00.50727.42 Copyright (C) Microsoft Corporation. All rights reserved. This linker doesn't match the SDK. ?????????? priv/hello.lib ?????? priv/hello.exp E:\firefox_download\hellonif-master>erl -pa ebin Eshell V5.10.1 (abort with ^G) 1> hello:init(). {error,{load_failed,"Failed to load NIF library priv/hello: 'Unspecified error'" }} 2> hello:world(). "NIF library not loaded" 3> Notice that, I change the code hello.erl slightly, it catch all the error when erlang:load_nif, I delete it, so it can output the error message I using operating system win7, is it matters? OS shouldn't matter, no. But the mismatch of compiler + linker + header files likely does matter. Can you try this with a clean environment (path set to minimum windows specific ones, and libs, include variables set to empty) and then run setsdk.cmd /x86 /release before adding erlang headers and path? That should be enough to keep a clean compile using only the SDK components, and skipping Visual C++ 8. ckwei From: Dave Cottlehuber Date: 2013-05-07 19:37 To: erlang-questions Subject: Re: [erlang-questions] nif problem in windows On 7 May 2013 12:00, ?????j wrote: hi I am testing nif in windows right now, the nif c code is very simple and it is copyed directly from http://www.erlang.org/doc/tutorial/nif.html#id64988 All the code is running well in Linux, and c code is compiled to .so in linux, .dll in windows Everything is ok before I running "c(complex6)" in erlang VM, in windows, it reports =ERROR REPORT==== 7-May-2013::17:36:19 === Error in process <0.70.0> with exit value: {{badmatch,{error,{load_failed,"Faile d to load NIF library ./complex6_nif: 'Unspecified error'"}}},[{complex6,init,0, [{file,"complex6.erl"},{line,7}]},{code_server,'-handle_on_load/4-fun-0-',1,[{fi le,"code_server.erl"},... =ERROR REPORT==== 7-May-2013::17:36:19 === The on_load function for module complex6 returned {{badmatch, {error, {load_failed, "Failed to load NIF library ./complex6_nif: 'Unspecified error'"}}}, [{complex6,init,0, [{file,...},{...}]}, {code_server, '-handle_on_load/4-fun-0-', 1, [{...}|...]}]} {error,on_load_failure} It is hard to find some useful information in google. Can anyone tell how to fix it? Hi ckwei, It might be useful to paste some of the source code that isn't working for you. I don't have windows handy, but here's a NIF example that works on both Windows using command shell and SDK 7.1, and LInux, that Volker & I did last year. The erlang side NIF loader was poached from one of Paul Davis' projects IIRC. https://github.com/vmx/hellonif A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Wed May 8 08:54:12 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 8 May 2013 08:54:12 +0200 Subject: [erlang-questions] erlexec and its environment In-Reply-To: <51892EFD.7090106@aleynikov.org> References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> <51892EFD.7090106@aleynikov.org> Message-ID: <1367996052.4860.3.camel@sekic1152.rnd.ki.sw.ericsson.se> Please document which one of these is used else where in Erlang/OTP, and that the other is not. bengt On Tue, 2013-05-07 at 12:42 -0400, Serge Aleynikov wrote: > The master branch of erlexec now has support for both {Name, Val} and > "VAR=VALUE" environment options. > > On 5/7/2013 4:55 AM, Bengt Kleberg wrote: > > Greetings, > > > > There are some Erlang functions that have {env, [...]} argument. Ex: > > erlang:open_port/2. > > > > I think it would be good to have the same syntax ("a list of tuples > > {Name, Val}, where Name is the name of an environment variable, and Val > > is the value it is to have in the spawned port process." and the same > > semantics for exec:run_link/2's {env, [...]}. > > > > > > bengt > > > > On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: > >> Hi, > >> > >> We're doing some work to fix up a few things we perceive as > >> deficiencies in erlexec, and thought about asking for a community > >> opinion. > >> > >> Currently, this: > >> > >> exec:run_link("exec /tmp/print_env.sh", []) > >> > >> Launches the child process with no environment, which was quite a > >> surprise to us - the default on Unix systems is to have child > >> processes inherit the parent process' environment. We're hacking the > >> system so that it'll behave like we expect. > >> > >> When an environment is specified, we were thinking to use that to say > >> "ok, clear out the environment and don't inherit it": > >> > >> exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) > >> > >> So that if you really needed to pass it in, you could do: > >> > >> exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) > >> > >> Thoughts? > >> -- > >> David N. Welton > >> > >> http://www.welton.it/davidw/ > >> > >> http://www.dedasys.com/ > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anders.otp@REDACTED Wed May 8 08:56:10 2013 From: anders.otp@REDACTED (Anders Svensson) Date: Wed, 8 May 2013 08:56:10 +0200 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Hi Samuel. On Wed, May 8, 2013 at 12:19 AM, S X wrote: > Hi Anders, > > Thanks for your quick responses and detailed comments. I am learning erlang > during my spare time and sometimes progress slowly since no enough time to > engage into it. > > Yes, like you mentioned, I started the diameter client configuration > (capability exchange) in a separate process and wait for peer up callback > (which I think should be same as diameter:subscribe, though I haven't try it There is one significant difference between peer_up and subscribe: there's a callback for each Diameter application that capabilities exchange agrees upon (the callback module being configured per Diameter application, while there's a single 'up' event. > yet). Meanwhile I started workers which wait for notifications from the peer > up callback via ets table approach to proceed with sending diameter > messages. The problem is that if there is a lot of workers, after peer up > notifications being received, some of them don't receive diameter answer > from the server, instead, they still get no_connection errors from > diameter_traffic:send function, and others can get proper diameter answers. Are you sure the transport connection hasn't gone down? If it does then whether or not a client is able to send will just be down to timing. > Sometimes, all the workers can get the answers after sending requests upon > getting notifications from the sole peer up callback. > > The confusing part is that once the client receives the peer up, it should > mean the server is ready to process diameter requests, but why still get > no_connection errors. As you mentioned in the previous emails "If you call > before the > > relevant peer_up callback then the result is {error, no_connection}." You'll get the same result if the peer goes down before you have a chance to send. That is, you may get a peer_up callback and try to send but if the connection is lost before you've managed to do so then {error, no_connection} will be the result. There'll be peer_down callbacks to note the loss of connectivity but this can be concurrent with your attempt to send. > The other question is not diameter related. It is about the erlang BIF > function register. I tried to register every process in a loop. However I > found through debugger or using whereis function that not all process names > can be registered properly. As I check the erlang document, it says If by "not registered properly" you mean that whereis(Name) returns undefined rather than the name you expect then it's probably just that your process has died. > register(Name, Pid) -> true. It supposes to be always successful unless the > process name exists already then get the badarg exception. > util:foreach_index( > fun(N) -> > ProcessName = list_to_atom(atom_to_list(test) ++ > integer_to_list(N)), > register(ProcessName, spawn(executer, execute, [])) > end, MyList) > > Do you have any suggestions why register could fail without any > warning/error reported? Like I said, I don't think it's that register is failing, it's just that your processes are dieing after registration, in which case their registrations are removed. erlang:register/2 fails if you try to register a non-existent process but successful registration doesn't mean that the process will still be alive (and registered) when you call erlang:whereis/1. Your code above will fail if any of your processes die before calling register. Better that each spawned process registers itself. Anders > Thanks! > > Samuel > > > > On Tue, Apr 30, 2013 at 5:04 AM, Anders Svensson > wrote: >> >> Hi Samuel. >> >> On Tue, Apr 30, 2013 at 4:35 AM, S X wrote: >> > Hi Anders, >> > >> > Thanks for your explanation. Setting additional options in >> > diameter:add_transport and retrieving them with diameter:service_info >> > from >> > the peer_up callback work well. And the descriptions about the peer >> > states >> > according to RFC3539 and 6733 make a lot of sense. >> > >> > The case I am trying to resolve is: >> > Start multiple processes on one pc, each process tries to connect to one >> > same server (obviously I will get already_connected warning, but it can >> > keep >> > going), in each process, then I try to send a diameter message to the >> > server, I used to insert a certain amount of sleep time before I send >> > the >> > message after connecting to the server and it works ok. But I wanted to >> > use >> > the peer up callback to behave like a handshaking procedure among the >> > client >> > and the server, so the server is supposed to be ready when the client >> > process sends the message. >> > >> > Now, the fact is that only one peer up callback is invoked for multiple >> > processes. Then I tried to notify all the processes in the peer up >> > callback >> > when invoked and the processes send out the diameter messages. However, >> > the >> > processes don't receive any responses. Or sometimes only part of the >> > processes get responses. Why inserting an amount of sleep time always >> > works. >> > >> > This is a little bit confusing me, how peer up should be used properly >> > to >> > get the notification done right for multiple processes. >> >> It's difficult to say what your problem is without seeing code but >> here are a few points. >> >> It's not your client processes that connect to the server. A transport >> connection results from someone calling diameter:add_transport/2. Once >> diameter establishes the connection, successfully performs >> capabilities exchange, and makes peer_up callbacks, that transport >> connection is available to any process that wants to send to the peer >> in question. In your case, you have multiple processes that want to do >> so, but if you're calling diameter:add_transport/2 with the same >> config from each of those processes then you're effectively telling >> diameter to establish multiple transport connections, only one of >> which will succeed by default. If you want to allow multiple transport >> connections per peer then you need to configure your service with >> {restrict_connections, false}, but the peer need not support this (see >> 2.1 in RFC 6733) so may reject all but one CER. Without >> restrict_connections it's diameter that disallows multiple >> connections. >> >> Given that you want multiple client processes to send from (not wrong, >> but why?), I would treat these as workers that have nothing to do with >> configuration. That is, call diameter:start_service/2 and >> diameter:add_transport/2 from elsewhere and have your workers kick >> into action after capabilities exchange. >> >> Like you said, your workers want to know when the peer becomes >> available. There are two ways to get notification: a peer_up callback >> or a diameter event. The latter are subscribed to with >> diameter:subscribe/1 so one way is for each worker to subscribe to >> events and react to them. If you want to use peer_up as a trigger then >> you need to code some way for the callback to notify your worker >> processes. There are plenty of ways to do this, none of which are >> diameter-specific. For example: maintain your worker pids in an ets >> table that the callback examines; have workers register with a >> registered gen_server that finds out about callbacks; spawn workers as >> a result of peer_up. Sleep is never a reliable solution. >> >> Anders >> >> >> > >> > Thanks for any suggestions! >> > >> > Samuel >> > >> > >> > >> > >> > On Thu, Apr 25, 2013 at 5:40 AM, Anders Svensson >> > wrote: >> >> >> >> On Wed, Apr 24, 2013 at 5:23 PM, S X wrote: >> >> > Thanks a lot for your comments. >> >> > >> >> > The diameter:call function allows to send extra arguments. It works >> >> > fine >> >> > after changing the callback function signatures accordingly. This is >> >> > very >> >> > useful when I want to notify the other process to do the next thing. >> >> > >> >> > Before sending a diameter message, we call diameter:add_transport to >> >> > connect >> >> > to a peer and peform CER/CEA capability information exchange. On the >> >> > caller >> >> > side, let's say the client callback module, the peer_up will be >> >> > invoked >> >> > when >> >> > CER/CEA is completed. Why it doesn't have the similar mechanism like >> >> > diameter:call to allow insert additional arguments so we can utilize >> >> > them, >> >> > for example, notify the others to send diameter messages? >> >> >> >> No particular reason aside from history and that the need hasn't come >> >> up. As it is, you can pass arbitrary options to >> >> diameter:add_transport/2 (history again) and retrieve these in a >> >> callback with diameter:service_info(PeerRef), so that can be used a >> >> substitute for extra arguments in this case. >> >> >> >> > diameter:add_transport is a sync call, but it doesn't mean you can >> >> > send >> >> >> >> add_transport *doesn't* wait for the config it's passed to result in a >> >> connection before returning. (It might never happen for one.) >> >> >> >> > diameter messages successfully when the function returns, i.e. it >> >> > usually >> >> > gets {error, no_connection} or { error, timeout }(this might because >> >> > of >> >> > the >> >> > server side) if you call diameter:call right after >> >> > diameter:add_transport. >> >> >> >> This is because the peer won't be ready to respond to requests until >> >> capabilities exchange has completed (at least). If you call before the >> >> relevant peer_up callback then the result is {error, no_connection}. >> >> After peer_up the result will be {error, timeout} if the peer doesn't >> >> answer. One case in which this is expected is after a connection has >> >> been reestablished following an exchange of 3 x DWR/DWA. (ie. RFC 3539 >> >> DOWN -> REOPEN -> OKAY.) Since both ends of the connection do this, >> >> the client can consider the connection to be reestablished before the >> >> server. If the client sends a request before the server is done with >> >> it's exchange (ie. reached OKAY) then RFC 6733 says it should discard >> >> the request, resulting in a timeout on the client end. >> >> >> >> > So is there any reason why not allowing to add additional arguments >> >> > and >> >> > use >> >> > them in peer_up callback function? Since it means capability exchange >> >> > is >> >> > done and the peer is ready, at this point it should be safe to send >> >> > diameter >> >> > messages to the peer. >> >> >> >> See above. >> >> >> >> Anders >> >> >> >> >> >> > >> >> > Any suggestions? >> >> > >> >> > Thanks! >> >> > >> >> > Samuel >> >> > >> >> > >> >> > On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson >> >> > >> >> > wrote: >> >> >> >> >> >> On Sun, Apr 14, 2013 at 7:01 PM, S X >> >> >> wrote: >> >> >> > Hello, >> >> >> > >> >> >> > Based on the erlang diameter library and the sample code, I want >> >> >> > to >> >> >> > start >> >> >> > multiple diameter client processes in one erlang node(one client >> >> >> > IP), >> >> >> > and >> >> >> > the client needs to define a diameter_app callback module for >> >> >> > certain >> >> >> > application, for example: >> >> >> > >> >> >> > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ >> >> >> > ".example.com"}, >> >> >> > {'Origin-Realm', "example.com"}, >> >> >> > {'Vendor-Id', 193}, >> >> >> > {'Product-Name', "Client"}, >> >> >> > {'Auth-Application-Id', >> >> >> > [?DIAMETER_APP_ID_COMMON, >> >> >> > ?DIAMETER_APP_ID_CCRA]}, >> >> >> > {application, [{alias, ?APP_CCR_ALIAS}, >> >> >> > {dictionary, >> >> >> > ?DIAMETER_DICT_CCRA}, >> >> >> > {module, >> >> >> > client_cb_ccra}]}]). >> >> >> > >> >> >> > >> >> >> > First question: >> >> >> > how the diameter library handles this situation? Will all diameter >> >> >> > client >> >> >> > processes share one single diameter_app callback module >> >> >> > "client_cb_ccra" >> >> >> > or >> >> >> > it will automatically attach different instance of the callback >> >> >> > module >> >> >> > (process) to the different client process by using spawn_monitor? >> >> >> > So >> >> >> > from >> >> >> > the callback handle_answer in "client_cb_ccra" I can notify the >> >> >> > proper >> >> >> > client proce by just calling client:notify() something. >> >> >> >> >> >> diameter doesn't know anything about your client process. If you >> >> >> want >> >> >> a callback to be able to contact the client process associated with >> >> >> the service in question (ie. the service whose name the callback >> >> >> gets >> >> >> as an argument) then you need to give the callback the means to do >> >> >> so. >> >> >> One way would be to map the service name to your process (eg. it's >> >> >> the >> >> >> registered name of your client process), another would be to pass >> >> >> some >> >> >> identification as extra arguments to the callbacks. (Eg. {module, >> >> >> [client_cb_ccra, X]} in the config above.) >> >> >> >> >> >> > Second question: >> >> >> > Notice that diameter:call allows to set extra arguments, so I was >> >> >> > wondering >> >> >> > I could set some data like: >> >> >> > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). >> >> >> > which sets the client process ID and I hope to deliver to the >> >> >> > callback >> >> >> > module "client_cb_ccra" and when the callback module knows which >> >> >> > client >> >> >> > process sends request and response accordingly, for example using >> >> >> > the >> >> >> > client >> >> >> > process ID in the callback function handle_answer. >> >> >> >> >> >> You can do that, but what is it you're trying to accomplish? if it's >> >> >> handle_answer that's supposed to communicate something then it >> >> >> already >> >> >> does: the return value of handle_answer is returned by >> >> >> diameter:call. >> >> >> >> >> >> > However, I don't want pack the extra arguments into the diameter >> >> >> > packet >> >> >> > since the diameter server doesn't know what they are and the extra >> >> >> > ones >> >> >> > are >> >> >> > not part of standard diameter packet. >> >> >> >> >> >> Not sure what you mean here. There's no way for your client to send >> >> >> the server anything other than a Diameter message. >> >> >> >> >> >> > Now I changed the pick_peer callback signatures to allow extra >> >> >> > arguments, >> >> >> > pick_peer([Peer | _], _, _SvcName, _State, A) >> >> >> > >> >> >> > But I got encoding error in the callback prepare_request >> >> >> > >> >> >> > =ERROR REPORT==== 14-Apr-2013::11:40:41 === >> >> >> > Error in process <0.175.0> with exit value: >> >> >> > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, >> >> >> >> >> >> The arity of your callback probably doesn't agree with the number of >> >> >> extra arguments you've specified: prepare_request will also get your >> >> >> extra arguments. >> >> >> >> >> >> >> >> >> > >> >> >> > > >> >> >> > > > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... >> >> >> > >> >> >> > {error,encode} >> >> >> > >> >> >> > In overall, I read the online documents, which have limited >> >> >> > information >> >> >> > on >> >> >> > how to use the extra arguments in the library and don't quite get >> >> >> > how >> >> >> > to >> >> >> > utilize the extra arguments to do something tricky, >> >> >> > >> >> >> > Are there any suggestions on how to deal with multiple client >> >> >> > processes? >> >> >> >> >> >> Not sure I understand what problem it is you're trying to solve. >> >> >> Multiple processes invoking diameter:call is nothing strange. You >> >> >> typically just return something useful (eg. the answer message from >> >> >> the peer) from handle_answer, which diameter:call then returns for >> >> >> the >> >> >> caller to deal with. >> >> >> >> >> >> Anders >> >> >> >> >> >> >> >> >> > Thanks a lot! >> >> >> > >> >> >> > Samuel >> >> >> > >> >> > >> >> > >> > >> > > > From matthias@REDACTED Wed May 8 10:59:31 2013 From: matthias@REDACTED (Matthias Lang) Date: Wed, 8 May 2013 10:59:31 +0200 Subject: [erlang-questions] xslt question: building the Erlang FAQ Message-ID: <20130508085931.GA26144@corelatus.se> Hi, Something, I suspect xsltproc, is eating the non-ASCII characters in the Erlang FAQ. I'm hoping someone here can give me a clue what it is. The Erlang FAQ is generated using the same tools as the rest of the Erlang documentation. But the transformation on erlang.org drops letters, whereas the same transformation on my PC works fine: Source: Erlang Programmation (Mickaël Rémond) On erlang.org: Erlang Programmation (Mickal Rmond) Output on my PC: Erlang Programmation (Micka?l R?mond) Can someone give me a clue as to what's going on? Can anyone reproduce the bad version? I can't. I have a workaround, but I'd prefer to know why it's going wrong. More information ---------------- The Erlang FAQ source is on github: https://github.com/matthiasl/Erlang-FAQ just type 'make' to build it on *nix machines, assuming you have xsltproc installed. Here's what it looks like on my machine: Erlang-FAQ >make erl_docgen_privdir=`escript erl_docgen_privdir.escript`; \ date=`date +"%B %e %Y"`; \ xsltproc --stringparam outdir obj --stringparam gendate "$date" --stringparam topdocdir . --xinclude -path $erl_docgen_privdir/docbuilder_dtd -path $erl_docgen_privdir/dtd_html_entities faq_html.xsl faq.xml ... Erlang-FAQ >grep Mick *.xml obj/* obtaining.xml: French: Erlang Programmation (Mickaël Rémond) obj/obtaining.html: French: Erlang Programmation (Micka?l R?mond) Erlang-FAQ >wget -q -O - http://www.erlang.org/faq/obtaining.html | grep Mick French: Erlang Programmation (Mickal Rmond) The version information I have for the 'erlang.org' machine which is generating the input I don't want: $ xsltproc --version Using libxml 20623, libxslt 10115 and libexslt 812 xsltproc was compiled against libxml 20623, libxslt 10115 and libexslt 812 libxslt 10115 was compiled against libxml 20623 libexslt 812 was compiled against libxml 20623 $ escript erl_docgen_privdir.escript /usr/local/otp/releases/sles10_64_R16A/lib/erl_docgen-0.3.4/priv I've built exactly the same version of xlstproc, with exactly the same libraries on my PC, but I don't get the dropped letters in the output. Raimo is fairly sure that it's not the webserver's fault. So maybe xsltproc is behaving differently on that machine for some reason. But why? One workaround i have is to write é instead of é. That solves the problem, but it's ugly. Might be a clue, though. I haven't tried seeing what happens if I just type ??? etc. in the XML source; I'd prefer to keep the FAQ source plain ASCII. A brutal workaround would be just to rename people. Mickael Remond, Clas Wikstrom and Bjorn. ;-) Matt From bsvancara@REDACTED Wed May 8 11:14:23 2013 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Wed, 8 May 2013 11:14:23 +0200 Subject: [erlang-questions] xslt question: building the Erlang FAQ In-Reply-To: <20130508085931.GA26144@corelatus.se> References: <20130508085931.GA26144@corelatus.se> Message-ID: Hi, make sure that DTD files are accessible. Bohuslav 2013/5/8 Matthias Lang > Hi, > > Something, I suspect xsltproc, is eating the non-ASCII characters in > the Erlang FAQ. I'm hoping someone here can give me a clue what it is. > > The Erlang FAQ is generated using the same tools as the rest of the > Erlang documentation. But the transformation on erlang.org drops > letters, whereas the same transformation on my PC works fine: > > Source: Erlang Programmation (Mickaël > Rémond) > On erlang.org: Erlang Programmation (Mickal Rmond) > Output on my PC: Erlang Programmation (Micka?l R?mond) > > Can someone give me a clue as to what's going on? > Can anyone reproduce the bad version? I can't. > I have a workaround, but I'd prefer to know why it's going wrong. > > > More information > ---------------- > > The Erlang FAQ source is on github: > > https://github.com/matthiasl/Erlang-FAQ > > just type 'make' to build it on *nix machines, assuming you have > xsltproc installed. Here's what it looks like on my machine: > > Erlang-FAQ >make > erl_docgen_privdir=`escript erl_docgen_privdir.escript`; \ > date=`date +"%B %e %Y"`; \ > xsltproc --stringparam outdir obj --stringparam gendate "$date" > --stringparam topdocdir . --xinclude -path > $erl_docgen_privdir/docbuilder_dtd -path > $erl_docgen_privdir/dtd_html_entities faq_html.xsl faq.xml > ... > Erlang-FAQ >grep Mick *.xml obj/* > obtaining.xml: French: Erlang Programmation (Mickaël > Rémond) > obj/obtaining.html: French: Erlang Programmation > (Micka?l R?mond) > Erlang-FAQ >wget -q -O - http://www.erlang.org/faq/obtaining.html | > grep Mick > French: Erlang Programmation (Mickal Rmond) > > The version information I have for the 'erlang.org' machine which is > generating the input I don't want: > > $ xsltproc --version > Using libxml 20623, libxslt 10115 and libexslt 812 > xsltproc was compiled against libxml 20623, libxslt 10115 and libexslt > 812 > libxslt 10115 was compiled against libxml 20623 > libexslt 812 was compiled against libxml 20623 > > $ escript erl_docgen_privdir.escript > /usr/local/otp/releases/sles10_64_R16A/lib/erl_docgen-0.3.4/priv > > I've built exactly the same version of xlstproc, with exactly the same > libraries on my PC, but I don't get the dropped letters in the output. > Raimo is fairly sure that it's not the webserver's fault. So maybe > xsltproc is behaving differently on that machine for some reason. But > why? > > One workaround i have is to write é instead of é. That > solves the problem, but it's ugly. Might be a clue, though. I haven't > tried seeing what happens if I just type ??? etc. in the XML source; > I'd prefer to keep the FAQ source plain ASCII. > > A brutal workaround would be just to rename people. Mickael Remond, > Clas Wikstrom and Bjorn. ;-) > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed May 8 11:54:32 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 8 May 2013 10:54:32 +0100 Subject: [erlang-questions] Name Issues Message-ID: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> Hi guys, Me again :-) So, I'm having trouble setting the long name for a release. It used to work fine, but now I'm getting noname@REDACTED as my app name. I'm starting it using: rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/Mnesia.myapp.node1"' -name myapp I know I can use net_kernel:start to set the name, but if I do this, I still can't connect to other nodes on the same host that started correctly. How am I supposed to pass the name to the release when starting it? Thanks, Lee From essen@REDACTED Wed May 8 12:49:37 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 08 May 2013 12:49:37 +0200 Subject: [erlang-questions] Name Issues In-Reply-To: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> Message-ID: <518A2DC1.4080905@ninenines.eu> If you're going to set "myapp", use -sname myapp. -name is for specifying a long name, for example -name myapp@REDACTED On 05/08/2013 11:54 AM, Lee Sylvester wrote: > Hi guys, > > Me again :-) > > So, I'm having trouble setting the long name for a release. It used to work fine, but now I'm getting noname@REDACTED as my app name. I'm starting it using: > > rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/Mnesia.myapp.node1"' -name myapp > > I know I can use net_kernel:start to set the name, but if I do this, I still can't connect to other nodes on the same host that started correctly. How am I supposed to pass the name to the release when starting it? > > Thanks, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From quiquepaz@REDACTED Wed May 8 13:08:44 2013 From: quiquepaz@REDACTED (Enrique Paz) Date: Wed, 8 May 2013 13:08:44 +0200 Subject: [erlang-questions] Name Issues In-Reply-To: <518A2DC1.4080905@ninenines.eu> References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> <518A2DC1.4080905@ninenines.eu> Message-ID: "-name myapp" is the same as "-name myapp@`hostname -f`" If you don't add the @ part to -name, Erlang will try to retrieve it from your network configuration. 2013/5/8 Lo?c Hoguin > If you're going to set "myapp", use -sname myapp. -name is for specifying > a long name, for example -name myapp@REDACTED > > > On 05/08/2013 11:54 AM, Lee Sylvester wrote: > >> Hi guys, >> >> Me again :-) >> >> So, I'm having trouble setting the long name for a release. It used to >> work fine, but now I'm getting noname@REDACTED as my app name. I'm >> starting it using: >> >> rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/**Mnesia.myapp.node1"' >> -name myapp >> >> I know I can use net_kernel:start to set the name, but if I do this, I >> still can't connect to other nodes on the same host that started correctly. >> How am I supposed to pass the name to the release when starting it? >> >> Thanks, >> Lee >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- quique -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed May 8 13:18:29 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 8 May 2013 12:18:29 +0100 Subject: [erlang-questions] Name Issues In-Reply-To: References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> <518A2DC1.4080905@ninenines.eu> Message-ID: <01876ED6-2E1D-4BCE-AC83-409299BC88DD@gmail.com> Thanks, Lo?c, but I've tried using the following: -name myapp -name myapp@REDACTED -name 'myapp@REDACTED' -name '"myapp@REDACTED"' All have been unsuccessful. If I use an sname, then it works fine, but I need to use long names. Thanks again, Lee On 8 May 2013, at 12:08, Enrique Paz wrote: > "-name myapp" is the same as "-name myapp@`hostname -f`" > If you don't add the @ part to -name, Erlang will try to retrieve it from your network configuration. > > > 2013/5/8 Lo?c Hoguin > If you're going to set "myapp", use -sname myapp. -name is for specifying a long name, for example -name myapp@REDACTED > > > On 05/08/2013 11:54 AM, Lee Sylvester wrote: > Hi guys, > > Me again :-) > > So, I'm having trouble setting the long name for a release. It used to work fine, but now I'm getting noname@REDACTED as my app name. I'm starting it using: > > rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/Mnesia.myapp.node1"' -name myapp > > I know I can use net_kernel:start to set the name, but if I do this, I still can't connect to other nodes on the same host that started correctly. How am I supposed to pass the name to the release when starting it? > > Thanks, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > quique -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed May 8 13:49:06 2013 From: matthias@REDACTED (Matthias Lang) Date: Wed, 8 May 2013 13:49:06 +0200 Subject: [erlang-questions] xslt question: building the Erlang FAQ In-Reply-To: References: <20130508085931.GA26144@corelatus.se> Message-ID: <20130508114906.GA27149@corelatus.se> On Wednesday, May 08, Bohuslav Svancara wrote: > make sure that DTD files are accessible. Thanks! That was pretty much the problem. The DTDs and .ent files are stored in different places for different Erlang versions, which is why it works for me but not on erlang.org: R14B03: lib/erl_docgen/priv/docbuilder_dtd/ R15B03: lib/erl_docgen/priv/dtd/ A bit surprising that xsltproc doesn't bomb out when it encounters errors like that. Thanks again, Matt From bombadil@REDACTED Wed May 8 14:07:37 2013 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Wed, 08 May 2013 14:07:37 +0200 Subject: [erlang-questions] Name Issues In-Reply-To: <01876ED6-2E1D-4BCE-AC83-409299BC88DD@gmail.com> References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> <518A2DC1.4080905@ninenines.eu> <01876ED6-2E1D-4BCE-AC83-409299BC88DD@gmail.com> Message-ID: Hi Lee, Loic said about you use your FQDN instead of hostname. Try to execute in the shell: hostname -f You can use it as: -name myapp -name myapp@$(hostname -f) A try: erl -name myapp (myapp@REDACTED)> erl -name myapp@$(hostname -f) (myapp@REDACTED)> If it doesn't works for you, review your /etc/hosts file, you should have an entry as: 127.0.0.1 barbol.bosqueviejo.net barbol First, the FQDN and after that the single machine name. Regards. Manuel Rubio. From magnus@REDACTED Wed May 8 14:32:38 2013 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 08 May 2013 13:32:38 +0100 Subject: [erlang-questions] Name Issues In-Reply-To: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> (Lee Sylvester's message of "Wed, 8 May 2013 10:54:32 +0100") References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> Message-ID: Lee Sylvester writes: > So, I'm having trouble setting the long name for a release. It used > to work fine, but now I'm getting noname@REDACTED as my app name. I'm > starting it using: > > rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/Mnesia.myapp.node1"' -name myapp > > I know I can use net_kernel:start to set the name, but if I do this, I > still can't connect to other nodes on the same host that started > correctly. How am I supposed to pass the name to the release when > starting it? The start scripts generated by rebar doesn't do anything with emulator arguments passed on the command line. You need to put "-name myapp" in the vm.args file instead. Regards, Magnus From bbarvish@REDACTED Wed May 8 15:28:42 2013 From: bbarvish@REDACTED (Boris Barvish) Date: Wed, 8 May 2013 08:28:42 -0500 Subject: [erlang-questions] Help for a newbie... Message-ID: I have just started on my Erlang journey and am struggling with a compiler error that I am hoping someone with experience can spent 5 secs and tell me what i'm doing wrong, as I am not very good at understanding what the compiler is telling me. Thanks! I am passing in a string such as "morning, 1, 2, 3" as the UserEntry and simply want to create a variable which holds a list of the tokens. I am getting a message from the compiler saying "syntax error before: 'case'. -module(practicum). -export([placeorder/1]). placeorder(UserEntry) -> AllTokens = string:tokens(UserEntry, ","). case hd(AllTokens) of "morning" -> processMorning(lists:nthtail(AllTokens,2)); "night" -> processEvening(lists:nthtail(AllTokens,2)); _ -> io:fwrite("error\n") end. processMorning(Dishes) -> Dishes. processEvening(Dishes) -> Dishes. -- A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away. Antoine de Saint-Exupery -------------- next part -------------- An HTML attachment was scrubbed... URL: From oj@REDACTED Wed May 8 15:32:40 2013 From: oj@REDACTED (OJ Reeves) Date: Wed, 8 May 2013 23:32:40 +1000 Subject: [erlang-questions] Help for a newbie... In-Reply-To: References: Message-ID: Hi Boris, Your mistake is that the first line of your placeorder function is terminated with period (.) instead of a comma (,). If you make this change you should be fine. Best regards OJ On Wed, May 8, 2013 at 11:28 PM, Boris Barvish wrote: > I have just started on my Erlang journey and am struggling with a compiler > error that I am hoping someone with experience can spent 5 secs and tell me > what i'm doing wrong, as I am not very good at understanding what the > compiler is telling me. Thanks! > > I am passing in a string such as "morning, 1, 2, 3" as the UserEntry and > simply want to create a variable which holds a list of the tokens. I am > getting a message from the compiler saying "syntax error before: 'case'. > > -module(practicum). > > -export([placeorder/1]). > > placeorder(UserEntry) -> > AllTokens = string:tokens(UserEntry, ","). > > case hd(AllTokens) of > "morning" -> > processMorning(lists:nthtail(AllTokens,2)); > "night" -> > processEvening(lists:nthtail(AllTokens,2)); > _ -> io:fwrite("error\n") > end. > > processMorning(Dishes) -> > Dishes. > > processEvening(Dishes) -> > Dishes. > > > -- > A designer knows he has achieved perfection not when there is nothing left > to add, but when there is nothing left to take away. Antoine de > Saint-Exupery > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- OJ Reeves +61 431 952 586 http://buffered.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed May 8 15:34:00 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 08 May 2013 15:34:00 +0200 Subject: [erlang-questions] Help for a newbie... In-Reply-To: References: Message-ID: <518A5448.5030707@ninenines.eu> On 05/08/2013 03:28 PM, Boris Barvish wrote: > I am passing in a string such as "morning, 1, 2, 3" as the UserEntry and > simply want to create a variable which holds a list of the tokens. I am > getting a message from the compiler saying "syntax error before: 'case'. > > -module(practicum). > > -export([placeorder/1]). > > placeorder(UserEntry) -> > AllTokens = string:tokens(UserEntry, ","). You want a , at the end of the line there, not a . Expressions are separated with commas. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jesper.louis.andersen@REDACTED Wed May 8 15:34:09 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 8 May 2013 15:34:09 +0200 Subject: [erlang-questions] Help for a newbie... In-Reply-To: References: Message-ID: <10075994-60B5-4614-8BEF-E6530B43C435@erlang-solutions.com> On May 8, 2013, at 3:28 PM, Boris Barvish wrote: The following line: > AllTokens = string:tokens(UserEntry, ","). contains a dot (.) in the end. It should have been a comma (,) instead. In Erlang the dot (.) "ends sentences" whereas the comma signifies the wording "andalso" as in "do this andalso do that". Finally the semicolon can be read as an alternation "orelse" as in "do this orelse do that". Hope this helps. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From lee.sylvester@REDACTED Wed May 8 15:38:43 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 8 May 2013 14:38:43 +0100 Subject: [erlang-questions] Name Issues In-Reply-To: References: <0D212190-A5D6-46BB-90FA-FF61ABD2EFB3@gmail.com> Message-ID: <3F72CE3D-370C-4278-86B5-761B2B1E2400@gmail.com> Thanks, guys. It was the vm.args file that solved it. These things aren't overly obvious ;-) Btw, Magnus, it seems you can add arguments when running rebar, so long as they're not duplicated in the vm.args file. I have been passing a -port flag to my app for weeks, now, and it gets through fine. Thanks again. Regards, Lee On 8 May 2013, at 13:32, Magnus Henoch wrote: > Lee Sylvester writes: > >> So, I'm having trouble setting the long name for a release. It used >> to work fine, but now I'm getting noname@REDACTED as my app name. I'm >> starting it using: >> >> rel/myapp/bin/myapp console -mnesia dir '"/Users/leesylvester/tmp/Mnesia.myapp.node1"' -name myapp >> >> I know I can use net_kernel:start to set the name, but if I do this, I >> still can't connect to other nodes on the same host that started >> correctly. How am I supposed to pass the name to the release when >> starting it? > > The start scripts generated by rebar doesn't do anything with emulator > arguments passed on the command line. You need to put "-name myapp" > in the vm.args file instead. > > Regards, > Magnus From egil@REDACTED Wed May 8 16:18:57 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 8 May 2013 16:18:57 +0200 Subject: [erlang-questions] Maps Message-ID: <518A5ED1.2030507@erlang.org> Hi everyone! We finally have a Maps EEP for you. This will get you an idea on what we are working on. Some of the ideas presented here was presented at Erlang Factory SF Bay Area 2013. I am sure that there will be some discussions about the contents of this EEP and I hope the discussions will be both fruitful and civilized. The journey of Maps and this EEP has been long and by no means a straight-forward or continuous one. I had a crystal clear picture of what I wanted Maps to be when we first started discussing it within OTP about two-three years ago. This EEP resembles that vision but it has had a lot of contributions of other ideas from both within and outside of OTP. The idea was a functional data-type, a syntax aware mapping of key-value associations with pattern matching. A syntax similar to records but without the hazzle of compile-time dependency and with arbitrary terms as keys. Order was not important and it could be implemented with a Hash-Array-Mapped-Trie with good performance and memory trade-offs. This was not an approach to replace records. It was meant to replace records where suitable and in other regards not be a replacement but its own thing. From the community there has been many wishes of a Map like data-type and a few suggestions. The one suggestion that stands out is of course the Frames proposal from Richard O'Keefe. It is the most complete proposal I've seen and is very well thought out. Its goal is to be a record replacement and the proposal satisfies this goal very well. - If Frames are that good, why a separate EEP? - It boils down to goals and constraints. A record replacement is just that, a replacement. It's like asking the question, "What do we have?" instead of "What can we get?" The instant rebuttal would be "What do we need?" I say Maps. Frames has certainly influenced Maps. In many regards Maps also encompasses Frames but Maps tries to do more. I think the most significant difference would be, arbitrary terms as keys and how many different keys we would have in a Map. In the end I believe they are two different things and have different goals. Some Notes and Disclaimers: Later iterations of Maps has gone through some changes, most significantly, * From a set of key-values to a ordered set of key-value associations I was originally against this change since it forces restrictions on the implementation and it illuminates the lack of distinction between arithmetic order and term order, i.e. the problem of mixing integer and float types as keys in a tree. However, I was later persuaded that key ordering is necessary. We have to respect the totalitarian order of terms. Considerations has been made on how to, if at all possible, apply Frames characteristics to Maps. Most significantly memory space and key-sharing characteristics. This is not detailed in the EEP though, just mentioned. The function interface has had many revisions as well. At some stage the API was considered to be a drop-in-replacement for `dict` and thus would have the same function-names. This goal/constraint was dropped by Technical Board decision recently. From the very beginning Maps was envisioned to have the ability to bind variables derived from the environment. Like this: function(K1, #{ K1 := K2, K2 := V }) -> V. This feature is a beast. Powerful and scary. It is not confined to only Maps but should also be applicable to other types as well: function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. It is uncertain how effective such an implementation would be and in the end we might not want this feature at all. In this EEP we will describe syntax and semantics of Maps but very little is disclosed of its actual implementation. Current prototypes stems from using sparse tuples in a HAMT-like data structure and tuple-like data structures. The HAMT-like data structure is discontinued and will be replaced by some ordered tree. The proposal is included as an attachment but can also be viewed at this git-repository: https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md Regards, Bj?rn-Egil Dahlberg Erlang/OTP -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: eep-XXX.md Type: text/x-markdown Size: 51297 bytes Desc: not available URL: From essen@REDACTED Wed May 8 16:31:19 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 08 May 2013 16:31:19 +0200 Subject: [erlang-questions] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <518A61B7.8030201@ninenines.eu> On 05/08/2013 04:18 PM, Bj?rn-Egil Dahlberg wrote: > Hi everyone! Hi! There was examples of updating a map in a map in the SF talk, what have they become? I see nothing about this in the EEP. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed May 8 17:20:18 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 8 May 2013 19:20:18 +0400 Subject: [erlang-questions] Maps In-Reply-To: <518A61B7.8030201@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> Message-ID: It is great. I haven't understood what is the difference from Frames? Syntax? From freeakk@REDACTED Wed May 8 17:33:56 2013 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 8 May 2013 19:33:56 +0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> Message-ID: Max, map is just new name for frames. On Wed, May 8, 2013 at 7:20 PM, Max Lapshin wrote: > It is great. > > I haven't understood what is the difference from Frames? Syntax? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ? ?????????, ?????? ??????. Best regards, Uvarov Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed May 8 17:39:56 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 8 May 2013 11:39:56 -0400 Subject: [erlang-questions] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <20130508153955.GB79143@ferdair.local> I was looking at an example: 1> M = #{ 1 => a }, #{ 1 => a } 2> M#{ 1.0 => b }. #{ 1.0 => b }. 3> M#{ 1 := b }. #{ 1 => b } 4> M#{ 1.0 := b }. ** exception error: bad argument And given the definition where we have 'match' and 'equal' being two different things, am I right in understanding that '=>' can both create a mapping and update it based on 'equal' whereas ':=' can only update them using 'match' ? That seems like tricky semantics to me. Any reason why one can both update and insert but not the other? It's not like one operator is the alternative of the other exclusively on update/create or match/equal, but they're different in both forms. For the Dialyzer syntax, is there any distinction for types that have to do with empty/non_empty maps? I don't think there is any mention of the expected behaviour when matching on #{_ := V} = #{a => 3}, unless I missed it (there were cases in map comprehensions, but these are obviously not the same). Does this need to be specified? Similarly, #{_ := V} = #{a => 1, b => 1} and #{K := 1} = #{a => 1, b => 1} do not seem specified to me. For the rest of it, that looks like a decent proposal, and I'm interested by map comprehensions, but I'll let guys like ROK or jlouis take a more critical look at it ;) I'm just hoping nobody's gonna end up using maps as the thing that all state of all apps of all processes use and send over messages and you just have to look inside the map, son, because I'm not gonna go through the bother of sending concise messages to you! Regards, Fred. From jose.valim@REDACTED Wed May 8 17:39:56 2013 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Wed, 8 May 2013 09:39:56 -0600 Subject: [erlang-questions] Maps In-Reply-To: <518A61B7.8030201@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> Message-ID: Thanks Bj?rn-Egil, this proposal is great! I've also found myself agreeing with all the proposals discussed in Open Questions. There was examples of updating a map in a map in the SF talk, what have > they become? I see nothing about this in the EEP. For those who haven't seen the talk yet, I believe Loic is talking about this: A = #{ a => false }, B = #{ b => A }, C = B#{b}#{a => true}, C == #{ b => #{a => true} }. Notice how we did a nested update by nesting the access and update syntax. If there is a reason for it not to be included in the EEP I would love to hear it too. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed May 8 20:27:26 2013 From: matthias@REDACTED (Matthias Lang) Date: Wed, 8 May 2013 20:27:26 +0200 Subject: [erlang-questions] newer ways of benchmarking Erlang Message-ID: <20130508182726.GA4335@corelatus.se> Hi, I'm updating a few FAQ entries. In a question about benchmarking, I reference a couple of Bj?rn's pages: http://www.erlang.se/~bjorn http://www.erlang.se/~bjorn/benchmarks2/ they haven't been updated for a long time---they refer to R7B! One newer ways I know of benchmarking is the language shootout. Any more suggestions? Matthias From n.oxyde@REDACTED Thu May 9 00:50:07 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 9 May 2013 00:50:07 +0200 Subject: [erlang-questions] [eeps] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <9152811A-F464-4E47-9E3C-4D8245F496BE@gmail.com> Hello Bj?rn-Egil, So what happens with the following code? X = 2, F = fun (X, <>) end. If we allow to bind variables derived from the environment, shouldn't we forbid confusing expressions where a variable is both shadowed and used? With the current bindings, this is equivalent to: > X = 2, F = fun (Y, <>) -> {X,Y} end. #Fun > F(1, <<1:2>>). {2,1} > F(1, <<1:1>>). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(1,<<1:1voil?>>) With the new bindings, this could mean: > X = 2, F = fun (Y, <>) -> {X,Y} end. #Fun > F(1, <<1:1>>). {1,1} > F(1, <<1:2>>). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(1,<<1:2>>) If variables from the outer environment are still used over the ones from the pattern's, the current behaviour would still be exhibited; but it would be weird to have such backwards bindings. What I suggest on that matter is to forbid variables both used and shadowed at the same time in a pattern when maps are introduced, or maybe even before. I can write a patch for that. On the matter of the patterns themselves, will this be allowed? fun (<>, <>) -> {A,B,C,D} end. To fully use the limited scope of patterns, we could also broaden the support of constant expressions in patterns to guard expressions using only variables bound in both environments. As the question of dependencies is already answered by the previous convoluted pattern, allowing guard expressions to be used that way wouldn't be difficult. fun (<>, <>) -> {A,B,C,D} end. keyreplace(element(N, Tuple0), N, [Tuple0|L], Tuple) -> [Tuple|L]. I find this last example quite soothing. Obviously, that can be done already in Erlang: keyreplace(Key, N, [Tuple0|L], Tuple) when Key =:= element(N, Tuple0) -> [Tuple|L]. But what about that one? decode(<<5, Len, Data:(Len-1)/binary>>) -> {p5,Data}; % decode other packets. This one is less obvious to rewrite in current Erlang and I won't bother to do it. Regards, -- Anthony Ramine Le 8 mai 2013 ? 16:18, Bj?rn-Egil Dahlberg a ?crit : > function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. From mjtruog@REDACTED Thu May 9 01:25:42 2013 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 08 May 2013 16:25:42 -0700 Subject: [erlang-questions] newer ways of benchmarking Erlang In-Reply-To: <20130508182726.GA4335@corelatus.se> References: <20130508182726.GA4335@corelatus.se> Message-ID: <518ADEF6.3040508@gmail.com> I think the work here https://github.com/richcarl/berk with the goals here http://erlang.org/pipermail/erlang-questions/2013-April/073568.html might provide direction for newer formal benchmarking, though I am not sure what form it would take. I really wish there was more information on how erts options might be important for usage, in ways that aid performance, but without making the configuration architecture specific. If there were results that show the benefits of certain options, that would probably help. I know there is erts_alloc_config which helps figure out the garbage collection options for your situation. I also know there is the typical erl documentation which has brief mention of the options available. However, many of the options seem a bit obscure. For instance, "+scl false" was mentioned with the recent scheduling NIF problem, though it is unclear what situations "+scl false" is best for. "+spp true" looks like a cool option (and it is valid to say that its usage requires your own testing) but it would help if there was some example of where it works best. Perhaps there is information about options well-suited for embedded development that could help make the use-cases clearer. Benchmarks are often done with a particular release of Erlang without a good understanding of the effective options, so it may help to have a way of reporting the effective command line options (I don't think this currently exists) since various options are picked automatically (and there seems to be more automatic options than what is reported within the shell prompt header, just due to the number of erts options, but I could be wrong). A test or benchmark that shows the maximum acceptable mean (and possibly stddev) latency allowed within a NIF or BIF, while the expectation of "good" performance seems important. I think there have been suggestions that the maximum latency should be under 1 ms, but I could be wrong or the information could be anecdotal. Either way, it seems like a good thing to test, investigate, or expose as part of the scheduler changes. Then perhaps there could be more information about when you need to bump reductions. I don't necessarily understand why a NIF or BIF can't utilize a yield call, to yield to the scheduler, but there is probably a good reason, and it would be nice to understand more without going into the code to figure it out (probably trying to avoid the delay of a context switch? but there shouldn't be a delay here). So, any recent benchmarks done within the OTP team would help here. It seems like a common problem would be having lower average request latency when the system is under heavy load, which seems unusual/strange, but would be related to the schedulers being kept busier within the Erlang VM. Information on the proper usage of erts options for such issues, both with normal usage, and to make benchmarking more reliable seems important. Information from Basho that relates to Riak configuration might help here, if they are using some of the erts flags for specific situations. Thanks, Michael On 05/08/2013 11:27 AM, Matthias Lang wrote: > Hi, > > I'm updating a few FAQ entries. > > In a question about benchmarking, I reference a couple of Bj?rn's pages: > > http://www.erlang.se/~bjorn > http://www.erlang.se/~bjorn/benchmarks2/ > > they haven't been updated for a long time---they refer to R7B! > > One newer ways I know of benchmarking is the language shootout. > > Any more suggestions? > > Matthias > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Thu May 9 03:10:31 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 9 May 2013 13:10:31 +1200 Subject: [erlang-questions] Help for a newbie... In-Reply-To: References: Message-ID: I wonder if the original poster would have been so puzzled had the error message been "A function cannot begin with 'case'." which would have hinted that the compiler thought this was the beginning of a (new) function. The last parser I wrote, I switched from Yacc to plain old recursive descent and found it much easier to generate error messages that I found helpful. The original poster should also note that itIsNotConsideredGoodErlangStyleToUseWordsRunTogetherLikeThis. Multi-word identifiers in Erlang are normally separated by spaces unless they have to be compatible with something else. Itisalsonotgoodtorunwordstogetherwithnoindication. There is a problem with the calls to nthtail/2 (just because it's in the Erlang standard library doesn't mean it's a good name): the arguments are in the wrong order. In any case, you don't need it. -module(practicum). -export([place_order/1]). place_order(User_Entry) -> [Time,_|Dishes] = string:tokens(User_Entry, ","), case Time of "morning" -> process_morning(Dishes) ; "night" -> process_evening(Dishes) ; _ -> io:fwrite("error\n") end. process_morning(Dishes) -> Dishes. process_evening(Dishes) -> Dishes. You might prefer to eliminate the '_ -> io:fwrite(...)' case; if there's no match the 'case' will crash, and that's just what we want. Unless this is an assignment, in which case do what the assignment says. From ok@REDACTED Thu May 9 05:12:07 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 9 May 2013 15:12:07 +1200 Subject: [erlang-questions] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <2D90F718-52EB-4F3A-A215-F538BE251432@cs.otago.ac.nz> Summary: For their intended uses, frames are smaller and faster and safer than maps. For uses *other* than replacing records, maps are superior to frames. It would be good to have BOTH something with clean syntax that makes a good replacement for records AND something that does even better than the 'dict' module for dictionaries. It would be a very bad thing if we got maps *instead* of frames. Joe Armstrong is writing a new Erlang book in which he *does* use maps to replace records, but that would be a Bad Thing. Bj?rn-Egil Dahlberg is rather unkind about the Frames proposal when he says A record replacement is just that, a replacement. It's like asking the question, "What do we have?" instead of "What can we get?" The instant rebuttal would be "What do we need?" I say Maps. The central claim, that Frames are *JUST* a replacement for records, doing nothing novel, is quite untrue. We can see this with a point-by-point comparison. Quoting EEP 43: The new data-type shall have semantics, syntax and operations that: ? provides an association set from key terms to value terms which can be constructed, accessed and updated using language syntax Frames do that. ? can be uniquely distinguished from every other data-type in the language Frames do that. ? has no compile-time dependency for constructing, accessing or updating contents of maps nor for passing maps between modules, processes or over Erlang distribution Frames do that. ? can be used in matching expressions in the language Frames do that. ? has a one-to-one association between printing and parsing the data-type To the extent that I can make sense of that, NEITHER frames NOT maps do that. For example, <{ b = 1, a = 2 }> would be expected to print as <{ a = 2, b = 1 }>. The only way to understand the requirement so that maps satisfy it is if you print one and read back what you just read you get a new term which is indistinguishable from the old one. Frames do that. Frames do one more thing: ++++ has a canonical print form (up to white space) so that frames print the same if and only if they are equal. When maps were based on hash tables it did not appear that they would satisfy this requirement, and for Erlang I think it _is_ a requirement. ? has a well defined order between terms of the type and other Erlang types Frames do that. ? has at most O(log N) time complexity in insert and lookup operations, where N is the number of key-value associations. Frames have at most O(log N) time complexity in lookup operations. They do not satisfy a "fast insert" require", but that's not because I didn't _think_ of it, but because I thought of it and deliberately rejected it. Frames are BY DELIBERATE CAREFUL CHOICE not a general purpose dictionary data type. That's because they offer something that EEP 43's maps do not and cannot: ++++ costs aN+b worst case space (yes, Maps do that do) with b small and a <= 2 (Maps cannot do that) and typical cases should see a == 1 (Maps cannot do that). There is another explicit requirement for frames: ++++ lend themselves to type checking. If you look at type checked dictionaries in other languages, they require _all_ values to have the _same_ type. We would presumably have something like map(Domain, Range) in Erlang types. That makes them unsuitable for record-like uses. Frames are for record-like uses where you want millions of the things, holding few enough slots that you could bear to write them all down by hand, and where if you are already using -type specifications and the dialyzer, you want to continue to benefit from that. If Maps are "meant to replace records where suitable", the answer is that they are hardly ever suitable for that purpose. I see no problem with a really good general purpose dictionary data type _as such_. What I'm terrified is that after waiting all this time for something good to replace records, we'll get maps ***INSTEAD*** of frames, rather than ***AS WELL AS*** frames. We know (thanks to my micro-VM experiments) that frames can be compact _enough_ and efficient _enough_ to be used instead of records, and it's also clear that with type checking they can be safe _enough_ (defined as "safer than records") for this to be a good thing. It's clear that a data structure that offers O(log N) update instead of frames' O(N) --- well actually, that's the cost to update *one* field, updating F fields is still O(N) so it's O(N/F) per field --- has to pay for it somehow, and it's going to be more memory and more time. "In many records Maps also encompasses Frames but Maps tries to do more" is misleading. The Maps proposal does not in any practical sense "encompass Frames" because it does not achieve what the Frames proposal set out to achieve. The Frames proposal does more of what I *want* than the Maps proposal. What are the two main user-visible differences? - Frames are limited to atom keys in the interests of compile-time error detection (both simply noticing that a key is not an atom at all and allowing the possibility of Dialyzer support). Maps allow any term as a key. Arbitrary keys is something I wanted the way I wanted my hand nailed to a desk. If I want dictionaries with arbitrary keys, I want the 'dict' module, and a more efficient implementation of that would be just dandy. But by far the commoner case is to need a restricted key space so that my mistakes get caught. - Frames have O(lg N) access to a single field or O(N/k) access to k fields. They have O(N/k) cost for copying a frame with k fields changed. Maps have O(lg N) access and O(lg N) update. With a bit of data flow analysis, typical field access can be reduced to O(1) time with frames. I cannot recall if I've written that up, but it should be pretty obvious. The interesting thing here is that for smallish N, even fetching a *single* field can be faster with a frame than with a Map. And this is precisely the use-case for frames. From ok@REDACTED Thu May 9 05:33:46 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 9 May 2013 15:33:46 +1200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> Message-ID: <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> On 9/05/2013, at 3:33 AM, Michael Uvarov wrote: > Max, map is just new name for frames. ABSOLUTELY *NOT*! The desire for O(lg N) update forces some crucial implementation differences. For record-like uses, frames would be smaller, faster, and safer than maps. For dictionary-like uses, maps would be superior to frames. Clearly distinguishing between record-like data structures and dictionary-like data structures is the core of good design in this area. Looking at the example function(K1, #{ K1 := K2, K2 := V }) -> V. we see an example of the kind of thinking that has gone into the two proposals. The Frames proposal won't have anything to do with something like this, because I am a bear of very little brain who makes an embarrassing number of mistakes; I am _far_ more likely to have typed K1 by mistake (intending k1) than to want a variable here, and I would not give you a thank-you for a compiler that let this kind of thing through. In particular, frames were designed to be Dialyzer-friendly, so that there'd be a good chance that the Erlang tools would not just tell me "Hey, K1 is not known to be an atom here" but "hey, you asked for slot k1, but there isn't any such slot, did you mean k2?" The Maps proposal is about *breadth* (a jack of all trades data structure) and *expressiveness*. Note the example above, which _has_ to be matched from left to right. Frame patterns can be matched in any order, just like tuple patterns. (No, I _don't_ want patterns that can only be matched from left to right, thank you. I want the pattern match compiler to be able to choose what subpattern to do next based on information gain, not order.) I don't believe it is possible to type check that example, at least not in situations where the compiler doesn't know exactly what keys might be present. In contrast, <{ k1 = X, k2 = Y }> _can_ be type checked no matter what other fields might be present. Being able to write functions that require strict left to right matching is definitely more expressive than not being able to. Using frames, you'd have to write the running example as function(K1, Frame) -> frame_value(Frame, frame_value(Frame, K1)). However, sometimes we like to re-order the arguments of our functions. In the absence of binary patterns, we can do so freely now. Making it hard to spot when a refactoring is safe? That's not something the frames proposal is supposed to contain, and if anyone can find such a thing, I promise I'll fix it. From max.lapshin@REDACTED Thu May 9 10:10:31 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 May 2013 12:10:31 +0400 Subject: [erlang-questions] Maps In-Reply-To: <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> Message-ID: Richard. Would you, kindly, tell: where is the difference between maps and frames? As far as I understand, it is not about syntax, it is about internal way of storing data? From serge@REDACTED Thu May 9 13:45:22 2013 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 09 May 2013 07:45:22 -0400 Subject: [erlang-questions] erlexec and its environment In-Reply-To: <1367996052.4860.3.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> <51892EFD.7090106@aleynikov.org> <1367996052.4860.3.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <518B8C52.4000900@aleynikov.org> Actually both representations are used in parts of Erlang/OTP, e.g.: 1> os:getenv(). ["DISPLAY=:0","WINDOWPATH=1", ...]. 2> Env = [{"ABC", "Test"}], 3> erlang:open_port({spawn, "echo -n $ABC"}, [{env, Env}]). 4> f(X), receive {_, {data, X}} -> X end. "Test" But I'll mention this in the erlexec's doc. On 5/8/2013 2:54 AM, Bengt Kleberg wrote: > Please document which one of these is used else where in Erlang/OTP, and > that the other is not. > > > bengt > > On Tue, 2013-05-07 at 12:42 -0400, Serge Aleynikov wrote: >> The master branch of erlexec now has support for both {Name, Val} and >> "VAR=VALUE" environment options. >> >> On 5/7/2013 4:55 AM, Bengt Kleberg wrote: >>> Greetings, >>> >>> There are some Erlang functions that have {env, [...]} argument. Ex: >>> erlang:open_port/2. >>> >>> I think it would be good to have the same syntax ("a list of tuples >>> {Name, Val}, where Name is the name of an environment variable, and Val >>> is the value it is to have in the spawned port process." and the same >>> semantics for exec:run_link/2's {env, [...]}. >>> >>> >>> bengt >>> >>> On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: >>>> Hi, >>>> >>>> We're doing some work to fix up a few things we perceive as >>>> deficiencies in erlexec, and thought about asking for a community >>>> opinion. >>>> >>>> Currently, this: >>>> >>>> exec:run_link("exec /tmp/print_env.sh", []) >>>> >>>> Launches the child process with no environment, which was quite a >>>> surprise to us - the default on Unix systems is to have child >>>> processes inherit the parent process' environment. We're hacking the >>>> system so that it'll behave like we expect. >>>> >>>> When an environment is specified, we were thinking to use that to say >>>> "ok, clear out the environment and don't inherit it": >>>> >>>> exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) >>>> >>>> So that if you really needed to pass it in, you could do: >>>> >>>> exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) >>>> >>>> Thoughts? >>>> -- >>>> David N. Welton >>>> >>>> http://www.welton.it/davidw/ >>>> >>>> http://www.dedasys.com/ >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu May 9 22:56:42 2013 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 9 May 2013 21:56:42 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> Message-ID: <220951800.105489563.1368133002713.JavaMail.root@erlang-solutions.com> Even before taking a really deep dive into studying the EEP one thing I can immediately say: get rid of this having both equal and match and USE ONLY MATCH . Keys are the same when they match. Period. This paragraph: If key K does not equal any existing key in the map, a new association will be created from key K to value V . If key K is equal to an existing key in map M its associated value will be replaced by the new value V and the new key K will replace the old key if the key does not match . In both cases the evaluated map expression will return a new map. is weird. Yes I know what it means but it is not intuitive. When keys are replaced or not replaced when they are equal is not can seem very strange to those not deep into erlang semantics. Push Yes, I think we made an error with the different types of equalities and that comparisons covert integers to floats. With 20-20 hindsight I would prefer ==,/=,<,=<,>,>= to ONLY work on numbers and convert while another set @==,@/=,@<,@=<,@>,@>= (say) to work on terms and never convert. Pop While we are at it I think the evaluation order for keys and values should be defined as left-to-right. The rest of erlang is so why not here? Robert ----- Original Message ----- > From: "Bj?rn-Egil Dahlberg" > Hi everyone! > We finally have a Maps EEP for you. This will get you an idea on what > we are working on. Some of the ideas presented here was presented at > Erlang Factory SF Bay Area 2013. > I am sure that there will be some discussions about the contents of > this EEP and I hope the discussions will be both fruitful and > civilized. > The journey of Maps and this EEP has been long and by no means a > straight-forward or continuous one. I had a crystal clear picture of > what I wanted Maps to be when we first started discussing it within > OTP about two-three years ago. This EEP resembles that vision but it > has had a lot of contributions of other ideas from both within and > outside of OTP. > The idea was a functional data-type, a syntax aware mapping of > key-value associations with pattern matching. A syntax similar to > records but without the hazzle of compile-time dependency and with > arbitrary terms as keys. Order was not important and it could be > implemented with a Hash-Array-Mapped-Trie with good performance and > memory trade-offs. This was not an approach to replace records. It > was meant to replace records where suitable and in other regards not > be a replacement but its own thing . > From the community there has been many wishes of a Map like data-type > and a few suggestions. The one suggestion that stands out is of > course the Frames proposal from Richard O'Keefe. It is the most > complete proposal I've seen and is very well thought out. Its goal > is to be a record replacement and the proposal satisfies this goal > very well. > - If Frames are that good, why a separate EEP? > - It boils down to goals and constraints. > A record replacement is just that, a replacement. > It's like asking the question, "What do we have?" instead of "What > can we get?" > The instant rebuttal would be "What do we need?" I say Maps. > Frames has certainly influenced Maps. In many regards Maps also > encompasses Frames but Maps tries to do more. I think the most > significant difference would be, arbitrary terms as keys and how > many different keys we would have in a Map. In the end I believe > they are two different things and have different goals. > Some Notes and Disclaimers: > Later iterations of Maps has gone through some changes, most > significantly, > * From a set of key-values to a ordered set of key-value associations > I was originally against this change since it forces restrictions on > the implementation and it illuminates the lack of distinction > between arithmetic order and term order, i.e. the problem of mixing > integer and float types as keys in a tree. However, I was later > persuaded that key ordering is necessary. We have to respect the > totalitarian order of terms. > Considerations has been made on how to, if at all possible, apply > Frames characteristics to Maps. Most significantly memory space and > key-sharing characteristics. This is not detailed in the EEP though, > just mentioned. > The function interface has had many revisions as well. At some stage > the API was considered to be a drop-in-replacement for `dict` and > thus would have the same function-names. This goal/constraint was > dropped by Technical Board decision recently. > From the very beginning Maps was envisioned to have the ability to > bind variables derived from the environment. Like this: > function(K1, #{ K1 := K2, K2 := V }) -> V. > This feature is a beast. Powerful and scary. It is not confined to > only Maps but should also be applicable to other types as well: > function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. > It is uncertain how effective such an implementation would be and in > the end we might not want this feature at all. > In this EEP we will describe syntax and semantics of Maps but very > little is disclosed of its actual implementation. Current prototypes > stems from using sparse tuples in a HAMT-like data structure and > tuple-like data structures. The HAMT-like data structure is > discontinued and will be replaced by some ordered tree. > The proposal is included as an attachment but can also be viewed at > this git-repository: > https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md > Regards, > Bj?rn-Egil Dahlberg > Erlang/OTP > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From iqbaltalaat@REDACTED Thu May 9 17:20:02 2013 From: iqbaltalaat@REDACTED (I-T) Date: Thu, 9 May 2013 20:20:02 +0500 Subject: [erlang-questions] Mixing of http:request response Message-ID: I was experiencing an interesting issue. I need my mochiweb view, to post to another service and proxy the response. That happens within a receive block which looks like:- ?INFO("Posting to TransactionID ~s Payload ~s ~n", [TraceTransactionId, Temp]), case http:request(post, {Url, [], "application/x-www-form-urlencoded", Temp}, [], []) of {_, {_,_,JsonData}} -> ?INFO("Response Data TransactionID ~s Response ~s ~n", [TraceTransactionId, JsonData]), success(Req, "text/plain", JsonData); _ -> ?INFO("Error Posting to Adam ~n", []), error(Req, 500, "Problem Getting Data") end Then in concurrent scenarios, sometimes the response of one will be logged against the request of another while the TransactionID would remain the same. If However I swap this block with a simple curl command line executor, everything seems to work and the problem cant be repro'ed CurlCommand = io_lib:format("curl -s -d \"~s\" ~s", [Temp, Url]), ?INFO("Executing Curl Command ~s Curl ~s", [TraceTransactionId, CurlCommand]), CurlResponse = os:cmd(CurlCommand), ?INFO("Curl Response ~s Curl ~s", [TraceTransactionId, CurlResponse]), JsonData = string:strip(CurlResponse), success(Req, "text/plain", JsonData); This seems to work fine and I dont experience the concurrency issue. It should default to synchronous but in 10% of the cases it seems to mix the responses. My Questions 1) Am I using the inets http client the correct way? 2) If so then why does the curl wrapper work? 3) How can I investigate this further? Best Regards, Iqbal Talaat Bhatti "If we did all the things we are capable of doing, we would literally astound ourselves." - Thomas Edison -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri May 10 05:03:08 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 10 May 2013 15:03:08 +1200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> Message-ID: On 9/05/2013, at 8:10 PM, Max Lapshin wrote: > Richard. Would you, kindly, tell: where is the difference between maps > and frames? > > As far as I understand, it is not about syntax, it is about internal > way of storing data? I thought I already had, but here goes. There is one central semantic difference, which leads to two others. (U) What do you want to optimise the design for? Frames are optimised (pared to the bone, in fact) for use in record-like ways. They are somewhere between pathetic and hopeless as general purpose dictionaries. Maps optimised for service as general-purpose dictionaries. They are somewhere between almost usable and pathetic for record-like uses. I do not believe that it is possible to design a data structure which is *good* at both tasks. I suspect that it is possible to prove this, but I must admit I haven't tried. THIS is the reason that Bj?rn-Egil Dahlberg was able to sneer at frames as "just a replacement". I don't try to build perpetual motion machines. I don't try to square the circle with ruler and compass. I don't try to design bottle-openers that make good chisels. And given two sets of conflicting requirements (even if the sets overlap substantially), I don't try to satisfy them both with a single data structure. (Using a Java ArrayList as a queue is *possible*, but stupid.) (K) Frames only allow atoms as keys, maps allow any term. Frames can be manipulated using special syntax <{ K1 ~ V1, ... }>, as can maps. In the special syntax, the keys must be visibly and obviously literal atoms; they may not be expressions; maps allow keys to be expressions in expression context and variables (or is it guard expressions, and if not, why not?) in pattern context. Frames and maps can also be manipulated using functions; that is not a difference. (T) Frames take O(M+N) time to update/insert M fields into a (copy of a) frame that already has N keys. Maps take O(lg N) time to update/ insert a single field into an N-key map. Frames take O(M+N) time to match M fields in an N-key frame; this can be reduced to O(M.lg N) for small M and large N, but since the simple code can be blisteringly fast, there's not really much point. Frames go with a programming style in which instead of ... R#r.a ... ... R#r.b ... ... R#r.a ... R#r.c ... you write <{ a ~ A, b ~ B, c ~ C }> = R, ... A ... ... B ... ... A ... C ... which has the advantage of failing early if some of the required fields are not present. In fact my personal style for current Erlang says "never use R#r.a". These are not independent properties, and they bear on the key use-case difference: Frames were designed to be close to records in space and close to records in performance, for *record*-like uses (fixed collections of heterogeneous data) while also allowing a some flexibility, especially for upgrading. Maps were designed to be good for use as *dictionaries*; the design is optimised for *large* numbers of key/value associations. Such a design intrinsically satisfies the "functional requirements" for a record replacement (at least in an untyped language), but not the "non-functional" or quality requirements. As for implementation, Bj?rn-Egil Dahlberg wrote: > Current prototypes stems from using sparse tuples in a > HAMT-like data structure and tuple-like data structures. > The HAMT-like data structure is discontinued and will be > replaced by some ordered tree. The HAMT-like data structure for OCAML that I've looked at takes *at least* four words for every key/value association. (I have been able to find plenty of numbers for the *speed* of HAMTs but nothing for the *space* they need other than a claim of "small".) That's not necessarily the same thing, and the "tuple-like" part makes me wonder if something much cleverer is being done. However, we're told that's going "and will be replaced by some ordered tree". Suppose you have a 2-3-4 tree. We can play SML/Mercury-style games to move the size of a node into the pointer to it, so we 2-node: C1 K1 V1 C2 4 words/pair 3-node: C1 K1 V1 C2 K2 V2 C3 3.5 words/pair 4-node: C1 K1 V1 C2 K2 V2 C3 K3 V3 C4 3.3+words/pair So it's fair to say that this would take *roughly* 3.5 words per field. Playing similar move-the-balance-info-into-the-pointers games would let us do red-black trees or AVL trees in 4 words/pair. There are other games one can play, at the price of more complicated code, where we take advantage of the leaves of a 2-3-4 tree being at the same depth, so that leaf nodes are K1 V1 or K1 V1 K2 V2 or K1 V1 K2 V2 K3 V3 or K1 V1 K2 V2 K3 V3 K4 V4 and the tree as a whole is roughly 2.3 words per pair. I don't know what exactly Bj?rn-Egil Dahlberg has in mind, but if we charge 3 words per key/value association, it's not likely that we're overestimating the space cost. What about frames? A frame <{ K1 ~ V1, ..., Kn ~ Vn }> is represented as ---> { size-and-tag , | , V1, ..., Vn } / / { size-and-tag, '', K1, ..., Kn } It is, in fact, the same representation as a record, except that instead of the first element being the record *name* it is a pointer to the record's *descriptor*. The space needed is 4+2N words. But it gets better. In dictionary-like uses, you have *few* dictionaries with key sets that tend to be *different*. In record-like uses, you have *many* 'records, with a much smaller number of key-sets, so the descriptors can be *shared*. A typical record can be replaced by a frame with the *same* space cost: 2+N words. Segregating the keys is old idea for implementing environments in Lisp-like languages. It works brilliantly for key-sets that don't change much. One really cool thing is that it really lends itself to supporting basic operations like comparison. The storage representation is *canonical*. Two equal frames have the same parts in the same order. To compare two frames for equality: - if the sizes are different, the frames are different. - if the descriptors are not the same pointer, then if the descriptors are not *bitwise* identical in contents, the frames are different. (This can be implemented by a tiny chunk of machine code that's faster than memcmp().) - compare the rest of the frames as if comparing tuples. To compare two frames for ordering: - if the sizes are different, the shorter one is less - if the descriptors are not the same pointer, then compare the descriptors for ordering as tuples. If they are not equal, the lesser frame is the one with the lesser descriptor. - compare the rest of the frames as if comparing tuples. I don't understand HAMTs well enough to tell if they are canonical. In the absence of hash collisions, they might be, but what if there are collisions? It's certainly hard to see how they could be canonical for both == and =:= at the same time. (Not a problem for frames, because == and =:= coincide for atoms). For search trees, however, it is well understood that two trees can represent the same dictionary while having quite different shapes. This makes comparison much much harder. You can still do it in linear time, but you really have to work at it. From bengt.kleberg@REDACTED Fri May 10 11:18:23 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 10 May 2013 09:18:23 +0000 Subject: [erlang-questions] erlexec and its environment In-Reply-To: <518B8C52.4000900@aleynikov.org> References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> <51892EFD.7090106@aleynikov.org> <1367996052.4860.3.camel@sekic1152.rnd.ki.sw.ericsson.se>, <518B8C52.4000900@aleynikov.org> Message-ID: Erlang/OTP is inconsistent. Setting the env with tuples, getting it with strings. You either will be consistent with OTP, or consistent setting/ getting with yourself. IMHO name/ value pairs in OTP are mostly tuples so that is what I would like. Bengt Sent from Moxier Mail (http://www.moxier.com) ----- Ursprungligt meddelande ----- Fr?n: Serge Aleynikov Till: "erlang-questions@REDACTED" Skickat: 09-05-2013 1:47 em ?mne: Re: [erlang-questions] erlexec and its environment Actually both representations are used in parts of Erlang/OTP, e.g.: 1> os:getenv(). ["DISPLAY=:0","WINDOWPATH=1", ...]. 2> Env = [{"ABC", "Test"}], 3> erlang:open_port({spawn, "echo -n $ABC"}, [{env, Env}]). 4> f(X), receive {_, {data, X}} -> X end. "Test" But I'll mention this in the erlexec's doc. On 5/8/2013 2:54 AM, Bengt Kleberg wrote: > Please document which one of these is used else where in Erlang/OTP, and > that the other is not. > > > bengt > > On Tue, 2013-05-07 at 12:42 -0400, Serge Aleynikov wrote: >> The master branch of erlexec now has support for both {Name, Val} and >> "VAR=VALUE" environment options. >> >> On 5/7/2013 4:55 AM, Bengt Kleberg wrote: >>> Greetings, >>> >>> There are some Erlang functions that have {env, [...]} argument. Ex: >>> erlang:open_port/2. >>> >>> I think it would be good to have the same syntax ("a list of tuples >>> {Name, Val}, where Name is the name of an environment variable, and Val >>> is the value it is to have in the spawned port process." and the same >>> semantics for exec:run_link/2's {env, [...]}. >>> >>> >>> bengt >>> >>> On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: >>>> Hi, >>>> >>>> We're doing some work to fix up a few things we perceive as >>>> deficiencies in erlexec, and thought about asking for a community >>>> opinion. >>>> >>>> Currently, this: >>>> >>>> exec:run_link("exec /tmp/print_env.sh", []) >>>> >>>> Launches the child process with no environment, which was quite a >>>> surprise to us - the default on Unix systems is to have child >>>> processes inherit the parent process' environment. We're hacking the >>>> system so that it'll behave like we expect. >>>> >>>> When an environment is specified, we were thinking to use that to say >>>> "ok, clear out the environment and don't inherit it": >>>> >>>> exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) >>>> >>>> So that if you really needed to pass it in, you could do: >>>> >>>> exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) >>>> >>>> Thoughts? >>>> -- >>>> David N. Welton >>>> >>>> http://www.welton.it/davidw/ >>>> >>>> http://www.dedasys.com/ >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Fri May 10 13:48:19 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 10 May 2013 13:48:19 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> Message-ID: <518CDE83.4090401@ninenines.eu> On 05/10/2013 05:03 AM, Richard A. O'Keefe wrote: > Frames are optimised (pared to the bone, in fact) for use in > record-like ways. They are somewhere between pathetic and > hopeless as general purpose dictionaries. I think that's the bigger issue with frames. Are they worth spending the time implementing considering they are essentially a records replacement? Records work good enough for most purposes, with the exception of upgrades, which few people do anyway. Maps have the potential to replace proplists, dicts, and also some record misuses, where an interface requires you to include a file to have the records definition (the file example found in the EEP is one). They're much more interesting to implement. As far as I understand maps are not meant as a replacement for #state{} records. > (K) Frames only allow atoms as keys, maps allow any term. Atoms are only good for record-like use or list of options. They are a minority in a reasonably sized application (Cowboy, 8k+ lines, for example). Maps on the other hand with keys as iodata() would be much helpful to store many things related to requests and responses, like headers, cookies and more, and to convert them quickly with maps comprehensions. > But it gets better. In dictionary-like uses, you have *few* > dictionaries with key sets that tend to be *different*. Not my experience. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From amindfv@REDACTED Fri May 10 15:19:12 2013 From: amindfv@REDACTED (Tom Murphy) Date: Fri, 10 May 2013 06:19:12 -0700 Subject: [erlang-questions] Maps In-Reply-To: <518CDE83.4090401@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> Message-ID: On Fri, May 10, 2013 at 4:48 AM, Lo?c Hoguin wrote: > On 05/10/2013 05:03 AM, Richard A. O'Keefe wrote: > >> Frames are optimised (pared to the bone, in fact) for use in >> record-like ways. They are somewhere between pathetic and >> hopeless as general purpose dictionaries. >> > > I think that's the bigger issue with frames. Are they worth spending the > time implementing considering they are essentially a records replacement? > Records work good enough for most purposes, with the exception of upgrades, > which few people do anyway. > One of the things that's very compelling to me about frames as a record replacement is that (as I understand it), frames are fully-distinguishable as a separate data type. The record abstraction is *very* leaky. Any Erlang coder who uses records has to know about - and contend with - its underlying representation as a tuple (insertion into ETS tables, for example, is something that's common and trips people up when the first atom (the record tag) is used as the key for the table). Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From amindfv@REDACTED Fri May 10 15:35:39 2013 From: amindfv@REDACTED (Tom Murphy) Date: Fri, 10 May 2013 06:35:39 -0700 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> Message-ID: Also, to anyone interested in the specifics of what frames actually are, I think the most recent/best resource for understanding them is here: http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf A lot of time has gone into refining this proposal - check it out! Tom On Fri, May 10, 2013 at 6:19 AM, Tom Murphy wrote: > On Fri, May 10, 2013 at 4:48 AM, Lo?c Hoguin wrote: > >> On 05/10/2013 05:03 AM, Richard A. O'Keefe wrote: >> >>> Frames are optimised (pared to the bone, in fact) for use in >>> record-like ways. They are somewhere between pathetic and >>> hopeless as general purpose dictionaries. >>> >> >> I think that's the bigger issue with frames. Are they worth spending the >> time implementing considering they are essentially a records replacement? >> Records work good enough for most purposes, with the exception of upgrades, >> which few people do anyway. >> > > One of the things that's very compelling to me about frames as a record > replacement is that (as I understand it), frames are fully-distinguishable > as a separate data type. > > The record abstraction is *very* leaky. Any Erlang coder who uses records > has to know about - and contend with - its underlying representation as a > tuple (insertion into ETS tables, for example, is something that's common > and trips people up when the first atom (the record tag) is used as the key > for the table). > > Tom > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shivajisainik@REDACTED Fri May 10 15:41:47 2013 From: shivajisainik@REDACTED (Natesh Manikoth) Date: Fri, 10 May 2013 09:41:47 -0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> Message-ID: Erlang newbie here. This comment therefore is not about the merits of the proposals. I detect a certain tendency to dismiss suggestions if the suggestions are not germane to that particular user's current (or past) needs. Maybe the suggestions didn't originate from a particular community of Erlang users. I hope things don't get dismissed because of NIH syndrome. There are very few (a handful) who take the time and trouble ROK seems to take to explore and explain the nuances of choices. I have followed his passionate support of Frames and read his proposal and see the value of something that replaces records. Since I am not a professional user of Erlang I will leave it to others to debate the technical merits. I just hope that folks ask the question whether it has to be an 'either/or' decision between Maps and Frames - can it be both. Now I will go back to watching these highly educational exchanges on the technical merits of the proposals. -A newbie On May 10, 2013 9:19 AM, "Tom Murphy" wrote: > On Fri, May 10, 2013 at 4:48 AM, Lo?c Hoguin wrote: > >> On 05/10/2013 05:03 AM, Richard A. O'Keefe wrote: >> >>> Frames are optimised (pared to the bone, in fact) for use in >>> record-like ways. They are somewhere between pathetic and >>> hopeless as general purpose dictionaries. >>> >> >> I think that's the bigger issue with frames. Are they worth spending the >> time implementing considering they are essentially a records replacement? >> Records work good enough for most purposes, with the exception of upgrades, >> which few people do anyway. >> > > One of the things that's very compelling to me about frames as a record > replacement is that (as I understand it), frames are fully-distinguishable > as a separate data type. > > The record abstraction is *very* leaky. Any Erlang coder who uses records > has to know about - and contend with - its underlying representation as a > tuple (insertion into ETS tables, for example, is something that's common > and trips people up when the first atom (the record tag) is used as the key > for the table). > > Tom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri May 10 16:34:58 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 10 May 2013 16:34:58 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> Message-ID: <518D0592.9000601@ninenines.eu> On 05/10/2013 03:41 PM, Natesh Manikoth wrote: > Erlang newbie here. This comment therefore is not about the merits of > the proposals. > I detect a certain tendency to dismiss suggestions if the suggestions > are not germane to that particular user's current (or past) needs. Not dismissing it here, I'd be happy to have both, but maps are solving a problem (dict manipulation is incredibly tedious and time consuming in Erlang) while frames simply improve what we already have. Personally not having maps prevents me from doing a particular project, unless I want to spend all my time manipulating dicts of dicts or records of records. Hence my only question about the maps proposal about recursive updating being built-in. Not having frames just requires me to know about a few pitfalls, but doesn't prevent me to get the job done. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jose.valim@REDACTED Fri May 10 17:17:46 2013 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Fri, 10 May 2013 09:17:46 -0600 Subject: [erlang-questions] Maps In-Reply-To: <518D0592.9000601@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: > Maps have the potential to replace proplists, dicts, and also some record misuses, where an interface requires you to include a file to have the records definition (the file example found in the EEP is one). They're much more interesting to implement. I am not sure realistically speaking how much Maps will replace proplists. They are extensively used in OTP and maps don't really seem to provide a huge advantage over them. My fear is adding more inconsistency to the libraries since some will accept maps as options, others accept proplists. The benefits would have to be considerable to replace proplists and I don't think it is the case now. But I agree; they can replace a good amount of dict usage (hard to say if it will be a full replacement without proper benchmarks) and record misuses. > Personally not having maps prevents me from doing a particular project, unless I want to spend all my time manipulating dicts of dicts or records of records. Hence my only question about the maps proposal about recursive updating being built-in. Agreed, nested access/updates are definitely a pain. Although there are other ways to ease this, like: access:find_in(NestedStructure, [dict_key, #some_record.a, another_key]) access:store_in(NestedStructure, [dict_key, #some_record.a, another_key], Value) So even if we don't get a recursive update built-in, it seems we can provide good solutions to the problem. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer On Fri, May 10, 2013 at 8:34 AM, Lo?c Hoguin wrote: > On 05/10/2013 03:41 PM, Natesh Manikoth wrote: > >> Erlang newbie here. This comment therefore is not about the merits of >> the proposals. >> I detect a certain tendency to dismiss suggestions if the suggestions >> are not germane to that particular user's current (or past) needs. >> > > Not dismissing it here, I'd be happy to have both, but maps are solving a > problem (dict manipulation is incredibly tedious and time consuming in > Erlang) while frames simply improve what we already have. > > Personally not having maps prevents me from doing a particular project, > unless I want to spend all my time manipulating dicts of dicts or records > of records. Hence my only question about the maps proposal about recursive > updating being built-in. > > Not having frames just requires me to know about a few pitfalls, but > doesn't prevent me to get the job done. > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Fri May 10 18:21:26 2013 From: juanjo@REDACTED (Juan Jose Comellas) Date: Fri, 10 May 2013 13:21:26 -0300 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: I think that maps give us the chance to greatly simplify inter-module communication. As Lo?c said, I'd also like to have both maps and frames, but right now if you want to send simple structured data to a process/function in another module you can: a) Use a record, with the added problem of sharing its definition and making sure that both modules are kept in sync with it b) Use a proplist and deal with very verbose code when manipulating it This is becoming increasingly relevant with all the JSON messaging being used in HTTP APIs and maps seem to be a great way to ease the pain in these situations (even erlson was already very convenient when dealing with JSON). I don't see the need to allow anything more than atoms, binaries and strings for the keys, though, but I do feel that forcing the keys to be literals would severely restrict their usability for the cases I mentioned. Juanjo On Fri, May 10, 2013 at 12:17 PM, Jos? Valim < jose.valim@REDACTED> wrote: > > Maps have the potential to replace proplists, dicts, and also some > record misuses, where an interface requires you to include a file to have > the records definition (the file example found in the EEP is one). They're > much more interesting to implement. > > I am not sure realistically speaking how much Maps will replace proplists. > They are extensively used in OTP and maps don't really seem to provide a > huge advantage over them. My fear is adding more inconsistency to the > libraries since some will accept maps as options, others accept proplists. > The benefits would have to be considerable to replace proplists and I don't > think it is the case now. > > But I agree; they can replace a good amount of dict usage (hard to say if > it will be a full replacement without proper benchmarks) and record misuses. > > > Personally not having maps prevents me from doing a particular project, > unless I want to spend all my time manipulating dicts of dicts or records > of records. Hence my only question about the maps proposal about recursive > updating being built-in. > > Agreed, nested access/updates are definitely a pain. Although there are > other ways to ease this, like: > > access:find_in(NestedStructure, [dict_key, #some_record.a, > another_key]) > access:store_in(NestedStructure, [dict_key, #some_record.a, > another_key], Value) > > So even if we don't get a recursive update built-in, it seems we can > provide good solutions to the problem. > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Lead Developer > > > On Fri, May 10, 2013 at 8:34 AM, Lo?c Hoguin wrote: > >> On 05/10/2013 03:41 PM, Natesh Manikoth wrote: >> >>> Erlang newbie here. This comment therefore is not about the merits of >>> the proposals. >>> I detect a certain tendency to dismiss suggestions if the suggestions >>> are not germane to that particular user's current (or past) needs. >>> >> >> Not dismissing it here, I'd be happy to have both, but maps are solving a >> problem (dict manipulation is incredibly tedious and time consuming in >> Erlang) while frames simply improve what we already have. >> >> Personally not having maps prevents me from doing a particular project, >> unless I want to spend all my time manipulating dicts of dicts or records >> of records. Hence my only question about the maps proposal about recursive >> updating being built-in. >> >> Not having frames just requires me to know about a few pitfalls, but >> doesn't prevent me to get the job done. >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 10 18:30:15 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 10 May 2013 20:30:15 +0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: Guys. erlang is a rather simple language, which is seriously spoiled by endless list_to_binary and binary_to_list conversions. Are you really sure that we need map_to_frame and frame_to_map ? From amindfv@REDACTED Fri May 10 19:14:22 2013 From: amindfv@REDACTED (amindfv@REDACTED) Date: Fri, 10 May 2013 13:14:22 -0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: <0728C1AA-CF6F-4A46-9A54-0A0EB866AC40@gmail.com> Just like in the case of lists/strings and binaries, the reason the conversions happen is because the data types converted to and from are useful in different ways! I agree we should avoid cluttering the language up, but if we view/use Maps as replacement for Dicts and Frames as replacement for Records, then the situation wouldn't be any more cluttered than it already is. Tom El May 10, 2013, a las 12:30 PM, Max Lapshin escribi?: > Guys. > > erlang is a rather simple language, which is seriously spoiled by > endless list_to_binary and binary_to_list conversions. Are you really > sure that we need map_to_frame and frame_to_map ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From serge@REDACTED Fri May 10 20:58:54 2013 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 10 May 2013 14:58:54 -0400 Subject: [erlang-questions] erlexec and its environment In-Reply-To: References: <1367916921.9627.6.camel@sekic1152.rnd.ki.sw.ericsson.se> <51892EFD.7090106@aleynikov.org> <1367996052.4860.3.camel@sekic1152.rnd.ki.sw.ericsson.se>, <518B8C52.4000900@aleynikov.org> Message-ID: <518D436E.3040807@aleynikov.org> Functionality and documentation updated: https://saleyn.github.io/erlexec/exec.html#type-cmd_options http://saleyn.github.io/erlexec/ On 5/10/2013 5:18 AM, Bengt Kleberg wrote: > > Erlang/OTP is inconsistent. Setting the env with tuples, getting it with strings. You either will be consistent with OTP, or consistent setting/ getting with yourself. > > IMHO name/ value pairs in OTP are mostly tuples so that is what I would like. > > > Bengt > > Sent from Moxier Mail > (http://www.moxier.com) > > > ----- Ursprungligt meddelande ----- > Fr?n: Serge Aleynikov > Till: "erlang-questions@REDACTED" > Skickat: 09-05-2013 1:47 em > ?mne: Re: [erlang-questions] erlexec and its environment > > > > Actually both representations are used in parts of Erlang/OTP, e.g.: > > 1> os:getenv(). > ["DISPLAY=:0","WINDOWPATH=1", ...]. > > 2> Env = [{"ABC", "Test"}], > 3> erlang:open_port({spawn, "echo -n $ABC"}, [{env, Env}]). > 4> f(X), receive {_, {data, X}} -> X end. > "Test" > > But I'll mention this in the erlexec's doc. > > On 5/8/2013 2:54 AM, Bengt Kleberg wrote: >> Please document which one of these is used else where in Erlang/OTP, and >> that the other is not. >> >> >> bengt >> >> On Tue, 2013-05-07 at 12:42 -0400, Serge Aleynikov wrote: >>> The master branch of erlexec now has support for both {Name, Val} and >>> "VAR=VALUE" environment options. >>> >>> On 5/7/2013 4:55 AM, Bengt Kleberg wrote: >>>> Greetings, >>>> >>>> There are some Erlang functions that have {env, [...]} argument. Ex: >>>> erlang:open_port/2. >>>> >>>> I think it would be good to have the same syntax ("a list of tuples >>>> {Name, Val}, where Name is the name of an environment variable, and Val >>>> is the value it is to have in the spawned port process." and the same >>>> semantics for exec:run_link/2's {env, [...]}. >>>> >>>> >>>> bengt >>>> >>>> On Tue, 2013-05-07 at 10:34 +0200, David Welton wrote: >>>>> Hi, >>>>> >>>>> We're doing some work to fix up a few things we perceive as >>>>> deficiencies in erlexec, and thought about asking for a community >>>>> opinion. >>>>> >>>>> Currently, this: >>>>> >>>>> exec:run_link("exec /tmp/print_env.sh", []) >>>>> >>>>> Launches the child process with no environment, which was quite a >>>>> surprise to us - the default on Unix systems is to have child >>>>> processes inherit the parent process' environment. We're hacking the >>>>> system so that it'll behave like we expect. >>>>> >>>>> When an environment is specified, we were thinking to use that to say >>>>> "ok, clear out the environment and don't inherit it": >>>>> >>>>> exec:run_link("/tmp/print_env.sh", [{env, ["VIVA=ERLANG"]}]) >>>>> >>>>> So that if you really needed to pass it in, you could do: >>>>> >>>>> exec:run_link("/tmp/print_env.sh", [{env, os:getenv() ++ ["VIVA=ERLANG"]}]) >>>>> >>>>> Thoughts? >>>>> -- >>>>> David N. Welton >>>>> >>>>> http://www.welton.it/davidw/ >>>>> >>>>> http://www.dedasys.com/ >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Sat May 11 00:42:28 2013 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 10 May 2013 23:42:28 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: <0728C1AA-CF6F-4A46-9A54-0A0EB866AC40@gmail.com> Message-ID: <1118490925.107196924.1368225748950.JavaMail.root@erlang-solutions.com> I am personally a ambivalent to maps as they have been presented. They are trying to solve two problems which both need solving but I don't think *one* solution is the way to go. So maps: - are to be a dynamic alternative to records which is a Good Thing. - are to be a faster replacement to dicts, which is always useful. The problems are different enough so that I think that in trying to be a solution to both will mean that they will do neither in the best way. So either pick one and solve it properly, or come with two solutions. >From my own experience I would say save the new syntax for the record alternative, this is where I would get the most out of it. Having atoms as keys would pose me no problems. Use a new data type here. For the dict alternative having a special syntax would give me very little so I wouldn't bother about it, using functions calls handles my usage very well. It also makes it easy to test alternate implementations to find the one with the most appropriate properties, just change the name of the module. For a built in type I would prefer speed to ordering. Actually you probably wouldn't need a built in type, just suitable BIFs to manage existing tuples/lists. I think I agree with ROK here. Anyway, that's my take on it, Robert ----- Original Message ----- > From: amindfv@REDACTED > > Just like in the case of lists/strings and binaries, the reason the > conversions happen is because the data types converted to and from > are useful in different ways! > > I agree we should avoid cluttering the language up, but if we > view/use Maps as replacement for Dicts and Frames as replacement for > Records, then the situation wouldn't be any more cluttered than it > already is. > > Tom > > > El May 10, 2013, a las 12:30 PM, Max Lapshin > escribi?: > > > Guys. > > > > erlang is a rather simple language, which is seriously spoiled by > > endless list_to_binary and binary_to_list conversions. Are you > > really > > sure that we need map_to_frame and frame_to_map ? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From steven.charles.davis@REDACTED Sat May 11 00:43:28 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 10 May 2013 15:43:28 -0700 (PDT) Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: I offer the suggestion that if "strings" were not processed as "just lists of integers" but treated as the binary data that they are then you'd not be needing to do this kind of thing... However, owing to that incorrect insistence in most of the platform apis, I can see that it's hard to avoid. best, /s On Friday, May 10, 2013 11:30:15 AM UTC-5, Max Lapshin wrote: > > Guys. > > erlang is a rather simple language, which is seriously spoiled by > endless list_to_binary and binary_to_list conversions. Are you really > sure that we need map_to_frame and frame_to_map ? > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat May 11 00:45:13 2013 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 10 May 2013 23:45:13 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: Message-ID: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> Processing lists of integers is faster than processing binaries, which is not surprising really. Robert ----- Original Message ----- > From: "Steve Davis" > I offer the suggestion that if "strings" were not processed as "just > lists of integers" but treated as the binary data that they are then > you'd not be needing to do this kind of thing... > However, owing to that incorrect insistence in most of the platform > apis, I can see that it's hard to avoid. > best, > /s > On Friday, May 10, 2013 11:30:15 AM UTC-5, Max Lapshin wrote: > > Guys. > > > erlang is a rather simple language, which is seriously spoiled by > > > endless list_to_binary and binary_to_list conversions. Are you > > really > > > sure that we need map_to_frame and frame_to_map ? > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-q...@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat May 11 00:47:14 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 11 May 2013 00:47:14 +0200 Subject: [erlang-questions] Maps In-Reply-To: <1118490925.107196924.1368225748950.JavaMail.root@erlang-solutions.com> References: <1118490925.107196924.1368225748950.JavaMail.root@erlang-solutions.com> Message-ID: <518D78F2.2020308@ninenines.eu> On 05/11/2013 12:42 AM, Robert Virding wrote: > For the dict alternative having a special syntax would give me very little You break my heart. It's the one thing that is important to me. I don't care about the implementation speed at all. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Sat May 11 00:52:38 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 11 May 2013 00:52:38 +0200 Subject: [erlang-questions] Maps In-Reply-To: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> References: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> Message-ID: <518D7A36.3030405@ninenines.eu> Matching no (match context and sub binaries prevent the need for copying anything), sending to another process no (smaller, plus above 64 bytes binaries are not copied), appending no (you can always prepend a string, but that's hardly intuitive use when manipulating text), etc. On 05/11/2013 12:45 AM, Robert Virding wrote: > Processing lists of integers is faster than processing binaries, which > is not surprising really. > > Robert > > ------------------------------------------------------------------------ > > *From: *"Steve Davis" > > I offer the suggestion that if "strings" were not processed as "just > lists of integers" but treated as the binary data that they are then > you'd not be needing to do this kind of thing... > > However, owing to that incorrect insistence in most of the platform > apis, I can see that it's hard to avoid. > > best, > /s > > On Friday, May 10, 2013 11:30:15 AM UTC-5, Max Lapshin wrote: > > Guys. > > erlang is a rather simple language, which is seriously spoiled by > endless list_to_binary and binary_to_list conversions. Are you > really > sure that we need map_to_frame and frame_to_map ? > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From steven.charles.davis@REDACTED Sat May 11 00:57:00 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 10 May 2013 15:57:00 -0700 (PDT) Subject: [erlang-questions] Maps In-Reply-To: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> References: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> Message-ID: I agree... if they are actually lists of integers. However, if the list of integers represents a reference (e.g. path "strings", attribute name "strings") does this also hold? On Friday, May 10, 2013 5:45:13 PM UTC-5, Robert Virding wrote: > > Processing lists of integers is faster than processing binaries, which is > not surprising really. > > Robert > > ------------------------------ > > *From: *"Steve Davis" > > > I offer the suggestion that if "strings" were not processed as "just lists > of integers" but treated as the binary data that they are then you'd not be > needing to do this kind of thing... > > However, owing to that incorrect insistence in most of the platform apis, > I can see that it's hard to avoid. > > best, > /s > > On Friday, May 10, 2013 11:30:15 AM UTC-5, Max Lapshin wrote: >> >> Guys. >> >> erlang is a rather simple language, which is seriously spoiled by >> endless list_to_binary and binary_to_list conversions. Are you really >> sure that we need map_to_frame and frame_to_map ? >> _______________________________________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat May 11 01:01:28 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 11 May 2013 00:01:28 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: <518D78F2.2020308@ninenines.eu> Message-ID: <1068052208.107218136.1368226888782.JavaMail.root@erlang-solutions.com> I am curious. Could you give me an example of how you would like to use the new syntax? Or a new syntax? Robert ----- Original Message ----- > From: "Lo?c Hoguin" > > On 05/11/2013 12:42 AM, Robert Virding wrote: > > For the dict alternative having a special syntax would give me very > > little > > You break my heart. > > It's the one thing that is important to me. I don't care about the > implementation speed at all. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > From mjtruog@REDACTED Sat May 11 02:41:19 2013 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 10 May 2013 17:41:19 -0700 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> Message-ID: <518D93AF.5030502@gmail.com> I am a bit disturbed by the concern over Frames. Since I started using Erlang I have always wanted native, efficient dictionary functionality as an actual type within the language (not just a dict module, despite the fact dict has one of the best possible implementations within native Erlang source code). It is a major disappointment coming to the Erlang language, being given the process dictionary, and then being told that you should never use it, since it is dirty mutable state that ruins our pure referential transparency.. So, the native dictionary type, Maps, seems long (long) overdue when Erlang is compared with newer languages like Python, Ruby., Perl and Java. Having a newer and better type of records named Frames seems like an opportunity to take something decent and make it slower (though I understand the extreme effort and attention to make performance close). The argument is hopefully not: "But wait, we have neat syntax which allows us to make neat examples, and they might be useful.... and now everyone should use Frames in their code instead of records", since that seems least desirable. I think it should be hard to make something more efficient than a simple array index lookup, which seems to be the basic idea behind a tuple (with the record syntactic sugar). I don't necessarily care about the records syntax being this preprocessor trick since that is made clear within the documentation. I think the idea that Frames provide a record alternative relies on the benefits of Frames, which appear too minimal because they seem to be focusing on improving the type system, more than extending functionality (so the concern looks more academic than pragmatic). One of the basic benefits of Frames that has been mentioned is that the type can be used explicitly for external data, however an external API could easily wrap the creation of an atom and tuple, so that justification of Frames doesn't feel very compelling (though the Frame type would make a record/tuple object more naturally a C struct-like type within Erlang). I understand often performance is not a concern, but I can't help but think it should always be the most important concern for compiler writers and VM writers (at the very least, not to mention the database writers, the message bus writers, and the algorithm library writers), which is why I believe it carries so much importance to the decision of the implementation of Maps or Frames. So, I am very happy with the pursuit of the Map type, while the pursuit of the Frame type seems inappropriate (in the sense of a feature triage). On 05/09/2013 08:03 PM, Richard A. O'Keefe wrote: > On 9/05/2013, at 8:10 PM, Max Lapshin wrote: > >> Richard. Would you, kindly, tell: where is the difference between maps >> and frames? >> >> As far as I understand, it is not about syntax, it is about internal >> way of storing data? > I thought I already had, but here goes. > > There is one central semantic difference, which leads to two > others. > > (U) What do you want to optimise the design for? > > Frames are optimised (pared to the bone, in fact) for use in > record-like ways. They are somewhere between pathetic and > hopeless as general purpose dictionaries. > > Maps optimised for service as general-purpose dictionaries. > They are somewhere between almost usable and pathetic for > record-like uses. > > I do not believe that it is possible to design a data structure > which is *good* at both tasks. I suspect that it is possible to > prove this, but I must admit I haven't tried. > > THIS is the reason that Bj?rn-Egil Dahlberg was able to sneer at > frames as "just a replacement". I don't try to build perpetual > motion machines. I don't try to square the circle with ruler and > compass. I don't try to design bottle-openers that make good > chisels. And given two sets of conflicting requirements (even if > the sets overlap substantially), I don't try to satisfy them both > with a single data structure. (Using a Java ArrayList as a queue > is *possible*, but stupid.) > > (K) Frames only allow atoms as keys, maps allow any term. > Frames can be manipulated using special syntax <{ K1 ~ V1, ... }>, > as can maps. In the special syntax, the keys must be visibly and > obviously literal atoms; they may not be expressions; maps allow > keys to be expressions in expression context and variables (or is > it guard expressions, and if not, why not?) in pattern context. > > Frames and maps can also be manipulated using functions; that is > not a difference. > > (T) Frames take O(M+N) time to update/insert M fields into a (copy of a) > frame that already has N keys. Maps take O(lg N) time to update/ > insert a single field into an N-key map. Frames take O(M+N) time to > match M fields in an N-key frame; this can be reduced to O(M.lg N) > for small M and large N, but since the simple code can be blisteringly > fast, there's not really much point. > > Frames go with a programming style in which instead of > > ... R#r.a ... > ... R#r.b ... > ... R#r.a ... R#r.c ... > > you write > > <{ a ~ A, b ~ B, c ~ C }> = R, > ... A ... > ... B ... > ... A ... C ... > > which has the advantage of failing early if some of the required fields > are not present. In fact my personal style for current Erlang says > "never use R#r.a". > > These are not independent properties, and they bear on the key > use-case difference: > > Frames were designed to be close to records in space and close to > records in performance, for *record*-like uses (fixed collections > of heterogeneous data) while also allowing a some flexibility, > especially for upgrading. > > Maps were designed to be good for use as *dictionaries*; the design > is optimised for *large* numbers of key/value associations. Such a > design intrinsically satisfies the "functional requirements" for a > record replacement (at least in an untyped language), but not the > "non-functional" or quality requirements. > > As for implementation, Bj?rn-Egil Dahlberg wrote: > >> Current prototypes stems from using sparse tuples in a >> HAMT-like data structure and tuple-like data structures. >> The HAMT-like data structure is discontinued and will be >> replaced by some ordered tree. > The HAMT-like data structure for OCAML that I've looked at > takes *at least* four words for every key/value association. > (I have been able to find plenty of numbers for the *speed* > of HAMTs but nothing for the *space* they need other than a > claim of "small".) That's not necessarily the same thing, > and the "tuple-like" part makes me wonder if something much > cleverer is being done. However, we're told that's going "and > will be replaced by some ordered tree". Suppose you have a > 2-3-4 tree. We can play SML/Mercury-style games to move the > size of a node into the pointer to it, so we > > 2-node: C1 K1 V1 C2 4 words/pair > 3-node: C1 K1 V1 C2 K2 V2 C3 3.5 words/pair > 4-node: C1 K1 V1 C2 K2 V2 C3 K3 V3 C4 3.3+words/pair > > So it's fair to say that this would take *roughly* 3.5 words > per field. > > Playing similar move-the-balance-info-into-the-pointers games > would let us do red-black trees or AVL trees in 4 words/pair. > > There are other games one can play, at the price of more > complicated code, where we take advantage of the leaves of > a 2-3-4 tree being at the same depth, so that leaf nodes > are K1 V1 > or K1 V1 K2 V2 > or K1 V1 K2 V2 K3 V3 > or K1 V1 K2 V2 K3 V3 K4 V4 > and the tree as a whole is roughly 2.3 words per pair. > > I don't know what exactly Bj?rn-Egil Dahlberg has in mind, > but if we charge 3 words per key/value association, it's > not likely that we're overestimating the space cost. > > What about frames? A frame <{ K1 ~ V1, ..., Kn ~ Vn }> > is represented as > > ---> { size-and-tag , | , V1, ..., Vn } > / > / > { size-and-tag, '', K1, ..., Kn } > > It is, in fact, the same representation as a record, except that > instead of the first element being the record *name* it is a > pointer to the record's *descriptor*. The space needed is 4+2N > words. > > But it gets better. In dictionary-like uses, you have *few* > dictionaries with key sets that tend to be *different*. In > record-like uses, you have *many* 'records, with a much smaller > number of key-sets, so the descriptors can be *shared*. A > typical record can be replaced by a frame with the *same* > space cost: 2+N words. > > Segregating the keys is old idea for implementing environments in > Lisp-like languages. It works brilliantly for key-sets that don't > change much. > > One really cool thing is that it really lends itself to supporting > basic operations like comparison. The storage representation is > *canonical*. Two equal frames have the same parts in the same order. > To compare two frames for equality: > - if the sizes are different, the frames are different. > - if the descriptors are not the same pointer, then > if the descriptors are not *bitwise* identical in contents, > the frames are different. (This can be implemented by a > tiny chunk of machine code that's faster than memcmp().) > - compare the rest of the frames as if comparing tuples. > To compare two frames for ordering: > - if the sizes are different, the shorter one is less > - if the descriptors are not the same pointer, then > compare the descriptors for ordering as tuples. > If they are not equal, the lesser frame is the one with > the lesser descriptor. > - compare the rest of the frames as if comparing tuples. > > I don't understand HAMTs well enough to tell if they are canonical. > In the absence of hash collisions, they might be, but what if there > are collisions? It's certainly hard to see how they could be > canonical for both == and =:= at the same time. (Not a problem for > frames, because == and =:= coincide for atoms). > > For search trees, however, it is well understood that two trees can > represent the same dictionary while having quite different shapes. > This makes comparison much much harder. You can still do it in > linear time, but you really have to work at it. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Sat May 11 11:46:26 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 11 May 2013 11:46:26 +0200 Subject: [erlang-questions] Maps In-Reply-To: <1068052208.107218136.1368226888782.JavaMail.root@erlang-solutions.com> References: <1068052208.107218136.1368226888782.JavaMail.root@erlang-solutions.com> Message-ID: <518E1372.3050605@ninenines.eu> Here's a simplified snippet: add_item(Player=#{ inventory => #{ ItemID => # { quantity = Q }}}, ItemID, ItemInfos) -> Player#{ inventory => #{ ItemID => #{ quantity => Q + 1 }}}; add_item(Player, ItemID, ItemInfos) -> Player#{ inventory => #{ ItemID => ItemInfos }}. Pretty clear what this code is doing, the equivalent with functions leaves me scratching my head. This kind of syntax also allows you to get it right the first time a lot more often than with functions. I need to do things like this for more than 500 different actions. There's not enough time in the world to do it with functions. That particular project is on hold until I can avoid wasting my time btw. On 05/11/2013 01:01 AM, Robert Virding wrote: > I am curious. Could you give me an example of how you would like to use the new syntax? Or a new syntax? > > Robert > > ----- Original Message ----- >> From: "Lo?c Hoguin" >> >> On 05/11/2013 12:42 AM, Robert Virding wrote: >>> For the dict alternative having a special syntax would give me very >>> little >> >> You break my heart. >> >> It's the one thing that is important to me. I don't care about the >> implementation speed at all. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From brunogirin@REDACTED Sat May 11 16:22:03 2013 From: brunogirin@REDACTED (Bruno Girin) Date: Sat, 11 May 2013 15:22:03 +0100 Subject: [erlang-questions] Help for a newbie... In-Reply-To: References: Message-ID: Another question from a newbie based on your code above. In this line: [Time,_|Dishes] = string:tokens(User_Entry, ","), The way I understand it, Time will point to the first token, the second will be dropped (as it maps to _) and Dishes will be the rest of the list. Am I correct? If yes, did you include _ to reflect the original code where lists:nthtail(AllTokens, 2) would have resulted in Dishes starting at the 3rd token effectively ignoring the 2nd token? I'm not sure that's what was intended based on the description. Also, would it be sensible to add space as a separator to avoid having them included in tokens? In which case, that line may be a bit closer to what Boris intended. [Time|Dishes] = string:tokens(User_Entry, ", "), Please correct me if I misunderstood something, I'm learning Erlang too :-) Cheers, Bruno On 9 May 2013 02:10, Richard A. O'Keefe wrote: > I wonder if the original poster would have been so puzzled had > the error message been > > "A function cannot begin with 'case'." > > which would have hinted that the compiler thought this was > the beginning of a (new) function. > > The last parser I wrote, I switched from Yacc to plain old > recursive descent and found it much easier to generate > error messages that I found helpful. > > The original poster should also note that > itIsNotConsideredGoodErlangStyleToUseWordsRunTogetherLikeThis. > Multi-word identifiers in Erlang are normally separated by > spaces unless they have to be compatible with something else. > Itisalsonotgoodtorunwordstogetherwithnoindication. > > There is a problem with the calls to nthtail/2 (just because > it's in the Erlang standard library doesn't mean it's a good > name): the arguments are in the wrong order. In any case, > you don't need it. > > -module(practicum). > -export([place_order/1]). > > place_order(User_Entry) -> > [Time,_|Dishes] = string:tokens(User_Entry, ","), > case Time > of "morning" -> process_morning(Dishes) > ; "night" -> process_evening(Dishes) > ; _ -> io:fwrite("error\n") > end. > > process_morning(Dishes) -> > Dishes. > > process_evening(Dishes) -> > Dishes. > > You might prefer to eliminate the '_ -> io:fwrite(...)' case; > if there's no match the 'case' will crash, and that's just what > we want. Unless this is an assignment, in which case do what > the assignment says. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Bruno Visit my weblog: http://brunogirin.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Sun May 12 17:08:48 2013 From: pablo.platt@REDACTED (pablo platt) Date: Sun, 12 May 2013 18:08:48 +0300 Subject: [erlang-questions] uncatched cowboy websocket handler termination In-Reply-To: <87cfad59-71ad-475b-8509-48fb63c98a83@googlegroups.com> References: <87cfad59-71ad-475b-8509-48fb63c98a83@googlegroups.com> Message-ID: I don't know enough in websocket_init/3 to create the session. I need the userid and auth token which are sent with the first websocket packet. If the session is linked to the websocket pid, Is it possible that websocket_terminate/3 will be called but the session won't get an exit message? >From my code, I thought that the only options are: - The websocket pid is dead when trying to link to it so the session will crash. - A link is created. If the websocket pid dies, the session will get an exit message. Is there a race condition in my implementation? On Sun, May 12, 2013 at 6:01 PM, fxmy wang wrote: > hi pablo, > When a websocket connection shuts down, the function *ws_handler:websocket_terminate(_Reason, > _Req, _State) *will be called. > So you probably should handle you session-ending through this function > call > > And more generally, it's a good way to initialize your session through > *websocket_init(_TransportName, Req, _Opts) *and do the clean up work in > *ws_handler:websocket_terminate(_Reason, _Req, _State) * > > On Tuesday, May 7, 2013 2:28:51 AM UTC+8, pablo platt wrote: >> >> Hi, >> >> I'm using a cowboy websocket handler as a simple chat transport. >> The websocket handler pass messages to a session gen_server. >> The session gen_server traps exits and links to the websocket pid >> so if the websocket pid dies, the session gen_server should terminate as >> well. >> >> I sometimes see sessions that thinks that the websocket pid is still >> alive although checking with is_process_alive/1 shows it is not. That leads >> to zombie sessions that accumulate over time. >> >> Is there a race condition in my code or something else I'm missing in >> catching the termination of the websocket pid? >> >> Please see a simplified version of the websocket handler and the session >> gen_server below. >> >> Thanks >> >> -module(my_session). >> -behaviour(gen_server). >> >> -export([handle_message/2]). >> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, >> terminate/2, code_change/3]). >> >> -record(state, {ws}). >> >> handle_message(undefined, Msg) -> >> case supervisor:start_child(my_session_sup, [Msg, self()]) of >> {error, Reason} -> {close, 1000, <<"">>}; >> {ok, Pid} -> >> gen_server:call(Pid, {websocket, self()}), >> self() ! {session, Pid}, >> {text, <<"ok">>} >> end; >> handle_message(Pid, Msg) -> >> gen_server:call(Pid, {msg, Msg}). >> >> init([Msg, WS]) -> >> case auth(Msg) of >> false -> {stop, normal}; >> true -> >> process_flag(trap_exit,true), >> {ok, #state{}) >> end. >> >> handle_call({websocket, Pid}, _, State) -> >> link(Pid), >> {reply, ok, State#state{ws=Pid}}; >> handle_call({msg, Msg}, _, State) -> >> Resp = msg(Msg, State), >> {reply, Resp, State}. >> >> handle_info({'EXIT', _Pid, _Reason}, State) -> >> {stop, normal, State}. >> >> ... >> >> >> -module(ws_handler). >> -behaviour(cowboy_websocket_handler). >> >> -export([init/3]). >> -export([websocket_init/3]). >> -export([websocket_handle/3]). >> -export([websocket_info/3]). >> -export([websocket_terminate/3]). >> >> -record(state, {tref, session}). >> >> init({tcp, http}, _Req, _Opts) -> >> {upgrade, protocol, cowboy_websocket}. >> >> websocket_init(_TransportName, Req, _Opts) -> >> Tref = erlang:send_after(10000, self(), timeout), >> {ok, Req, #state{tref=Tref}}. >> >> websocket_handle({text, ""}, Req, State) -> >> erlang:cancel_timer(State#state.tref), >> Tref = erlang:send_after(10000, self(), timeout), >> {ok, Req, State#state{tref=Tref}}; >> websocket_handle({text, Msg}, Req, State) -> >> Tref = erlang:send_after(10000, self(), timeout), >> Resp = my_session:handle_message(State#state.session, Msg), >> {reply, Resp, Req, State#state{tref=Tref}}; >> websocket_handle(_Data, Req, State) -> >> {ok, Req, State}. >> >> websocket_info({session, Pid}, Req, State) -> >> {ok, Req, State#state{session=Pid}}; >> websocket_info({terminate}, Req, State) -> >> {reply, {close, 1000, <<"">>}, Req, State}; >> websocket_info({timeout, _Ref, Msg}, Req, State) -> >> {shutdown, Req, State}; >> websocket_info(_Info, Req, State) -> >> {ok, Req, State}. >> >> websocket_terminate(_Reason, _Req, _State) -> >> ok. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Sun May 12 18:17:17 2013 From: pablo.platt@REDACTED (pablo platt) Date: Sun, 12 May 2013 19:17:17 +0300 Subject: [erlang-questions] uncatched cowboy websocket handler termination In-Reply-To: <21cec6d9-d7bd-4401-9387-5742dc70bccd@googlegroups.com> References: <87cfad59-71ad-475b-8509-48fb63c98a83@googlegroups.com> <21cec6d9-d7bd-4401-9387-5742dc70bccd@googlegroups.com> Message-ID: In my tests the session thinks that the websocket pid is alive although it is not. How come it didn't get the exit message? I'll use websocket_terminate/3 as a last resort but I want to understand why I'm not getting the exit message first. Is it possible that cowboy somehow prevent sending this message? On Sun, May 12, 2013 at 6:42 PM, fxmy wang wrote: > >I don't know enough in websocket_init/3 to create the session. I need the > userid and auth token which are sent with the first websocket packet. > The *Req* in *init({tcp, http}, Req, Opts)* and*websocket_init(_TransportName, Req, _Opts) > *contains a lot of things including User-Agents , Cookies, RequestedUrls > etc.. > You might find things you need there. > And if unlucky, you can just let the browser to send informations > explicitely after websocket connection is set up , then handled in *websocket_handle({text, > Msg}, Req, State)* > * > * > > If the session is linked to the websocket pid, Is it possible that > websocket_terminate/3 will be called but the session won't get an exit > message? * > * > Once the websocket connection shuts down , *websocket_terminate/3* will > be called , according to here > . > And maybe cowboy just keeps this pid alive after called * > websocket_terminate/3* for some other use. I can`t really be sure cause > I`m kind of lost in cowboy`s source code :( . This is one possibility > that your linked session process does not get a EXIT message. > > And , have you considering add a call in *websocket_terminate/3* like *my_session:kill_session( > self(), ...) * to explicitly end your session? > > On Sunday, May 12, 2013 11:08:48 PM UTC+8, pablo platt wrote: > >> I don't know enough in websocket_init/3 to create the session. I need the >> userid and auth token which are sent with the first websocket packet. >> >> If the session is linked to the websocket pid, Is it possible that >> websocket_terminate/3 will be called but the session won't get an exit >> message? >> >> From my code, I thought that the only options are: >> - The websocket pid is dead when trying to link to it so the session will >> crash. >> - A link is created. If the websocket pid dies, the session will get an >> exit message. >> >> >> Is there a race condition in my implementation? >> >> >> On Sun, May 12, 2013 at 6:01 PM, fxmy wang wrote: >> >>> hi pablo, >>> When a websocket connection shuts down, the function *ws_handler:websocket_terminate(_Reason, >>> _Req, _State) *will be called. >>> So you probably should handle you session-ending through this >>> function call >>> >>> And more generally, it's a good way to initialize your session >>> through *websocket_init(_TransportName, Req, _Opts) *and do the clean >>> up work in *ws_handler:websocket_terminate(_Reason, _Req, _State) * >>> >>> On Tuesday, May 7, 2013 2:28:51 AM UTC+8, pablo platt wrote: >>>> >>>> Hi, >>>> >>>> I'm using a cowboy websocket handler as a simple chat transport. >>>> The websocket handler pass messages to a session gen_server. >>>> The session gen_server traps exits and links to the websocket pid >>>> so if the websocket pid dies, the session gen_server should terminate >>>> as well. >>>> >>>> I sometimes see sessions that thinks that the websocket pid is still >>>> alive although checking with is_process_alive/1 shows it is not. That leads >>>> to zombie sessions that accumulate over time. >>>> >>>> Is there a race condition in my code or something else I'm missing in >>>> catching the termination of the websocket pid? >>>> >>>> Please see a simplified version of the websocket handler and the >>>> session gen_server below. >>>> >>>> Thanks >>>> >>>> -module(my_session). >>>> -behaviour(gen_server). >>>> >>>> -export([handle_message/2]). >>>> -export([init/1, handle_call/3, handle_cast/2, handle_info/2, >>>> terminate/2, code_change/3]). >>>> >>>> -record(state, {ws}). >>>> >>>> handle_message(undefined, Msg) -> >>>> case supervisor:start_child(my_session_sup, [Msg, self()]) of >>>> {error, Reason} -> {close, 1000, <<"">>}; >>>> {ok, Pid} -> >>>> gen_server:call(Pid, {websocket, self()}), >>>> self() ! {session, Pid}, >>>> {text, <<"ok">>} >>>> end; >>>> handle_message(Pid, Msg) -> >>>> gen_server:call(Pid, {msg, Msg}). >>>> >>>> init([Msg, WS]) -> >>>> case auth(Msg) of >>>> false -> {stop, normal}; >>>> true -> >>>> process_flag(trap_exit,true), >>>> {ok, #state{}) >>>> end. >>>> >>>> handle_call({websocket, Pid}, _, State) -> >>>> link(Pid), >>>> {reply, ok, State#state{ws=Pid}}; >>>> handle_call({msg, Msg}, _, State) -> >>>> Resp = msg(Msg, State), >>>> {reply, Resp, State}. >>>> >>>> handle_info({'EXIT', _Pid, _Reason}, State) -> >>>> {stop, normal, State}. >>>> >>>> ... >>>> >>>> >>>> -module(ws_handler). >>>> -behaviour(cowboy_websocket_handler). >>>> >>>> -export([init/3]). >>>> -export([websocket_init/3]). >>>> -export([websocket_handle/3]). >>>> -export([websocket_info/3]). >>>> -export([websocket_terminate/3]). >>>> >>>> -record(state, {tref, session}). >>>> >>>> init({tcp, http}, _Req, _Opts) -> >>>> {upgrade, protocol, cowboy_websocket}. >>>> >>>> websocket_init(_TransportName, Req, _Opts) -> >>>> Tref = erlang:send_after(10000, self(), timeout), >>>> {ok, Req, #state{tref=Tref}}. >>>> >>>> websocket_handle({text, ""}, Req, State) -> >>>> erlang:cancel_timer(State#state.tref), >>>> Tref = erlang:send_after(10000, self(), timeout), >>>> {ok, Req, State#state{tref=Tref}}; >>>> websocket_handle({text, Msg}, Req, State) -> >>>> Tref = erlang:send_after(10000, self(), timeout), >>>> Resp = my_session:handle_message(State#state.session, Msg), >>>> {reply, Resp, Req, State#state{tref=Tref}}; >>>> websocket_handle(_Data, Req, State) -> >>>> {ok, Req, State}. >>>> >>>> websocket_info({session, Pid}, Req, State) -> >>>> {ok, Req, State#state{session=Pid}}; >>>> websocket_info({terminate}, Req, State) -> >>>> {reply, {close, 1000, <<"">>}, Req, State}; >>>> websocket_info({timeout, _Ref, Msg}, Req, State) -> >>>> {shutdown, Req, State}; >>>> websocket_info(_Info, Req, State) -> >>>> {ok, Req, State}. >>>> >>>> websocket_terminate(_Reason, _Req, _State) -> >>>> ok. >>>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon May 13 07:22:32 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 13 May 2013 17:22:32 +1200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> Message-ID: <5F1DAC01-F45D-4F7D-86BD-5DB5A449CBC6@cs.otago.ac.nz> On 11/05/2013, at 1:41 AM, Natesh Manikoth wrote: > Erlang newbie here. This comment therefore is not about the merits of the proposals. > I detect a certain tendency to dismiss suggestions if the suggestions are not germane to that particular user's current (or past) needs. The way I see it, we need *BOTH* a good replacement for records (which are pretty much ubiquitous in OTP) *AND* a top notch implementation of dictionaries. I don't just think it *can* be both, I think it *should* be both. One thing I note about dictionaries is that sometimes I want hashed dictionaries and sometimes I want sorted dictionaries, and when I want sorted dictionaries I often need to provide my own comparison function. (Java has HashMap and TreeMap. My Smalltalk has Dictionary and SortedDictionary.) The fact that I want more than one kind of dictionary and that I want more than one kind of hashing+equality (does string case matter?) or sorting (ascending or descending? what fields)? Suggests to me that no one kind of dictionary, however, good, should be privileged with special syntax, unless they _all_ can be. Just today, in the space of a few dozen lines of Java code, I found myself juggling both hashed and sorted sets and maps, with three different kinds of keys/elements. Given the possibility of a top notch implementation of dictionaries, a top notch implementation of sets should not be far away (hint hint). From ok@REDACTED Mon May 13 07:26:47 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 13 May 2013 17:26:47 +1200 Subject: [erlang-questions] Maps In-Reply-To: <518D0592.9000601@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> Message-ID: <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> On 11/05/2013, at 2:34 AM, Lo?c Hoguin wrote: > Not dismissing it here, I'd be happy to have both, but maps are solving a problem (dict manipulation is incredibly tedious and time consuming in Erlang) while frames simply improve what we already have. There is another EEP that I haven't finished trying to set up a *general* solution for the "deep updates" problem. It really isn't a data structure issue. Maps are about three things: - a data structure (although it appears that the data structure will be changed) - a syntax for that data structure which makes it seem as though it is strongly recommended as a record replacement (and that is *precisely* how Joe Armstrong has taken it; the new book he's working on uses maps where it would otherwise have used records or frames) - an attempt to solve the deep updates problem. These things can be decoupled. We can have a "deep updates" solution that works for lists AND tuples AND records AND frames AND maps. We can have a top notch data structure. And these things do not require maps to use frame-like syntax. From jesper.louis.andersen@REDACTED Mon May 13 11:02:20 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 13 May 2013 11:02:20 +0200 Subject: [erlang-questions] Maps In-Reply-To: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> References: <1132139205.107200571.1368225913493.JavaMail.root@erlang-solutions.com> Message-ID: On May 11, 2013, at 12:45 AM, Robert Virding wrote: > Processing lists of integers is faster than processing binaries, which is not surprising really. I would contend this claim. Processing a binary is more cache-friendly than processing a list, so in certain cases, cache locality would have the binary win by a large margin. Or are you thinking of something entirely different? Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From davidw@REDACTED Mon May 13 11:43:30 2013 From: davidw@REDACTED (David Welton) Date: Mon, 13 May 2013 11:43:30 +0200 Subject: [erlang-questions] Dealing with "transient" external service (database) Message-ID: Hi, I'm thinking about what needs to be done for this Chicago Boss feature request/improvement: https://github.com/evanmiller/ChicagoBoss/issues/254 With the key point being that CB should start even if the database is not up. It should also handle someone stopping the database and having it be down for a few minutes. I'm trying to think of how this should work in Erlang, as I'm not sure it fits into the supervisor model. What I want to happen (I think) is something along these lines: * boss_db, which handles db connections, starts up, and tries to connect to the db driver/database itself. * If it can't connect, it sets some kind of flag, and keeps trying at intervals (possibly with exponential backoff until it hits a certain maximum interval). * Something similar happens if the connection goes down because the database has been stopped for whatever reason. * Attempts to use boss_db when things are not ok generate some kind of specific error that can be dealt with in other parts of the system, in order to at least inform the end user that there is something unusual going on, in a nice way. Sorry if this is all a bit fuzzy, but I think the general idea is clear: a web site should continue to run, but give warnings that can be defined by the user when the DB is down, rather than crash completely. -- David N. Welton http://www.dedasys.com/ From erlang@REDACTED Mon May 13 12:15:03 2013 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 13 May 2013 12:15:03 +0200 Subject: [erlang-questions] Maps In-Reply-To: <220951800.105489563.1368133002713.JavaMail.root@erlang-solutions.com> References: <518A5ED1.2030507@erlang.org> <220951800.105489563.1368133002713.JavaMail.root@erlang-solutions.com> Message-ID: On Thu, May 9, 2013 at 10:56 PM, Robert Virding < robert.virding@REDACTED> wrote: > Even before taking a really deep dive into studying the EEP one thing I > can immediately say: get rid of this having both *equal* and *match* and *USE > ONLY MATCH. *Keys are the same when they match. Period. This paragraph: > > If key K does not *equal* any existing key in the map, a new association > will be created from key K to value V. If key K is *equal* to an existing > key in map M its associated value will be replaced by the new value V *and > * the new key K will replace the old key if the key does not *match*. In > both cases the evaluated map expression will return a new map. > > is weird. Yes I know what it means but it is not intuitive. When keys are > replaced or not replaced when they are equal is not can seem very strange > to those not deep into erlang semantics. > I think the problem here is the description - not the semantics. It's not the keys which are replaced, the crucial idea is to have two different syntaxes. This needs a longer explanation to make sense. Short explanation ============== ':=' means "update an existing key - crash if they key is not present" '=>' means "update an existing key OR add a new key" The value is pretty much irrelevant Long explanation of why this is good ============================= We can update an existing map M with the syntax: M#{ K1 Op V1, K2 Op V2, ... Kn Op Vn} Where Op is either => or :=. The syntax K => V never generates an error and is used to introduce a new key or to update an existing key with a new value. The syntax K := V is used to update the value of an *existing* key and will raise an exception if the key K does not already exist in the map M. The difference between these two modes of update is crucial, but needs a couple of examples to explain: Assume we define a map M as follows: M = #{ foo => 1, bar => 2, baz => 3 } The update M # {foo := 12, bary := 24} Will fail (raise an exception) since there is no key called bary in the original map. This is good idea (ROK suggested this in his frames paper) since we don't want too accidentally create a new key due to a spelling error. This is the crash-early property of the := update syntax. Crashing later would make debugging difficult, we would accidentally add a bad key to a map and learn about it way later. The update M # {foo := 12, bar := 24} Will succeed, but more importantly the new map has exactly the same keys as the old map (since all the updates are ':=' updates) - and so can *share* the same key descriptor. So if we have a very long list of maps they can be stored in a space efficient manner. (Again this idea comes from ROKs frames paper). Bj?rn-Egil's eep didn't mention this but the fact that we know that two maps have the same keys from the syntax make a lot of optimizations possible). All of this is possible because there are two operators not one :-) --- As regards efficiency, utility, beauty and so on these are subjective. If you want the last ounce of efficiency records and dicts are not going to go away when maps arrive. So if maps have the wrong performance characteristics then use the exiting mechanisms. In the latest addition of my book I've been documenting the changes to maps - this chapter has changed three times and has tended to be conservative so I haven't (yet) mentioned that keys can be any term (and not just atoms). I'm rather looking forward to being able to represent things like XML and JSON and property lists in a maps and to have an one-size-fits-all replacement for dicts and records. I've never really worried about the last ounce of efficiency - if I want real efficiency I'd change language and go to C or program an FPGA. Cheers /Joe > > Push > Yes, I think we made an error with the different types of equalities and > that comparisons covert integers to floats. With 20-20 hindsight I would > prefer ==,/=,<,=<,>,>= to *ONLY* work on numbers and convert while > another set @==,@/=,@<,@=<,@>,@>= (say) to work on terms and never convert. > Pop > > While we are at it I think the evaluation order for keys and values should > be defined as left-to-right. The rest of erlang is so why not here? > > Robert > > ------------------------------ > > *From: *"Bj?rn-Egil Dahlberg" > > > Hi everyone! > > We finally have a Maps EEP for you. This will get you an idea on what we > are working on. Some of the ideas presented here was presented at Erlang > Factory SF Bay Area 2013. > > I am sure that there will be some discussions about the contents of this > EEP and I hope the discussions will be both fruitful and civilized. > > The journey of Maps and this EEP has been long and by no means a > straight-forward or continuous one. I had a crystal clear picture of what I > wanted Maps to be when we first started discussing it within OTP about > two-three years ago. This EEP resembles that vision but it has had a lot of > contributions of other ideas from both within and outside of OTP. > > The idea was a functional data-type, a syntax aware mapping of key-value > associations with pattern matching. A syntax similar to records but without > the hazzle of compile-time dependency and with arbitrary terms as keys. > Order was not important and it could be implemented with a > Hash-Array-Mapped-Trie with good performance and memory trade-offs. This > was not an approach to replace records. It was meant to replace records > where suitable and in other regards not be a replacement but its own ** > thing**. > > From the community there has been many wishes of a Map like data-type and > a few suggestions. The one suggestion that stands out is of course the > Frames proposal from Richard O'Keefe. It is the most complete proposal I've > seen and is very well thought out. Its goal is to be a record replacement > and the proposal satisfies this goal very well. > > - If Frames are that good, why a separate EEP? > - It boils down to goals and constraints. > > A record replacement is just that, a replacement. > It's like asking the question, "What do we have?" instead of "What can we > get?" > The instant rebuttal would be "What do we need?" I say Maps. > > Frames has certainly influenced Maps. In many regards Maps also > encompasses Frames but Maps tries to do more. I think the most significant > difference would be, arbitrary terms as keys and how many different keys we > would have in a Map. In the end I believe they are two different things and > have different goals. > > Some Notes and Disclaimers: > > Later iterations of Maps has gone through some changes, most > significantly, > > * From a set of key-values to a ordered set of key-value associations > > I was originally against this change since it forces restrictions on the > implementation and it illuminates the lack of distinction between > arithmetic order and term order, i.e. the problem of mixing integer and > float types as keys in a tree. However, I was later persuaded that key > ordering is necessary. We have to respect the totalitarian order of terms. > > Considerations has been made on how to, if at all possible, apply Frames > characteristics to Maps. Most significantly memory space and key-sharing > characteristics. This is not detailed in the EEP though, just mentioned. > > The function interface has had many revisions as well. At some stage the > API was considered to be a drop-in-replacement for `dict` and thus would > have the same function-names. This goal/constraint was dropped by Technical > Board decision recently. > > From the very beginning Maps was envisioned to have the ability to bind > variables derived from the environment. Like this: > > function(K1, #{ K1 := K2, K2 := V }) -> V. > > This feature is a beast. Powerful and scary. It is not confined to only > Maps but should also be applicable to other types as well: > > function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. > > It is uncertain how effective such an implementation would be and in the > end we might not want this feature at all. > > In this EEP we will describe syntax and semantics of Maps but very little > is disclosed of its actual implementation. Current prototypes stems from > using sparse tuples in a HAMT-like data structure and tuple-like data > structures. The HAMT-like data structure is discontinued and will be > replaced by some ordered tree. > > The proposal is included as an attachment but can also be viewed at this > git-repository: > https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md > > > Regards, > Bj?rn-Egil Dahlberg > Erlang/OTP > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon May 13 13:11:02 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 13 May 2013 13:11:02 +0200 Subject: [erlang-questions] Dealing with "transient" external service (database) In-Reply-To: References: Message-ID: <5190CA46.30309@ninenines.eu> Take a look at what I did here: https://github.com/extend/bank/blob/master/src/bank_worker.erl Basically on startup a message is sent to itself to attempt connecting, then if that fails it sends another message N seconds later to retry. It retries until it finally connects, even if it takes days. Once it has connected if something goes wrong the process crashes and the supervisor restarts it, going back to the first step above. If a query is sent and the process crashes then an error is returned instead of the expecting value which can be handled as you wish. On 05/13/2013 11:43 AM, David Welton wrote: > Hi, > > I'm thinking about what needs to be done for this Chicago Boss feature > request/improvement: > > https://github.com/evanmiller/ChicagoBoss/issues/254 > > With the key point being that CB should start even if the database is not up. > > It should also handle someone stopping the database and having it be > down for a few minutes. > > I'm trying to think of how this should work in Erlang, as I'm not sure > it fits into the supervisor model. What I want to happen (I think) is > something along these lines: > > * boss_db, which handles db connections, starts up, and tries to > connect to the db driver/database itself. > > * If it can't connect, it sets some kind of flag, and keeps trying at > intervals (possibly with exponential backoff until it hits a certain > maximum interval). > > * Something similar happens if the connection goes down because the > database has been stopped for whatever reason. > > * Attempts to use boss_db when things are not ok generate some kind of > specific error that can be dealt with in other parts of the system, in > order to at least inform the end user that there is something unusual > going on, in a nice way. > > Sorry if this is all a bit fuzzy, but I think the general idea is > clear: a web site should continue to run, but give warnings that can > be defined by the user when the DB is down, rather than crash > completely. > > -- > David N. Welton > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Mon May 13 13:43:45 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 13 May 2013 13:43:45 +0200 Subject: [erlang-questions] Maps In-Reply-To: <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> Message-ID: <5190D1F1.7020608@ninenines.eu> On 05/13/2013 07:26 AM, Richard A. O'Keefe wrote: > > On 11/05/2013, at 2:34 AM, Lo?c Hoguin wrote: >> Not dismissing it here, I'd be happy to have both, but maps are solving a problem (dict manipulation is incredibly tedious and time consuming in Erlang) while frames simply improve what we already have. > > There is another EEP that I haven't finished trying to set up a *general* > solution for the "deep updates" problem. It really isn't a data structure > issue. Don't forget the "deep access" possible not only in expressions but also in function clauses. You'd only be solving half the problem otherwise. > - a syntax for that data structure which makes it seem as though it is strongly > recommended as a record replacement (and that is *precisely* how Joe Armstrong > has taken it; the new book he's working on uses maps where it would otherwise > have used records or frames) To be perfectly honest I don't think that for most record uses this is going to be much of a problem. There's plenty of #state{} records in user applications that aren't accessed enough to make this significant. And if you wait 10 years, any performance difference will be insignificant... > These things can be decoupled. We can have a "deep updates" solution > that works for lists AND tuples AND records AND frames AND maps. We > can have a top notch data structure. And these things do not require > maps to use frame-like syntax. I would agree for key/value structures, but I fear that by including lists and non key/value tuples you're going to make this much too complex. Another very important thing that maps have and your solution wouldn't have, is that the syntax carries semantics that your solution won't have. For example you can't add new fields to a record, while you can to maps. If the same syntax is used for everything then that means I need to look elsewhere to understand the code. The maps EEP doesn't force me to do that. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jesper.louis.andersen@REDACTED Mon May 13 17:49:27 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 13 May 2013 17:49:27 +0200 Subject: [erlang-questions] Dealing with "transient" external service (database) In-Reply-To: <5190CA46.30309@ninenines.eu> References: <5190CA46.30309@ninenines.eu> Message-ID: <5C0001A0-73E4-4863-837C-48B85A54B519@erlang-solutions.com> Also, in addition: * While you are in the connecting state on your pool, you can return an error to that extent. * See the DB as a service in a separate application. It tries hard to provide the service, but if something is amiss, it should report back that your query cannot complete due to DB connectivity. This way, the application programmer in CB can handle the error gracefully if possible. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On May 13, 2013, at 1:11 PM, Lo?c Hoguin wrote: > Take a look at what I did here: > > https://github.com/extend/bank/blob/master/src/bank_worker.erl > > Basically on startup a message is sent to itself to attempt connecting, then if that fails it sends another message N seconds later to retry. It retries until it finally connects, even if it takes days. > > Once it has connected if something goes wrong the process crashes and the supervisor restarts it, going back to the first step above. > > If a query is sent and the process crashes then an error is returned instead of the expecting value which can be handled as you wish. > > On 05/13/2013 11:43 AM, David Welton wrote: >> Hi, >> >> I'm thinking about what needs to be done for this Chicago Boss feature >> request/improvement: >> >> https://github.com/evanmiller/ChicagoBoss/issues/254 >> >> With the key point being that CB should start even if the database is not up. >> >> It should also handle someone stopping the database and having it be >> down for a few minutes. >> >> I'm trying to think of how this should work in Erlang, as I'm not sure >> it fits into the supervisor model. What I want to happen (I think) is >> something along these lines: >> >> * boss_db, which handles db connections, starts up, and tries to >> connect to the db driver/database itself. >> >> * If it can't connect, it sets some kind of flag, and keeps trying at >> intervals (possibly with exponential backoff until it hits a certain >> maximum interval). >> >> * Something similar happens if the connection goes down because the >> database has been stopped for whatever reason. >> >> * Attempts to use boss_db when things are not ok generate some kind of >> specific error that can be dealt with in other parts of the system, in >> order to at least inform the end user that there is something unusual >> going on, in a nice way. >> >> Sorry if this is all a bit fuzzy, but I think the general idea is >> clear: a web site should continue to run, but give warnings that can >> be defined by the user when the DB is down, rather than crash >> completely. >> >> -- >> David N. Welton >> >> http://www.dedasys.com/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andra.dinu@REDACTED Mon May 13 18:38:57 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Mon, 13 May 2013 17:38:57 +0100 (BST) Subject: [erlang-questions] Erlang User of the Year - submit your nomination until 17 May In-Reply-To: <2030137859.111425257.1368463049724.JavaMail.root@erlang-solutions.com> Message-ID: <2047087296.111427313.1368463137693.JavaMail.root@erlang-solutions.com> As every year, at EUC 2013 the Erlang community will reward one Erlang user whose contribution to the development of the Erlang language has been outstanding. You are invited to submit your proposed names with a short motivation until 17 May here: https://docs.google.com/forms/d/1vc9PhhNDMxGXUKLtL6OeHbPndajG35EBF27mNpbzHVA/viewform Nominations will be analised by the following panel members: Joe Armstrong, Robert Virding, Kenneth Lundin - head of the Ericsson Erlang/OTP team, Dave Smith - winner of the 2011 award and Fred Hebert - winner of the 2012 award. The panel is led by Bjarne D?cker. Cheers, Andra -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From jani.j.hakala@REDACTED Mon May 13 19:39:09 2013 From: jani.j.hakala@REDACTED (Jani Hakala) Date: Mon, 13 May 2013 20:39:09 +0300 Subject: [erlang-questions] mingw compiled R16B Message-ID: <87obceu6de.fsf@pingviini.dyndns.org> Hi, I have succesfully compiled R16B for 32bit and 64bit windows with gcc (mingw32/mingw-w64) compiler in msys environment. If someone is interested, I have put patches and some information here: http://iki.fi/jahakala/programming/erlang/ I haven't done extensive testing yet, only a quick wx:demo() 'test'. Jani Hakala From egil@REDACTED Mon May 13 20:27:56 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Mon, 13 May 2013 20:27:56 +0200 Subject: [erlang-questions] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <519130AC.8090303@erlang.org> Hi! Thank you all who have contributed in this discussion! Instead of replying to every mail I will try to address and correct some misconceptions. - As stated earlier by several people. Maps are *not* Frames. - There are several semantic and syntax differences. Most notably arbitrary terms as keys. - The implementation of Maps will have Frame like characteristics for "simple" Maps, i.e. small number of associations (at most ~30 - 40 entries) and immediates as keys. Granted, the type information will not be there at compile-time, and Maps will not have literals as keys (as proposed in Frames), but should otherwise be comparable in performance with Frames for those cases. I also believe this would cover most of the use-cases as a record-replacement. The internal structure in the "simple" case would best be described with two tuples (but it would not have some other header and might have additionally meta-data in the header if necessary): { {K1, ...,Kn}, V1, ..., Vn} This structure will also allow key-sharing between Maps of the same "type", i.e. Maps with identical keys. When the Map no longer satisfies this "simple" arrangement it will transform into a tree and loose the previous characteristics and gain the characteristics of a tree. - Maps are of type ordered set and follows term order. This has several implications. - Term order does not distinguish between types float and integer. - If data is stored in a tree where we search entries based on keys and we do so by term order. Since term order does not distinguish between float and integer, floats and integers which are equal occupy the same place. - When iterating over keys in a Map it is done so in Term order, or reversed Term order, meaning it is expected that floats and integer are mixed, ex. 1 < 2.0 < 3 < 4.0 < 5 < 6.0. If we distinguish between floats and integers in Maps, let's say integer < doubles, the previous order would be, ex: 1 < 3 < 5 < 2.0 < 4.0 < 6.0. We would no longer iterate in Term order and this would not be consistent with the rest of Erlang, hence unacceptable. Maps emulates the behaviour of ETS ordered set in this sense. I think we can all agree that this is an unfortunate feature of Erlangs term order. I also agree that it would be best to have *matching* only, and not *equals* but I don't see how this could be achieved easily. At least not in an acceptable way. For example, we cannot achieve this by specifying our own Term order since that would violate iteration order, printing order, etc, or at least expected order. As noted previously, if we had Maps of type set we would not have this problem. However, since Maps needs to fit into Erlangs term order, Maps needs to be ordered (unless we could somehow abandon this strict rule). - Deep updates of Maps are not covered by this EEP. We would welcome suggestions on this. Thank you! Regards, Bj?rn-Egil On 2013-05-08 16:18, Bj?rn-Egil Dahlberg wrote: > Hi everyone! > > We finally have a Maps EEP for you. This will get you an idea on what > we are working on. Some of the ideas presented here was presented at > Erlang Factory SF Bay Area 2013. > > I am sure that there will be some discussions about the contents of > this EEP and I hope the discussions will be both fruitful and civilized. > > The journey of Maps and this EEP has been long and by no means a > straight-forward or continuous one. I had a crystal clear picture of > what I wanted Maps to be when we first started discussing it within > OTP about two-three years ago. This EEP resembles that vision but it > has had a lot of contributions of other ideas from both within and > outside of OTP. > > The idea was a functional data-type, a syntax aware mapping of > key-value associations with pattern matching. A syntax similar to > records but without the hazzle of compile-time dependency and with > arbitrary terms as keys. Order was not important and it could be > implemented with a Hash-Array-Mapped-Trie with good performance and > memory trade-offs. This was not an approach to replace records. It was > meant to replace records where suitable and in other regards not be a > replacement but its own thing. > > From the community there has been many wishes of a Map like data-type > and a few suggestions. The one suggestion that stands out is of > course the Frames proposal from Richard O'Keefe. It is the most > complete proposal I've seen and is very well thought out. Its goal is > to be a record replacement and the proposal satisfies this goal very > well. > > - If Frames are that good, why a separate EEP? > - It boils down to goals and constraints. > > A record replacement is just that, a replacement. > It's like asking the question, "What do we have?" instead of "What can > we get?" > The instant rebuttal would be "What do we need?" I say Maps. > > Frames has certainly influenced Maps. In many regards Maps also > encompasses Frames but Maps tries to do more. I think the most > significant difference would be, arbitrary terms as keys and how many > different keys we would have in a Map. In the end I believe they are > two different things and have different goals. > > Some Notes and Disclaimers: > > Later iterations of Maps has gone through some changes, most > significantly, > > * From a set of key-values to a ordered set of key-value associations > > I was originally against this change since it forces restrictions on > the implementation and it illuminates the lack of distinction between > arithmetic order and term order, i.e. the problem of mixing integer > and float types as keys in a tree. However, I was later persuaded that > key ordering is necessary. We have to respect the totalitarian order > of terms. > > Considerations has been made on how to, if at all possible, apply > Frames characteristics to Maps. Most significantly memory space and > key-sharing characteristics. This is not detailed in the EEP though, > just mentioned. > > The function interface has had many revisions as well. At some stage > the API was considered to be a drop-in-replacement for `dict` and thus > would have the same function-names. This goal/constraint was dropped > by Technical Board decision recently. > > From the very beginning Maps was envisioned to have the ability to > bind variables derived from the environment. Like this: > > function(K1, #{ K1 := K2, K2 := V }) -> V. > > This feature is a beast. Powerful and scary. It is not confined to > only Maps but should also be applicable to other types as well: > > function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. > > It is uncertain how effective such an implementation would be and in > the end we might not want this feature at all. > > In this EEP we will describe syntax and semantics of Maps but very > little is disclosed of its actual implementation. Current prototypes > stems from using sparse tuples in a HAMT-like data structure and > tuple-like data structures. The HAMT-like data structure is > discontinued and will be replaced by some ordered tree. > > The proposal is included as an attachment but can also be viewed at > this git-repository: > https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md > > > Regards, > Bj?rn-Egil Dahlberg > Erlang/OTP > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Tue May 14 00:16:57 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Tue, 14 May 2013 01:16:57 +0300 Subject: [erlang-questions] How to capitalize the first letter of Unicode string? Message-ID: <51916659.10309@gmail.com> Hello, Ca you suggest a way to capitalize the first letter of a Unicode string? Thanks a lot, Martin From pierrefenoll@REDACTED Tue May 14 00:22:05 2013 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Tue, 14 May 2013 00:22:05 +0200 Subject: [erlang-questions] How to capitalize the first letter of Unicode string? In-Reply-To: <51916659.10309@gmail.com> References: <51916659.10309@gmail.com> Message-ID: Isn't there a unicode modifier code point that capitalizes the code point next to it? Just asking. On Tuesday, May 14, 2013, Martin Dimitrov wrote: > Hello, > > Ca you suggest a way to capitalize the first letter of a Unicode string? > > Thanks a lot, > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 14 05:33:28 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 14 May 2013 15:33:28 +1200 Subject: [erlang-questions] How to capitalize the first letter of Unicode string? In-Reply-To: <51916659.10309@gmail.com> References: <51916659.10309@gmail.com> Message-ID: On 14/05/2013, at 10:16 AM, Martin Dimitrov wrote: > Can you suggest a way to capitalize the first letter of a Unicode string? Pierre Fenoll replied: >Isn't there a unicode modifier code point that capitalizes the code point next to it? No, there isn't. There are four titlecase letters and 1294 lower case letters in the UnicodeData.txt I just checked -- not sure what version that is, and I didn't count the TAG LATIN SMALL LETTER ... entries, which should not be converted. Unicode case conversion, done properly, is locale dependent, and may change the length of a string. The operation wanted here is, in all likelihood, conversion to title case rather than upper case. I have a crude hack which has worked adequately for me, but I really can't bring myself to put something that awful in this mailing list. From ok@REDACTED Tue May 14 05:43:03 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 14 May 2013 15:43:03 +1200 Subject: [erlang-questions] Maps In-Reply-To: <5190D1F1.7020608@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> Message-ID: On 13/05/2013, at 11:43 PM, Lo?c Hoguin wrote: > On 05/13/2013 07:26 AM, Richard A. O'Keefe wrote: >> >> On 11/05/2013, at 2:34 AM, Lo?c Hoguin wrote: >>> Not dismissing it here, I'd be happy to have both, but maps are solving a problem (dict manipulation is incredibly tedious and time consuming in Erlang) while frames simply improve what we already have. >> >> There is another EEP that I haven't finished trying to set up a *general* >> solution for the "deep updates" problem. It really isn't a data structure >> issue. > > Don't forget the "deep access" possible not only in expressions but also in function clauses. You'd only be solving half the problem otherwise. I was talking about deep *update*, not deep *access*. > > > To be perfectly honest I don't think that for most record uses this is going to be much of a problem. There's plenty of #state{} records in user applications that aren't accessed enough to make this significant. > > And if you wait 10 years, any performance difference will be insignificant... The frames proposal has been around for 10 years already. If I wait 10 years, it is quite likely that nothing will happen. Machines are not getting faster these days, so I'm not sure why waiting would make the performance difference insignificant. >> I would agree for key/value structures, but I fear that by including lists and non key/value tuples you're going to make this much too complex. Single mechanism. ONE thing to understand. Proven technology: people using other languages with this approach don't seem to have any problem. From janos.n.hary@REDACTED Tue May 14 10:44:34 2013 From: janos.n.hary@REDACTED (Janos Hary) Date: Tue, 14 May 2013 10:44:34 +0200 Subject: [erlang-questions] Mnesia wildcard query Message-ID: <001a01ce507f$43261140$c97233c0$@gmail.com> Hi, Is there a way to specify a 'begins with' style query in Mnesia? My keys are strings, and I'd like to get all of them sharing a given prefix. Thanks, Janos -------------- next part -------------- An HTML attachment was scrubbed... URL: From brunogirin@REDACTED Tue May 14 10:47:33 2013 From: brunogirin@REDACTED (Bruno Girin) Date: Tue, 14 May 2013 08:48:33 +0001 Subject: [erlang-questions] Help for a newbie... In-Reply-To: <7A89DD91-91ED-4E8F-85C0-2352E25BDC9C@cs.otago.ac.nz> References: <7A89DD91-91ED-4E8F-85C0-2352E25BDC9C@cs.otago.ac.nz> Message-ID: <5191fa2d.ebbbb40a.3e42.ffff805f@mx.google.com> Hi Richard, Thanks for your reply. I'm posting this back to the list as I think some of the answers can be useful to others. On Tue, 14 May, 2013 at 2:29 AM, Richard A. O'Keefe wrote: > > On 12/05/2013, at 2:22 AM, Bruno Girin wrote: > >> Another question from a newbie based on your code above. In this >> line: >> >> [Time,_|Dishes] = string:tokens(User_Entry, ","), >> >> The way I understand it, Time will point to the first token, >> > s/point to/be/ > Ah yes, I'm used to object oriented languages where everything is always a point to the actual object :-) > >> the second will be dropped (as it maps to _) and Dishes will be the >> rest of the list. Am I correct? >> > Yes. >> >> If yes, did you include _ to reflect the original code where >> lists:nthtail(AllTokens, 2) would have resulted in Dishes starting >> at the 3rd token effectively ignoring the 2nd token? >> > Yes. > Good, I'm glad I understand something about Erlang then! > > >> I'm not sure that's what was intended based on the description. >> > You may well be right. > >> Also, would it be sensible to add space as a separator to avoid >> having them included in tokens? >> > Dunno. Ask the original poster. The whole thing looked a bit > unfinished, > so it's not clear that at this stage there _is_ a right answer. > True, I was trying to guess the OP's intent which is never a good thing to do with code. Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jani.j.hakala@REDACTED Tue May 14 11:03:36 2013 From: jani.j.hakala@REDACTED (Jani Hakala) Date: Tue, 14 May 2013 12:03:36 +0300 Subject: [erlang-questions] How to capitalize the first letter of Unicode string? In-Reply-To: <51916659.10309@gmail.com> (Martin Dimitrov's message of "Tue, 14 May 2013 01:16:57 +0300") References: <51916659.10309@gmail.com> Message-ID: <87hai6szkn.fsf@pingviini.dyndns.org> Martin Dimitrov writes: > Hello, > > Ca you suggest a way to capitalize the first letter of a Unicode string? > Perhaps this library will help you, see https://github.com/erlang-unicode/i18n https://github.com/erlang-unicode/i18n/blob/master/doc/i18n_string.md It has functions like split, to_lower, to_upper, to_title. Jani Hakala From nick@REDACTED Tue May 14 11:14:59 2013 From: nick@REDACTED (Niclas Eklund) Date: Tue, 14 May 2013 11:14:59 +0200 Subject: [erlang-questions] SSL renegotiation Message-ID: <51920093.5020704@tail-f.com> Hi! Out of curiosity, why is this timelimit hardcoded and why was 12 choosen? -define(WAIT_TO_ALLOW_RENEGOTIATION, 12000). 42 would have made more sense since it's the meaning of life ;-) /Nick https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl_connection.erl From brunogirin@REDACTED Tue May 14 11:18:46 2013 From: brunogirin@REDACTED (Bruno Girin) Date: Tue, 14 May 2013 09:19:46 +0001 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <220951800.105489563.1368133002713.JavaMail.root@erlang-solutions.com> Message-ID: <51920178.815a0f0a.26dc.ffffb635@mx.google.com> On Mon, 13 May, 2013 at 11:15 AM, Joe Armstrong wrote: > > > On Thu, May 9, 2013 at 10:56 PM, Robert Virding > wrote: >> Even before taking a really deep dive into studying the EEP one >> thing I can immediately say: get rid of this having both equal and >> match and USE ONLY MATCH. Keys are the same when they match. Period. >> This paragraph: >> >> If key K does not equal any existing key in the map, a new >> association will be created from key K to value V. If key K is equal >> to an existing key in map M its associated value will be replaced by >> the new value V and the new key K will replace the old key if the >> key does not match. In both cases the evaluated map expression will >> return a new map. >> >> is weird. Yes I know what it means but it is not intuitive. When >> keys are replaced or not replaced when they are equal is not can >> seem very strange to those not deep into erlang semantics. >> > > I think the problem here is the description - not the semantics. It's > not the keys which are replaced, the crucial > idea is to have two different syntaxes. This needs a longer > explanation to make sense. > > Short explanation > ============== > > ':=' means "update an existing key - crash if they key is not > present" > '=>' means "update an existing key OR add a new key" > > The value is pretty much irrelevant > > Long explanation of why this is good > ============================= > > We can update an existing map M with the syntax: > > M#{ K1 Op V1, K2 Op V2, ... Kn Op Vn} > > Where Op is either => or :=. > > The syntax K => V never generates an error and is used to introduce a > new key > or to update an existing key with a new value. > > The syntax K := V is used to update the value of an *existing* key > and will > raise an exception if the key K does not already exist in the map M. > > The difference between these two modes of update is crucial, but needs > a couple of examples to explain: > > Assume we define a map M as follows: > > M = #{ foo => 1, bar => 2, baz => 3 } > > > The update > > M # {foo := 12, bary := 24} > > Will fail (raise an exception) since there is no key called bary in > the > original map. This is good idea (ROK suggested this in his frames > paper) > since we don't want too accidentally create a new key due to a > spelling > error. This is the crash-early property of the := update syntax. > > Crashing later would make debugging difficult, we would accidentally > add > a bad key to a map and learn about it way later. > > The update > > M # {foo := 12, bar := 24} > > Will succeed, but more importantly the new map has exactly the same > keys as the old map (since all the updates are ':=' updates) - and so > can *share* the same key descriptor. So if we have a very long list of > maps they can be stored in a space efficient manner. (Again this idea > comes from ROKs frames paper). Bj?rn-Egil's eep didn't mention this > but the fact that we know that two maps have the same keys from the > syntax make a lot of optimizations possible). > > All of this is possible because there are two operators not one :-) > Exactly and that is very, very useful. Being used to other languages that have maps but don't differentiate between those two use cases, I regularly find myself writing code like: if key in map then do this else do that So being able to differentiate between both behaviours by just using the right operator for what I want to do at the time is nice. > > --- > > As regards efficiency, utility, beauty and so on these are subjective. > > If you want the last ounce of efficiency records and dicts are not > going to go away when maps arrive. So if maps have the wrong > performance characteristics then use the exiting mechanisms. > > In the latest addition of my book I've been documenting the changes > to maps > - this chapter has changed three times and has tended to be > conservative > so I haven't (yet) mentioned that keys can be any term (and not just > atoms). > > I'm rather looking forward to being able to represent things like XML > and JSON and property lists in a maps and to have an one-size-fits-all > replacement for dicts and records. I've never really worried about the > last ounce of efficiency - if I want real efficiency I'd change > language and go to C or program an FPGA. > I'd debate that. As you explain very well in chapter 7 of your book, Erlang is very good at network protocol processing because of its bit syntax. For simple protocols, I'd really want to keep it lightweight and use something efficient like records. Records also allow me to say: this data has a fixed structured format, don't add random keys to it. On the other hand, if I'm processing JSON or XML, I'd want the whole power of maps. Now if I can have maps behave as efficiently as records for simple structures while giving me a single syntax to deal with both simple and complex structures then I'm all for it. Cheers, Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Tue May 14 12:28:15 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Tue, 14 May 2013 13:28:15 +0300 Subject: [erlang-questions] How to capitalize the first letter of Unicode string? In-Reply-To: <87hai6szkn.fsf@pingviini.dyndns.org> References: <51916659.10309@gmail.com> <87hai6szkn.fsf@pingviini.dyndns.org> Message-ID: <519211BF.5010901@gmail.com> Thank you. It will do the trick. Martin On 5/14/2013 12:03 PM, Jani Hakala wrote: > Martin Dimitrov writes: > >> Hello, >> >> Ca you suggest a way to capitalize the first letter of a Unicode string? >> > Perhaps this library will help you, see > > https://github.com/erlang-unicode/i18n > https://github.com/erlang-unicode/i18n/blob/master/doc/i18n_string.md > > It has functions like split, to_lower, to_upper, to_title. > > Jani Hakala > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From karol.skocik@REDACTED Tue May 14 12:32:09 2013 From: karol.skocik@REDACTED (karol skocik) Date: Tue, 14 May 2013 12:32:09 +0200 Subject: [erlang-questions] Mnesia wildcard query In-Reply-To: <001a01ce507f$43261140$c97233c0$@gmail.com> References: <001a01ce507f$43261140$c97233c0$@gmail.com> Message-ID: You can use ets:fun2ms/1 to construct example match spec, which can be also used for select over mnesia table: 3> Tab = ets:new(tab, []). 20496 4> ets:insert(Tab, [{"abcdef", 1}, {"abxxxxx", 2}, {"bbbbbb", 3}]). true 5> MatchSpec = ets:fun2ms(fun ({[$a, $b | _], V}) -> V end). [{{[97,98|'_'],'$1'},[],['$1']}] 6> ets:select(Tab, MatchSpec). [1,2] ets:fun2ms/1 does not recognize literal funs like: fun ({"prefix" ++ _, V) -> V end, (++ is the problem) so it's more convenient to write a simple wrapper function returning the MatchSpec, given the prefix. Cheers, Karol On Tue, May 14, 2013 at 10:44 AM, Janos Hary wrote: > Hi,**** > > ** ** > > Is there a way to specify a ?begins with? style query in Mnesia? My keys > are strings, and I?d like to get all of them sharing a given prefix.**** > > ** ** > > Thanks,**** > > Janos**** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 14 13:15:32 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 May 2013 13:15:32 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> Message-ID: <51921CD4.4000706@ninenines.eu> On 05/14/2013 05:43 AM, Richard A. O'Keefe wrote: >> To be perfectly honest I don't think that for most record uses this is going to be much of a problem. There's plenty of #state{} records in user applications that aren't accessed enough to make this significant. >> >> And if you wait 10 years, any performance difference will be insignificant... > > The frames proposal has been around for 10 years already. > If I wait 10 years, it is quite likely that nothing will happen. I would suggest not waiting, then. > Machines are not getting faster these days, so I'm not sure why > waiting would make the performance difference insignificant. That's simply not true. Clock rate doesn't increase, but performance continues getting better. Core i7 CPUs (and Xeon equivalents) are a huge improvement over the previous generation, even for single-threaded operations. >>> I would agree for key/value structures, but I fear that by including lists and non key/value tuples you're going to make this much too complex. > > Single mechanism. ONE thing to understand. > Proven technology: people using other languages with this approach > don't seem to have any problem. You don't say what language though, so I'm assuming it's obscure languages and the technology has only been proven with a small population, probably mostly academic. The only thing Google returns me is Perl's Data::Deep which has a function applyPatch which is unreadable. If there is a fool proof solution I'm all for it, but I can't really envision a readable universal syntax for something like this. (And this is the part where you say that nobody needs syntax, they only need functions, I assume.) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Tue May 14 13:38:11 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 May 2013 13:38:11 +0200 Subject: [erlang-questions] Maps In-Reply-To: <519130AC.8090303@erlang.org> References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> Message-ID: <51922223.3040308@ninenines.eu> On 05/13/2013 08:27 PM, Bj?rn-Egil Dahlberg wrote: > - Deep updates of Maps are not covered by this EEP. We would welcome > suggestions on this. I think what Kenneth had in his talk was good. But Simple update: M#{ K => V } Nested update: M#{ K }#{ DK => DV } I personally would prefer a more powerful: M#{ K #{ DK => DV }} Because it allows you to write things like: M#{ K #{ DK => DV }, K2 => V2 } And update multiple levels at once. I know K can be a map, but deep update only works on values, the same way normal update only works on values, so no confusion is possible, we're only updating values, not K itself. If you feel an operator is needed it can be introduced between K and # in the two examples above. Programmatically this would translate as extracting the map found at key K, and ensuring it's a map, updating this map with DK => DV, then placing this map back into the key K in map M along with setting V2 in key K2. It doesn't sound hard to implement in the compiler, it's just unrolling things for access and rolling back again for the actual update. Exactly what we do manually today. It should also be easy to compile to an optimized deep update with this syntax as you got everything in a single expression. Note: If you are not interested in it despite how simple it is please at least ensure it can be done with a parse transform. Thoughts? -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jon@REDACTED Tue May 14 13:43:56 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 14 May 2013 12:43:56 +0100 Subject: [erlang-questions] Maps In-Reply-To: <51921CD4.4000706@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> <51921CD4.4000706@ninenines.eu> Message-ID: > That's simply not true. Clock rate doesn't increase, but performance > continues getting better. Core i7 CPUs (and Xeon equivalents) are a huge > improvement over the previous generation, even for single-threaded > operations. Some would use the word slight rather than huge. Available processing power is no excuse to throw it down the toilet. Jon From jesper.louis.andersen@REDACTED Tue May 14 13:49:14 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 14 May 2013 13:49:14 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> <51921CD4.4000706@ninenines.eu> Message-ID: <879AFE1D-9EB2-47DC-801E-6ED00D89BD76@erlang-solutions.com> On May 14, 2013, at 1:43 PM, "Jon Schneider" wrote: >> That's simply not true. Clock rate doesn't increase, but performance >> continues getting better. Core i7 CPUs (and Xeon equivalents) are a huge >> improvement over the previous generation, even for single-threaded >> operations. > > Some would use the word slight rather than huge. > > Available processing power is no excuse to throw it down the toilet. > One problem with more processing power is that we have a memory bandwidth bottleneck as well and we can't move data from the memory banks onto the CPU at ease. Optimizing for tighter data representations helps these days. Perhaps more than gaining a clock cycle here and there. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From lcastro@REDACTED Tue May 14 14:00:45 2013 From: lcastro@REDACTED (Laura M. Castro) Date: Tue, 14 May 2013 14:00:45 +0200 Subject: [erlang-questions] ACM SIGPLAN Erlang Workshop 2013 Second Call For Papers Message-ID: Hello, Please find below the Second Call for Papers for the Twelfth ACM SIGPLAN Erlang Workshop. Apologies for any duplicates you may receive. CALL FOR PAPERS ================= Twelfth ACM SIGPLAN Erlang Workshop ----------------------------------------------------------- Boston, Massachusetts, September 28, 2013 (tentative date, subject to change) Satellite event of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP 2013) September 25-27, 2013 Erlang is a concurrent, distributed functional programming language aimed at systems with requirements of massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for 15 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite three types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages. Note that this is a maximum length; we welcome shorter papers also, and the program committee will evaluate all papers on an equal basis independent of their lengths. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Chair ----------------------- Steve Vinoski, Basho Technologies, USA Program Chair ------------------- Laura M. Castro, University of A Coru?a, Spain Program Committee ----------------------------- (Note: the Workshop and Program Chairs are also committee members) Lars-Ake Fredlund, Universidad Polit?cnica de Madrid, Spain Kevin Hammond, University of St. Andrews, UK Torben Hoffman, Erlang Solutions Limited, UK Zolt?n Horv?th, E?tv?s Lor?nd University, Hungary Kenneth Lundin, Ericsson AB, Sweden Micka?l R?mond, ProcessOne, France Kenji Rikitake, Basho Japan KK, Japan Simon Thompson, University of Kent, UK Important Dates ----------------------- Submission deadline: Fri June 14, 2013 Author notification: Thu July 11, 2013 Final submission for the publisher: Thu July 25, 2013 Workshop date (tentative, subject to change): September 28, 2013 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2013" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2013 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2013 web site at: http://icfpconference.org/icfp2013/ Related Links -------------------- ICFP 2013 web site: http://www.icfpconference.org/icfp2013/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/account/signin.cgi?conf=erlang2013 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm -- Laura M. Castro From amindfv@REDACTED Tue May 14 14:51:08 2013 From: amindfv@REDACTED (amindfv@REDACTED) Date: Tue, 14 May 2013 08:51:08 -0400 Subject: [erlang-questions] Maps In-Reply-To: <879AFE1D-9EB2-47DC-801E-6ED00D89BD76@erlang-solutions.com> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> <51921CD4.4000706@ninenines.eu> <879AFE1D-9EB2-47DC-801E-6ED00D89BD76@erlang-solutions.com> Message-ID: <9DC35FD4-FC17-445D-8BE5-BE409B0AF9D4@gmail.com> El May 14, 2013, a las 7:49 AM, Jesper Louis Andersen escribi?: > > On May 14, 2013, at 1:43 PM, "Jon Schneider" wrote: > >>> That's simply not true. Clock rate doesn't increase, but performance >>> continues getting better. Core i7 CPUs (and Xeon equivalents) are a huge >>> improvement over the previous generation, even for single-threaded >>> operations. >> >> Some would use the word slight rather than huge. >> >> Available processing power is no excuse to throw it down the toilet. > +1 > One problem with more processing power is that we have a memory bandwidth bottleneck as well and we can't move data from the memory banks onto the CPU at ease. Optimizing for tighter data representations helps these days. Perhaps more than gaining a clock cycle here and there. > +1 Tom > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Tue May 14 15:07:09 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 14 May 2013 14:07:09 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: <518D93AF.5030502@gmail.com> Message-ID: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> One thing that has always puzzled me in these discussions is this fascination with *NATIVE* implementations. Why *native*? What is it with *native* that makes people want it? Is it the speed? Or is it the special syntax? Or what? Why care HOW things are done just as long as it fulfils my requirements. When I want a fast dictionary then FAST is the requirement, same with ORDERED or BAGS or ... . How it is done I don't care. If a special syntax is helpful then great, it is the syntax I am after. People complain that you need a special data type or else you can get them mixed up. I don't buy that. I work with lists, tuples, proplists, orddicts, dicts, gb_trees, records, etc together in one application and one problem I don't have is getting them mixed up. I see it as a solution to a problem which is at worst only very small. Michael, please don't the idea that I am coming down on you specifically here. I am not. You just had the misfortune to mention this near the top where I could easily see it. And it is something I have been pondering about for a long time. It is one of the classic complaints about Erlang (after ;, if, variables, ...), why isn't there a built-in map type. One of the ideas behind dicts/orddicts was to provide a standardised API which would allow you to choose and easily change which implementation you need/want. Unfortunately we never wrote this down anywhere and only two different versions exist (though I have a ttdict based on 2-3 trees if anyone needs a sorted tree, very fast too). While this forced you to think it would give you lots of options. One of the bad things of having a built-in type is that you don't have think, but you won't always get the most suitable. And seriously thinking about your data structures is always a Good Thing. Sorry for the rant here. What *I* would like is: - A dynamic record-like alternative which I call rr (Robert's records) to avoid getting into a discussion about what frames are/aren't. - A really fast dict implementation. I don't care how it's done but speed is the main requirement. If it represented as a built-in type or a tree of tuples/lists I don't care, just as long as it is fast. Lo?c you and I can meet at dawn like gentlemen to discuss special syntax or not. Swords or pistols? :-) Robert ----- Original Message ----- > From: "Michael Truog" > > I am a bit disturbed by the concern over Frames. Since I started > using Erlang I have always wanted native, efficient dictionary > functionality as an actual type within the language (not just a dict > module, despite the fact dict has one of the best possible > implementations within native Erlang source code). It is a major > disappointment coming to the Erlang language, being given the > process dictionary, and then being told that you should never use > it, since it is dirty mutable state that ruins our pure referential > transparency.. So, the native dictionary type, Maps, seems long > (long) overdue when Erlang is compared with newer languages like > Python, Ruby., Perl and Java. > > Having a newer and better type of records named Frames seems like an > opportunity to take something decent and make it slower (though I > understand the extreme effort and attention to make performance > close). The argument is hopefully not: "But wait, we have neat > syntax which allows us to make neat examples, and they might be > useful.... and now everyone should use Frames in their code instead > of records", since that seems least desirable. I think it should be > hard to make something more efficient than a simple array index > lookup, which seems to be the basic idea behind a tuple (with the > record syntactic sugar). I don't necessarily care about the records > syntax being this preprocessor trick since that is made clear within > the documentation. I think the idea that Frames provide a record > alternative relies on the benefits of Frames, which appear too > minimal because they seem to be focusing on improving the type > system, more than extending functionality (so the concern looks > more academic than pragmatic). One of the basic benefits of Frames > that has been mentioned is that the type can be used explicitly for > external data, however an external API could easily wrap the > creation of an atom and tuple, so that justification of Frames > doesn't feel very compelling (though the Frame type would make a > record/tuple object more naturally a C struct-like type within > Erlang). > > I understand often performance is not a concern, but I can't help but > think it should always be the most important concern for compiler > writers and VM writers (at the very least, not to mention the > database writers, the message bus writers, and the algorithm library > writers), which is why I believe it carries so much importance to > the decision of the implementation of Maps or Frames. So, I am very > happy with the pursuit of the Map type, while the pursuit of the > Frame type seems inappropriate (in the sense of a feature triage). > From max.lapshin@REDACTED Tue May 14 15:41:15 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 14 May 2013 17:41:15 +0400 Subject: [erlang-questions] Maps In-Reply-To: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: I think, that first problem is a convenient syntax for key-value structure. Syntax that will also work for pattern matching, not also editing existing structure. Second problem is a distinguishing between tuples and this special type. Many of us have problems trying to understand: is this a list of integers or a string? How should I pack it to json? As a result, many modern pieces of software are migrating to binary-only. From dgud@REDACTED Tue May 14 17:17:11 2013 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 14 May 2013 17:17:11 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: To summon up this thread with my view. We want both Frames and Dictionaries. Frames for it's small memory footprint and fast access, can replace records except in the most time critical places. Dictionaries for it's arbitrary key terms and the ability to handle many objects. Matching is nice so we want have syntax to use them both, we do not want to introduce two new syntax's in the language because that will make it harder to read and learn. Fred Hebert > #{_ := V} = #{a => 3}, > and variants. Value searching will not be allowed. Richard A. O'Keefe > Frames > Maps are basically an extension of Frames which allows for arbitrary key terms. Which could use your implementation proposal for small maps defined in source code, and switches to a tree (or something else) for larger dynamic sized Maps. >- Frames are limited to atom keys in the interests of > compile-time error detection (both simply noticing that > a key is not an atom at all and allowing the possibility > of Dialyzer support). So use := and you will get a runtime/compile check that the key exists. >- Frames have O(lg N) access to a single field or O(N/k) access > to k fields. They have O(N/k) cost for copying a frame with > k fields changed. You will still have that for small maps. Robert Virding > Use only matching We would like to but we believe we can not do that because we can't output a defined order, i.e. you can not sort 1.0 and 1. We would need to define a new term order in erlang and we need to introduce something like <:<, >:> =:< and >:=. /Dan On Tue, May 14, 2013 at 3:41 PM, Max Lapshin wrote: > I think, that first problem is a convenient syntax for key-value > structure. Syntax that will also work for pattern matching, not also > editing existing structure. > > Second problem is a distinguishing between tuples and this special > type. Many of us have problems trying to understand: is this a list of > integers or a string? How should I pack it to json? As a result, many > modern pieces of software are migrating to binary-only. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From colanderman@REDACTED Tue May 14 17:25:28 2013 From: colanderman@REDACTED (Chris King) Date: Tue, 14 May 2013 11:25:28 -0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: +1 for "Robert's records". I do NOT want to use a dictionary to store record data, for the same reason I don't want to use a list to store tuple data -- it is not the correct concept. All compound data structures are one of two kinds -- heterogeneous and static, or homogeneous and dynamic. By "heterogeneous and static", I mean that the components are potentially of differing types (heterogeneous), and that the form of the structure as a whole is determinable at compile time (static). This includes tuples, records, "Robert's records", frames, and functional objects (as they are known in OCaml). By "homogeneous and dynamic", I mean that the components are necessarily of the same type (homogeneous), and that the form of the structure as a whole is not necessarily determinable at compile time (dynamic). This includes lists, binaries, maps, and set- or map-like ADTs. The other two combinations -- heterogeneous and dynamic, and homogeneous and static -- are rather useless. The latter is merely a degenerate case of both the above kinds, and the former is almost a definite code smell (if you can't describe your data, how can you operate on it?). Maps, as presented, are heterogeneous and dynamic. I claim this is folly: If my data is heterogeneous and static, what use have I for non-atom keys, and why must I be bound by an explicit ordering of keys(!?)? If my data is homogeneous and dynamic, why must I pay the storage cost of identifying the type of each element, and preclude my typechecker from ensuring the homogeneity of my data? "Robert's records" / frames, on the other hand, are heterogeneous and static. They lend themselves well to type-checking (see OCaml's functional objects), and (IMPORTANTLY -- many seem to miss this!) they provide SUBTYPING OF RECORDS. This is HUGE (I would say the entire purpose of frames!), NOT merely syntactic, and importantly -- maps don't provide this! thanks to the above-mentioned ordering of keys. I personally don't care about a syntax for homogeneous dynamic data -- syntax is useless unless it provides additional expressivity, and I don't see that the current proposal does for this case. But if others do, all I ask is PLEASE don't present this syntax as a solution to the record subtyping problem, as it is not for reasons outlined above -- and I suspect that a "maps" proposal *ignoring* the record subtyping use case will result in a more efficient implementation. On Tue, May 14, 2013 at 9:41 AM, Max Lapshin wrote: > I think, that first problem is a convenient syntax for key-value > structure. Syntax that will also work for pattern matching, not also > editing existing structure. > > Second problem is a distinguishing between tuples and this special > type. Many of us have problems trying to understand: is this a list of > integers or a string? How should I pack it to json? As a result, many > modern pieces of software are migrating to binary-only. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rustompmody@REDACTED Tue May 14 17:30:33 2013 From: rustompmody@REDACTED (Rustom Mody) Date: Tue, 14 May 2013 21:00:33 +0530 Subject: [erlang-questions] ANN: Mooc course on functional programming Message-ID: Hi! We've floated this Mooc course on functional programming https://moocfellowship.org/submissions/the-dance-of-functional-programming-languaging-with-haskell-and-python I'd like to point out that there is an elementary core to functional programming: http://blog.languager.org/2012/10/functional-programming-lost-booty.html which is more or less common to most modern functional languages. Although haskell and python are named as two of the technologies that we will use as our medium, one of the aims of the course is to highlight this language-independent and common core. More details : http://blog.languager.org/2013/05/dance-of-functional-programming.html Rusi -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Tue May 14 17:38:47 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 14 May 2013 08:38:47 -0700 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: > The other two combinations -- heterogeneous and dynamic, and > homogeneous and static -- are rather useless. The latter is merely a > degenerate case of both the above kinds, and the former is almost a > definite code smell (if you can't describe your data, how can you > operate on it?). While I agree to some of your points, I don't quite agree here. For example, during a code upgrade, one may want to add a new field to the data (heterogeneous). With static data structures, this upgrade is trickier as it requires the origin and destination to have the correct number of fields or it won't work. With a dynamic structure, no state migration is necessary. In addition, just because a feature could cause code smell in certain circumstances doesn't mean it should be immediately excluded. Erlang provides list_to_atom (with appropriate caution) because occasionally, it is the right thing to do (convenience in small scripts for example). I think a dynamic heterogeneous structure can provide tremendous value. Take JSON for example. It's used everywhere, and people import JSON into Erlang as a bunch of nested keylists. This is the heterogeneous dynamic structure you are talking about and it is very clunky to work with! Using nested keyfinds, keystores, and keydeletes is extremely clunky and I feel forced to "take an object oriented approach" by creating modules providing getters and setters for each of these values. I hate doing this personally but it's better than the alternative. Cheers, Jeremy On Tue, May 14, 2013 at 8:25 AM, Chris King wrote: > +1 for "Robert's records". > > I do NOT want to use a dictionary to store record data, for the same > reason I don't want to use a list to store tuple data -- it is not the > correct concept. > > All compound data structures are one of two kinds -- heterogeneous and > static, or homogeneous and dynamic. > > By "heterogeneous and static", I mean that the components are > potentially of differing types (heterogeneous), and that the form of > the structure as a whole is determinable at compile time (static). > This includes tuples, records, "Robert's records", frames, and > functional objects (as they are known in OCaml). > > By "homogeneous and dynamic", I mean that the components are > necessarily of the same type (homogeneous), and that the form of the > structure as a whole is not necessarily determinable at compile time > (dynamic). This includes lists, binaries, maps, and set- or map-like > ADTs. > > > The other two combinations -- heterogeneous and dynamic, and > homogeneous and static -- are rather useless. The latter is merely a > degenerate case of both the above kinds, and the former is almost a > definite code smell (if you can't describe your data, how can you > operate on it?). > > Maps, as presented, are heterogeneous and dynamic. I claim this is > folly: If my data is heterogeneous and static, what use have I for > non-atom keys, and why must I be bound by an explicit ordering of > keys(!?)? If my data is homogeneous and dynamic, why must I pay the > storage cost of identifying the type of each element, and preclude my > typechecker from ensuring the homogeneity of my data? > > "Robert's records" / frames, on the other hand, are heterogeneous and > static. They lend themselves well to type-checking (see OCaml's > functional objects), and (IMPORTANTLY -- many seem to miss this!) they > provide SUBTYPING OF RECORDS. This is HUGE (I would say the entire > purpose of frames!), NOT merely syntactic, and importantly -- maps > don't provide this! thanks to the above-mentioned ordering of keys. > > I personally don't care about a syntax for homogeneous dynamic data -- > syntax is useless unless it provides additional expressivity, and I > don't see that the current proposal does for this case. But if others > do, all I ask is PLEASE don't present this syntax as a solution to the > record subtyping problem, as it is not for reasons outlined above -- > and I suspect that a "maps" proposal *ignoring* the record subtyping > use case will result in a more efficient implementation. > > On Tue, May 14, 2013 at 9:41 AM, Max Lapshin > wrote: > > I think, that first problem is a convenient syntax for key-value > > structure. Syntax that will also work for pattern matching, not also > > editing existing structure. > > > > Second problem is a distinguishing between tuples and this special > > type. Many of us have problems trying to understand: is this a list of > > integers or a string? How should I pack it to json? As a result, many > > modern pieces of software are migrating to binary-only. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From janos.n.hary@REDACTED Tue May 14 18:01:23 2013 From: janos.n.hary@REDACTED (Janos Hary) Date: Tue, 14 May 2013 18:01:23 +0200 Subject: [erlang-questions] Mnesia wildcard query In-Reply-To: References: <001a01ce507f$43261140$c97233c0$@gmail.com> Message-ID: <006901ce50bc$490c7530$db255f90$@gmail.com> Thanks for the answer. As far as I understand it works great if the prefix is known at compile time. I wasn't clear on this, but in my case the prefix is given in runtime. So far I come up with this solution. It understands a '*' character at a pattern's end. I'd be happy to get suggestions to improve it. %% finds '*' in pattern and return a minimum and maximum pattern %% e.g. <<"hello*">> -> {<<"hello">>, <<"hellp">>} %% if binary string X starts with 'hello' then <<"hello">> <= X < <<"hellp">> wildcard(Pattern) when is_binary(Pattern) -> case binary:match(Pattern, <<"*">>) of nomatch -> {Pattern, Pattern}; {0, _} -> {error, wrong_pattern}; {S, _} -> PatPref = binary:part(Pattern, 0, S-1), io:format("~s", [PatPref]), <> = binary:part(Pattern, S-1, 1), {<>, <>} end. find(Pat) -> {PatMin, PatMax} = wildcard(Pat), F = fun() -> qlc:eval(qlc:q( [Patient || #patient{search_name = N}=Patient <- mnesia:table(pat), N >= PatMin, N < PatMax] )) end, mnesia:activity(transaction, F, [], mnesia_frag). Regards, Janos From: karol skocik [mailto:karol.skocik@REDACTED] Sent: Tuesday, May 14, 2013 12:32 PM To: Janos Hary Cc: Erlang Questions Subject: Re: [erlang-questions] Mnesia wildcard query You can use ets:fun2ms/1 to construct example match spec, which can be also used for select over mnesia table: 3> Tab = ets:new(tab, []). 20496 4> ets:insert(Tab, [{"abcdef", 1}, {"abxxxxx", 2}, {"bbbbbb", 3}]). true 5> MatchSpec = ets:fun2ms(fun ({[$a, $b | _], V}) -> V end). [{{[97,98|'_'],'$1'},[],['$1']}] 6> ets:select(Tab, MatchSpec). [1,2] ets:fun2ms/1 does not recognize literal funs like: fun ({"prefix" ++ _, V) -> V end, (++ is the problem) so it's more convenient to write a simple wrapper function returning the MatchSpec, given the prefix. Cheers, Karol On Tue, May 14, 2013 at 10:44 AM, Janos Hary wrote: Hi, Is there a way to specify a 'begins with' style query in Mnesia? My keys are strings, and I'd like to get all of them sharing a given prefix. Thanks, Janos _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue May 14 18:22:46 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 14 May 2013 09:22:46 -0700 Subject: [erlang-questions] Maps In-Reply-To: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> References: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: <519264D6.2050305@gmail.com> Comments below: On 05/14/2013 06:07 AM, Robert Virding wrote: > One thing that has always puzzled me in these discussions is this fascination with *NATIVE* implementations. Why *native*? What is it with *native* that makes people want it? Is it the speed? Or is it the special syntax? Or what? Why care HOW things are done just as long as it fulfils my requirements. When I want a fast dictionary then FAST is the requirement, same with ORDERED or BAGS or ... . How it is done I don't care. If a special syntax is helpful then great, it is the syntax I am after. People complain that you need a special data type or else you can get them mixed up. I don't buy that. I work with lists, tuples, proplists, orddicts, dicts, gb_trees, records, etc together in one application and one problem I don't have is getting them mixed up. I see it as a solution to a problem which is at worst only very small. I think my concern about whether the implementation is native (Erlang-only source code) or not relates to the performance characteristics. It seems as if we need C integration to get faster than dict on a wide variety of key types. I agree that it is best to have many different types of dictionary implementations, and ideally we would have a bunch of dictionaries implemented in C also :-) However, I understand that more testing and considerations are necessary to make the C integration decent performance-wise and fault-tolerance-wise. Basically, what seems required, is performance similar to the process dictionary, or better, as a separate type, called Maps. > > Michael, please don't the idea that I am coming down on you specifically here. I am not. You just had the misfortune to mention this near the top where I could easily see it. And it is something I have been pondering about for a long time. It is one of the classic complaints about Erlang (after ;, if, variables, ...), why isn't there a built-in map type. > > One of the ideas behind dicts/orddicts was to provide a standardised API which would allow you to choose and easily change which implementation you need/want. Unfortunately we never wrote this down anywhere and only two different versions exist (though I have a ttdict based on 2-3 trees if anyone needs a sorted tree, very fast too). While this forced you to think it would give you lots of options. One of the bad things of having a built-in type is that you don't have think, but you won't always get the most suitable. And seriously thinking about your data structures is always a Good Thing. I agree, and I wish OTP/Erlang was able to add more data structures like your rbtree to compliment the gbtree, and possibly resolving any differences between the aatree and the gbtree. So, I think having more data structures is more beneficial than having less, since it gives developers more options. Though I understand these decisions are probably conservative and slow moving. Please release the ttdict! Have been wanting to see that one for awhile. > > Sorry for the rant here. > > What *I* would like is: > > - A dynamic record-like alternative which I call rr (Robert's records) to avoid getting into a discussion about what frames are/aren't. > > - A really fast dict implementation. I don't care how it's done but speed is the main requirement. If it represented as a built-in type or a tree of tuples/lists I don't care, just as long as it is fast. I agree a record-like alternative would be nice. I just see it as less important than the fast dictionary implementation. So, if given the choice between the two, I am happy we can get the fast dictionary implementation. The record-like alternative is supposedly most important for external term representation, since it is a separate type. Normally, usage of records as external terms are a type of message format used in communication with ports or other things (cnodes, or possibly port drivers). So, perhaps what we want is something a bit different from the concept of a "record" but rather a "message" type. A message could just be a generic container that defines a name, a header, and a body, where the header is normally empty (the header is always key->value) and the name is always an atom (or possibly atom or binary/string, to avoid excessive atom consumption). The body could be a binary that is accessed efficiently based on key->values where each key maps to an index. I will admit this idea is coming from a different angle, so it isn't well-aligned with the previous discussions, but it seems like trying to replace records as they are typically used is futile (unless we just make a type specific to tuples with an atom to keep the same performance, and satisfy both concerns). Either way, it seems important to focus on what the record-like alternative is suppose to solve and the problems it is trying to solve. In the past, the discussion of frames seems to have gotten bogged down in syntax discussions, and convenience, but whatever record-like alternative which is pursued seems to require clearly stated benefits. So far, the main one I have seen is just that there is a way to represent records as external terms, and that reason isn't entirely compelling when you first see it. Simplifying debugging and development is probably an obvious goal, but to really do that, requires the same performance to replace records usage, so then you get back to replicating a tuple atom combo as a separate type, instead of focusing on a new type with new characteristics. > > Lo?c you and I can meet at dawn like gentlemen to discuss special syntax or not. Swords or pistols? :-) > > Robert > > > > ----- Original Message ----- >> From: "Michael Truog" >> >> I am a bit disturbed by the concern over Frames. Since I started >> using Erlang I have always wanted native, efficient dictionary >> functionality as an actual type within the language (not just a dict >> module, despite the fact dict has one of the best possible >> implementations within native Erlang source code). It is a major >> disappointment coming to the Erlang language, being given the >> process dictionary, and then being told that you should never use >> it, since it is dirty mutable state that ruins our pure referential >> transparency.. So, the native dictionary type, Maps, seems long >> (long) overdue when Erlang is compared with newer languages like >> Python, Ruby., Perl and Java. >> >> Having a newer and better type of records named Frames seems like an >> opportunity to take something decent and make it slower (though I >> understand the extreme effort and attention to make performance >> close). The argument is hopefully not: "But wait, we have neat >> syntax which allows us to make neat examples, and they might be >> useful.... and now everyone should use Frames in their code instead >> of records", since that seems least desirable. I think it should be >> hard to make something more efficient than a simple array index >> lookup, which seems to be the basic idea behind a tuple (with the >> record syntactic sugar). I don't necessarily care about the records >> syntax being this preprocessor trick since that is made clear within >> the documentation. I think the idea that Frames provide a record >> alternative relies on the benefits of Frames, which appear too >> minimal because they seem to be focusing on improving the type >> system, more than extending functionality (so the concern looks >> more academic than pragmatic). One of the basic benefits of Frames >> that has been mentioned is that the type can be used explicitly for >> external data, however an external API could easily wrap the >> creation of an atom and tuple, so that justification of Frames >> doesn't feel very compelling (though the Frame type would make a >> record/tuple object more naturally a C struct-like type within >> Erlang). >> >> I understand often performance is not a concern, but I can't help but >> think it should always be the most important concern for compiler >> writers and VM writers (at the very least, not to mention the >> database writers, the message bus writers, and the algorithm library >> writers), which is why I believe it carries so much importance to >> the decision of the implementation of Maps or Frames. So, I am very >> happy with the pursuit of the Map type, while the pursuit of the >> Frame type seems inappropriate (in the sense of a feature triage). >> From essen@REDACTED Tue May 14 18:38:38 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 14 May 2013 18:38:38 +0200 Subject: [erlang-questions] Maps In-Reply-To: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> References: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: <5192688E.5090107@ninenines.eu> On 05/14/2013 03:07 PM, Robert Virding wrote: > Lo?c you and I can meet at dawn like gentlemen to discuss special syntax or not. Swords or pistols? :-) Pistols. I don't disagree with most of what you say. The special syntax isn't really needed until you get a project where you need to keep really many things in the state (hundreds if not thousands of terms per connection) and update them many times, in my case many times per second, while still striving for low latency. When you have that many values, your first enemy is verbosity, not only to be able to write the code fast, but also to find the bugs in it quicker. Execution speed only comes after. Of course by verbosity I don't mean saving a character or two, I mean that what the code says is simpler. For example, why would you want to say "I want to extract this value V1 then extract this value V2 then modify a value in V2 then put V2 back in V1 then put V1 in the original structure" instead of "I want to modify this deep value there"? We shouldn't have to worry about all these intermediate operations. They're a lot longer to write, and a bigger source of error. My personal use case is probably not that universal, but I believe it also applies to anyone who has to access or modify JSON or equivalent (not XML). How do you access deep values in JSON in Erlang? How do you update a value in JSON? How do you update 20 of them? Painfully. You can write functions to do it if you don't need to do it many times in your program, but the more you need to do it the more you'll want a special syntax to actually get things done instead of handling the details of the modification. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From colanderman@REDACTED Tue May 14 19:37:15 2013 From: colanderman@REDACTED (Chris King) Date: Tue, 14 May 2013 13:37:15 -0400 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: On Tue, May 14, 2013 at 12:25 PM, Jeremy Ong wrote: >> Code upgrades involve only invoking new functions from old functions >> -- these are syntactically different functions and hence may have >> different static types. > > What about running processes? How does a gen_server upgrade work? The state > isn't explicitly mutated but it does need to transition. The old loop() function invokes the new upgrade() function, which invokes the new loop() function. The two loop() functions are separate functions and hence may have different types. There's no magic or hand-waving involved. >> Yes, and OCaml provides Obj.magic (C-style type cast). However it, >> like list_to_atom, are *explicit* -- I know immediately by its >> presence that something very weird is going on which warrants closer >> inspection, and its uses are almost *always* at I/O boundaries. >> > C++ is an obvious extreme of a language featuring many things that could > involve "code smell" but each of which may be situationally useful. Again, > this is an extreme example but you get the idea. Extreme indeed ;) However if feature X is useful only in certain situations but is a code smell in most others, then I claim that feature X is poorly designed. C++ templates come to mind -- yes, they are useful for massive abstraction, but they provide enough means of abuse that one should consider them a poorly designed feature. (OCaml's module system is a superb example of a feature intended for massive abstraction that makes doing "bad" things nigh impossible.) >> JSON is not dynamic. I must know exactly what fields exist in any >> JSON object I'm working with, or I'm doing something wrong. (This >> design pattern is evinced by the fact that JSON keys may not be >> arbitrary values.) > > Perhaps I misunderstand what you mean by the term dynamic. I should be able > to access the field "foo" or 5 inside a json object regardless of whatever > other crap is there. Static structure does not preclude subtyping. OCaml's functional objects capture this notion perfectly -- an object has a set of methods which are known at compile time (hence static). However any given function need only be aware of the presence of the subset of methods in which it is interested. Note that this does not preclude a function from creating a copy of said object with only those methods modified! By "dynamic" I mean not determinable at compile time: e.g. the length of a list is generally not knowable at compile time, as the point of a list is expressly to sequence an arbitrary number of items. Not knowing which members are present in a structure renders the structure nigh useless -- if a structure is missing some data my code expects it to contain, then it is for all purposes broken. (Note that using the absence of a member to indicate a "null" or default value is folly -- how am I to distinguish between code wishing to use a default value, and code which incorrectly did not specify a value due to a typo?) Naturally we encounter a problem at the I/O boundaries of our program -- if we are reading JSON data from the network, we can in no way guarantee it will contain (a superset of) the fields we expect. But this is exactly where input sanitation must take place: if we blindly accept foreign data, we open ourselves up to security risks (viz. list_to_atom). To ensure correct and secure operation, we must explicitly sanitize the homogeneous dynamic data from the network (via parsing) to obtain heterogeneous static data (via construction). Of course we may permit the function "binary_to_frame" as a shortcut, but, like list_to_atom and the infamous eval, we lose the ability to typecheck the result (unless we decorate with assertions, which fortunately Erlang provides for), and we open up security holes. > Yea I'm wrong here, for some reason, I thought JSON keys could be integers > as well. IIRC Javascript object keys can be (this is how arrays work), perhaps this is what you were recalling? >> The problem you describe is one of data interface. Assume for a >> moment that the network does not exist, and all JSON data lives purely >> within Erlang. JSON data may then happily be represented as a >> heterogeneous static structure (e.g. a frame) rather than a >> homogeneous dynamic one (e.g. a keylist). > > A better example would be the interface between Erlang and Ruby. Ruby hashes > are heterogeneous and dynamic and allow for integer, atom, or string keys. > If I want full interoperability between them, I will need to use a > heterogeneous structure. In more strict statically typed languages (e.g. the ML family), this issue is resolved by tagged unions -- the type of a Ruby key could be "{int, integer()} | {atom, atom()} | {str, string()}". This preserves type-ability at the cost of (admittedly annoying) extra keystrokes. Erlang, being dynamically typed, maintains an implicit tag for each type, the full union of which is denoted "any()"; hence the above use case may be supported by claiming that the Ruby dict() maps type any() to type any(). Obviously this only works for maps, not frames -- the reason I argue *for* frames (not against maps! I don't care if there's both!) is precisely because I *don't* want to be able to do this: I want my frames to have static structure, and I want the compiler (or Dialyzer) to kick me in the pants if they don't! From wallentin.dahlberg@REDACTED Tue May 14 19:40:16 2013 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 14 May 2013 19:40:16 +0200 Subject: [erlang-questions] Maps In-Reply-To: <20130508153955.GB79143@ferdair.local> References: <518A5ED1.2030507@erlang.org> <20130508153955.GB79143@ferdair.local> Message-ID: 2013/5/8 Fred Hebert > I was looking at an example: > > 1> M = #{ 1 => a }, > #{ 1 => a } > 2> M#{ 1.0 => b }. > #{ 1.0 => b }. > 3> M#{ 1 := b }. > #{ 1 => b } > 4> M#{ 1.0 := b }. > ** exception error: bad argument > > And given the definition where we have 'match' and 'equal' being two > different things, am I right in understanding that '=>' can both create > a mapping and update it based on 'equal' whereas ':=' can only update > them using 'match' ? > Yes. '=>' is used to associate new key value pairs, or update an existing (*equal*) key. ':=' is used to update an existing (matching) key, or used in patterns in matching. > > I don't think there is any mention of the expected behaviour when > matching on #{_ := V} = #{a => 3}, unless I missed it (there > were cases in map comprehensions, but these are obviously not the same). > Does this need to be specified? Similarly, #{_ := V} = #{a => 1, b => 1} > and #{K := 1} = #{a => 1, b => 1} do not seem specified to me. > No, #{ _ := V } = #{ a => 1} is not allowed. Only keys which are bound by the environment are allowed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From karol.skocik@REDACTED Tue May 14 20:11:22 2013 From: karol.skocik@REDACTED (karol skocik) Date: Tue, 14 May 2013 20:11:22 +0200 Subject: [erlang-questions] Mnesia wildcard query In-Reply-To: <006901ce50bc$490c7530$db255f90$@gmail.com> References: <001a01ce507f$43261140$c97233c0$@gmail.com> <006901ce50bc$490c7530$db255f90$@gmail.com> Message-ID: On Tue, May 14, 2013 at 6:01 PM, Janos Hary wrote: > Thanks for the answer. As far as I understand it works great if the prefix > is known at compile time. I wasn?t clear on this, but in my case the prefix > is given in runtime.**** > > ** > Yes, literal fun as arg to ets:fun2ms is a compile time entity. It's good for getting the first sketch of match spec in shell. > So far I come up with this solution. It understands a ?*? character at a > pattern?s end. I?d be happy to get suggestions to improve it.**** > > ** ** > > %% finds '*' in pattern and return a minimum and maximum pattern**** > > %% e.g. <<"hello*">> -> {<<"hello">>, <<"hellp">>}**** > > %% if binary string X starts with 'hello' then <<"hello">> <= X < > <<"hellp">>**** > > wildcard(Pattern) *when* is_binary(Pattern) ->**** > > *case* binary:match(Pattern, <<"*">>) *of***** > > nomatch -> **** > > {Pattern, Pattern};**** > > {0, _} -> **** > > {error, wrong_pattern};**** > > {S, _} ->**** > > PatPref = binary:part(Pattern, 0, S-1),**** > > io:format("*~s*", [PatPref]),**** > > <> = binary:part(Pattern, S-1, 1),**** > > {<>, < PatLast+1):8>>}**** > > *end*.**** > > ** ** > > find(Pat) ->**** > > {PatMin, PatMax} = wildcard(Pat),**** > > F = *fun*() ->**** > > qlc:eval(qlc:q(**** > > [Patient || #patient{search_name = N}=Patient<- mnesia:table(pat), > **** > > N >= PatMin, N < PatMax]**** > > ))**** > > *end*,**** > > mnesia:activity(transaction, F, [], mnesia_frag).**** > > ** > This is a full scan of the table, and since match spec's first arg can't be a binary with prefix only the full scan can't be avoided. Function you have is pretty much the only way to do what you want. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Tue May 14 20:21:32 2013 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 14 May 2013 20:21:32 +0200 Subject: [erlang-questions] Maps In-Reply-To: <51922223.3040308@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> <51922223.3040308@ninenines.eu> Message-ID: 2013/5/14 Lo?c Hoguin > On 05/13/2013 08:27 PM, Bj?rn-Egil Dahlberg wrote: > >> - Deep updates of Maps are not covered by this EEP. We would welcome >> suggestions on this. >> > > I think what Kenneth had in his talk was good. But > > Simple update: > M#{ K => V } > > Nested update: > M#{ K }#{ DK => DV } > I believe this was a hasty example since it would, i gather, return the inner Map. 1> M = #{ a => #{ b => 1 } }. #{ a => #{ b => 1 } } 2> M#{ a }#{ b := 2 }. #{ b => 2 } > I personally would prefer a more powerful: > M#{ K #{ DK => DV }} > > Because it allows you to write things like: > M#{ K #{ DK => DV }, K2 => V2 } > > And update multiple levels at once. > > I know K can be a map, but deep update only works on values, the same way > normal update only works on values, so no confusion is possible, we're only > updating values, not K itself. If you feel an operator is needed it can be > introduced between K and # in the two examples above. > > Programmatically this would translate as extracting the map found at key > K, and ensuring it's a map, updating this map with DK => DV, then placing > this map back into the key K in map M along with setting V2 in key K2. > > It doesn't sound hard to implement in the compiler, it's just unrolling > things for access and rolling back again for the actual update. Exactly > what we do manually today. > > It should also be easy to compile to an optimized deep update with this > syntax as you got everything in a single expression. > > Note: If you are not interested in it despite how simple it is please at > least ensure it can be done with a parse transform. > > Thoughts? I think we want to be explicit here. Meaning using an operator and "seeing" the value in action. (I have bad memories of perl). Just some first thoughts on deep updates here to see if I get the intention right. What I think we need is someway to use an associated value within the map. I figure, something like "escaping" the key which would fetch the associated value. Let's use '~' in this example. Say I have a nested map like above, M = #{ a => #{ b => 1 }} and I would like to update the inner b value with it self plus four. I could write it like this: 3> M#{ a := ~a#{ b := ~b + 4 }}. #{ a => #{ b => 5 }} Another possibility would perhaps be to bind values first (wrapped in { .. } )and then use them in updates, say M#{ a := M }{ a := M#{ b := V }{ b := V + 4 } } .. that looks even more horrible .. =) I haven't given this much thought but I'm trying to get a sense of it. Something like this? // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Tue May 14 20:49:19 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 14 May 2013 14:49:19 -0400 Subject: [erlang-questions] Allow erl and c() to recognize escript shebangs Message-ID: I like being able to run my Erlang code through escript and erl / c(). Could we work on merging the recognized syntax for these? For example, I think it would be nice to treat shebang lines as comments in erl / c(), in order to reduce incompatibility. -- Cheers, Andrew Pennebaker www.yellosoft.us From freza@REDACTED Tue May 14 21:05:09 2013 From: freza@REDACTED (Jachym Holecek) Date: Tue, 14 May 2013 15:05:09 -0400 Subject: [erlang-questions] Maps In-Reply-To: <5192688E.5090107@ninenines.eu> References: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <5192688E.5090107@ninenines.eu> Message-ID: <20130514190509.GA13738@circlewave.net> # Lo?c Hoguin 2013-05-14: > On 05/14/2013 03:07 PM, Robert Virding wrote: > >Lo?c you and I can meet at dawn like gentlemen to discuss special syntax > >or not. Swords or pistols? :-) > > Pistols. > > I don't disagree with most of what you say. The special syntax isn't > really needed until you get a project where you need to keep really many > things in the state (hundreds if not thousands of terms per connection) > and update them many times, in my case many times per second, while > still striving for low latency. Have you considered ETS tables? A private ordered_set variety would seem to fit your needs rather well, based on what you just said and an example you gave in an earlier post. You mention keeping large state, be mindful this may, inherently, be playing against your low latency goal if you keep all the data on heap. Well, depending on what precisely you mean by "large state" and "low latency". Just briefly -- you mentioned you need to maintain nested objects along with attributes: true = ets:insert_new(Tid, {{Player_id, possesions, Item_id, count}, 1}) There's also ets:{update_counter/X|lookup_element/X}, and ets:select/X with known prefix behaves just nice too. Maybe a list would work better as key type than tuple. Anyway, you get the idea. Also -- you could consider using processes to encapsulate various bits of state. > When you have that many values, your first enemy is verbosity, not only > to be able to write the code fast, but also to find the bugs in it > quicker. Execution speed only comes after. Of course by verbosity I > don't mean saving a character or two, I mean that what the code says is > simpler. True. It may be that a native type would help, it may be that more careful choice of data representation and analysis of data flows in your application would help a lot more (not to come across patronizing here -- I'm reminded of a few messy protocol converters/gateways we did at ${dayjob}). > For example, why would you want to say "I want to extract this > value V1 then extract this value V2 then modify a value in V2 then put > V2 back in V1 then put V1 in the original structure" instead of "I want > to modify this deep value there"? We shouldn't have to worry about all > these intermediate operations. They're a lot longer to write, and a > bigger source of error. > > My personal use case is probably not that universal, but I believe it > also applies to anyone who has to access or modify JSON or equivalent > (not XML). The exact degree of ugliness and dismal design work in all these "modern" HTTP-ish "protocols" is perhaps new, dealing with deeply nested data structures on external interfaces is not... > How do you access deep values in JSON in Erlang? I don't. Usually it ends up being pretty reasonable to convert them to Erlang-friendly internal representations on protocol boundaries (good thing one mostly knows what parts of them are actually of intereset), these end up being mostly flat -- often tagged-value lists or records. For conversion itself -- layer-by-layer, accumulating the result as you go. But clearly, it all depends on the details here. > How do you update a value in JSON? How do you update 20 of them? > Painfully. Convert them to something sensible and only then process them? A few utility functions on top of tagged-value lists can work miracles for you, for example. > You can write functions to do it if you don't need to do it many times > in your program, but the more you need to do it the more you'll want a > special syntax to actually get things done instead of handling the > details of the modification. Just to be clear: I'm not expressing any opinions about any existing or fictitious EEP. Merely trying to approach your problem from a different angle, a more general one I think. Also my suggestion to use ETS in not necessarily related to following points about data representation. BR, -- Jachym From mjtruog@REDACTED Tue May 14 21:51:43 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 14 May 2013 12:51:43 -0700 Subject: [erlang-questions] [eeps] Maps In-Reply-To: <519130AC.8090303@erlang.org> References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> Message-ID: <519295CF.3000408@gmail.com> I haven't seen it mentioned yet, so I thought I would mention it... sorry if this repeats what others have suggested: It would help if we had a way to walk the tree representation of the Map. I know it might be an uncommon use-case, but it probably would provide support for deep-updates and complex path usage (JSONPath, JSONQuery, or JSONSelect type of stuff, http://stackoverflow.com/a/8481733). Walking the tree, could be emulated even if the data structure is not tree shaped due to a small size. The particular use case I am thinking of, is for doing matching on the key value. I have some trie source code here: https://github.com/okeuday/trie with a trie:find_match/2 function, which takes an exact string (list of integers) value and attempts to match it to stored key string patterns that possibly contain 1 or more "*" wildcard character (strings containing "**" do not exist, since the occurrence is considered an invalid entry). A "*" wildcard character matches one or more characters in the exact string value provided as a trie:find_match/2 function argument. The match could be ambiguous, except that it always finds the most exact match, going as deep into the tree as possible, and then iterating in alphabetical order (i.e., as exact as possible, based on the prefix). The opposite functionality (pattern matching exact string keys stored) is provided by trie:fold_match/4. This type of functionality should require an incomplete traversal of the tree structure (a set of functions that provide iterator behavior). Thanks, Michael On 05/13/2013 11:27 AM, Bj?rn-Egil Dahlberg wrote: > Hi! > > Thank you all who have contributed in this discussion! > > Instead of replying to every mail I will try to address and correct some misconceptions. > > - As stated earlier by several people. Maps are *not* Frames. > > - There are several semantic and syntax differences. Most notably arbitrary terms as keys. > > - The implementation of Maps will have Frame like characteristics for "simple" Maps, i.e. small number of associations (at most ~30 - 40 entries) and immediates as keys. Granted, the type information will not be there at compile-time, and Maps will not have literals as keys (as proposed in Frames), but should otherwise be comparable in performance with Frames for those cases. I also believe this would cover most of the use-cases as a record-replacement. > > The internal structure in the "simple" case would best be described with two tuples (but it would not have some other header and might have additionally meta-data in the header if necessary): > > { {K1, ...,Kn}, V1, ..., Vn} > > This structure will also allow key-sharing between Maps of the same "type", i.e. Maps with identical keys. > > When the Map no longer satisfies this "simple" arrangement it will transform into a tree and loose the previous characteristics and gain the characteristics of a tree. > > > - Maps are of type ordered set and follows term order. This has several implications. > - Term order does not distinguish between types float and integer. > - If data is stored in a tree where we search entries based on keys and we do so by term order. Since term order does not distinguish between float and integer, floats and integers which are equal occupy the same place. > - When iterating over keys in a Map it is done so in Term order, or reversed Term order, meaning it is expected that floats and integer are mixed, ex. 1 < 2.0 < 3 < 4.0 < 5 < 6.0. If we distinguish between floats and integers in Maps, let's say integer < doubles, the previous order would be, ex: 1 < 3 < 5 < 2.0 < 4.0 < 6.0. We would no longer iterate in Term order and this would not be consistent with the rest of Erlang, hence unacceptable. > > Maps emulates the behaviour of ETS ordered set in this sense. > > I think we can all agree that this is an unfortunate feature of Erlangs term order. I also agree that it would be best to have *matching* only, and not *equals* but I don't see how this could be achieved easily. At least not in an acceptable way. For example, we cannot achieve this by specifying our own Term order since that would violate iteration order, printing order, etc, or at least expected order. > > As noted previously, if we had Maps of type set we would not have this problem. However, since Maps needs to fit into Erlangs term order, Maps needs to be ordered (unless we could somehow abandon this strict rule). > > > - Deep updates of Maps are not covered by this EEP. We would welcome suggestions on this. > > Thank you! > > Regards, > Bj?rn-Egil > > On 2013-05-08 16:18, Bj?rn-Egil Dahlberg wrote: >> Hi everyone! >> >> We finally have a Maps EEP for you. This will get you an idea on what we are working on. Some of the ideas presented here was presented at Erlang Factory SF Bay Area 2013. >> >> I am sure that there will be some discussions about the contents of this EEP and I hope the discussions will be both fruitful and civilized. >> >> The journey of Maps and this EEP has been long and by no means a straight-forward or continuous one. I had a crystal clear picture of what I wanted Maps to be when we first started discussing it within OTP about two-three years ago. This EEP resembles that vision but it has had a lot of contributions of other ideas from both within and outside of OTP. >> >> The idea was a functional data-type, a syntax aware mapping of key-value associations with pattern matching. A syntax similar to records but without the hazzle of compile-time dependency and with arbitrary terms as keys. Order was not important and it could be implemented with a Hash-Array-Mapped-Trie with good performance and memory trade-offs. This was not an approach to replace records. It was meant to replace records where suitable and in other regards not be a replacement but its own thing. >> >> From the community there has been many wishes of a Map like data-type and a few suggestions. The one suggestion that stands out is of course the Frames proposal from Richard O'Keefe. It is the most complete proposal I've seen and is very well thought out. Its goal is to be a record replacement and the proposal satisfies this goal very well. >> >> - If Frames are that good, why a separate EEP? >> - It boils down to goals and constraints. >> >> A record replacement is just that, a replacement. >> It's like asking the question, "What do we have?" instead of "What can we get?" >> The instant rebuttal would be "What do we need?" I say Maps. >> >> Frames has certainly influenced Maps. In many regards Maps also encompasses Frames but Maps tries to do more. I think the most significant difference would be, arbitrary terms as keys and how many different keys we would have in a Map. In the end I believe they are two different things and have different goals. >> >> Some Notes and Disclaimers: >> >> Later iterations of Maps has gone through some changes, most significantly, >> >> * From a set of key-values to a ordered set of key-value associations >> >> I was originally against this change since it forces restrictions on the implementation and it illuminates the lack of distinction between arithmetic order and term order, i.e. the problem of mixing integer and float types as keys in a tree. However, I was later persuaded that key ordering is necessary. We have to respect the totalitarian order of terms. >> >> Considerations has been made on how to, if at all possible, apply Frames characteristics to Maps. Most significantly memory space and key-sharing characteristics. This is not detailed in the EEP though, just mentioned. >> >> The function interface has had many revisions as well. At some stage the API was considered to be a drop-in-replacement for `dict` and thus would have the same function-names. This goal/constraint was dropped by Technical Board decision recently. >> >> From the very beginning Maps was envisioned to have the ability to bind variables derived from the environment. Like this: >> >> function(K1, #{ K1 := K2, K2 := V }) -> V. >> >> This feature is a beast. Powerful and scary. It is not confined to only Maps but should also be applicable to other types as well: >> >> function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. >> >> It is uncertain how effective such an implementation would be and in the end we might not want this feature at all. >> >> In this EEP we will describe syntax and semantics of Maps but very little is disclosed of its actual implementation. Current prototypes stems from using sparse tuples in a HAMT-like data structure and tuple-like data structures. The HAMT-like data structure is discontinued and will be replaced by some ordered tree. >> >> The proposal is included as an attachment but can also be viewed at this git-repository: >> https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md >> >> >> Regards, >> Bj?rn-Egil Dahlberg >> Erlang/OTP >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > eeps mailing list > eeps@REDACTED > http://erlang.org/mailman/listinfo/eeps -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 14 22:21:29 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 May 2013 22:21:29 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> <51922223.3040308@ninenines.eu> Message-ID: <51929CC9.60003@ninenines.eu> On 05/14/2013 08:21 PM, Bj?rn-Egil Dahlberg wrote: > 2013/5/14 Lo?c Hoguin > > I personally would prefer a more powerful: > M#{ K #{ DK => DV }} > > Because it allows you to write things like: > M#{ K #{ DK => DV }, K2 => V2 } > > And update multiple levels at once. > > I know K can be a map, but deep update only works on values, the > same way normal update only works on values, so no confusion is > possible, we're only updating values, not K itself. If you feel an > operator is needed it can be introduced between K and # in the two > examples above. > > Programmatically this would translate as extracting the map found at > key K, and ensuring it's a map, updating this map with DK => DV, > then placing this map back into the key K in map M along with > setting V2 in key K2. > > It doesn't sound hard to implement in the compiler, it's just > unrolling things for access and rolling back again for the actual > update. Exactly what we do manually today. > > It should also be easy to compile to an optimized deep update with > this syntax as you got everything in a single expression. > > Note: If you are not interested in it despite how simple it is > please at least ensure it can be done with a parse transform. > > Thoughts? > > > I think we want to be explicit here. Meaning using an operator and > "seeing" the value in action. (I have bad memories of perl). > > Just some first thoughts on deep updates here to see if I get the > intention right. What I think we need is someway to use an associated > value within the map. I figure, something like "escaping" the key which > would fetch the associated value. Let's use '~' in this example. > > Say I have a nested map like above, M = #{ a => #{ b => 1 }} and I would > like to update the inner b value with it self plus four. I could write > it like this: > > 3> M#{ a := ~a#{ b := ~b + 4 }}. > #{ a => #{ b => 5 }} > > Another possibility would perhaps be to bind values first (wrapped in { > .. } )and then use them in updates, say > > M#{ a := M }{ a := M#{ b := V }{ b := V + 4 } } > > .. that looks even more horrible .. =) > > I haven't given this much thought but I'm trying to get a sense of it. > Something like this? That's taking it even further than I thought. I would be perfectly fine with binding separately from updating, especially because: M#{ a := ~a#{ b := ~b + 4 }}. isn't so pretty when done with some key types like: M#{ <<"a">> := ~<<"a">>#{ <<"b">> := ~<<"b">> + 4 }}. Now most of what I would be using would be atoms and integers, but this still would be the ugliest allowed syntax ever. It works fine if done in two steps (with the first step most likely done in the function clause itself). doit(M=#{ a := A=#{ b := B }}) -> M#{ a := A#{ b := B + 4 }}. Writing this I realize it's quite possible that you can already do it with the current EEP. Am I right? And it also fits: doit(M=#{ a := A=#{ b := B }, aa := yes_do_it}) -> M#{ a := A#{ b := B + 4 }, aa := done}. You can already do this with records. Can we do this with maps too? This + any type as key + dynamic is all I need. It would be cool if that could also be optimized at compile time, though that's more of an edge case so understandable if that comes later. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Tue May 14 22:38:43 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 May 2013 22:38:43 +0200 Subject: [erlang-questions] Maps In-Reply-To: <20130514190509.GA13738@circlewave.net> References: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <5192688E.5090107@ninenines.eu> <20130514190509.GA13738@circlewave.net> Message-ID: <5192A0D3.9010304@ninenines.eu> On 05/14/2013 09:05 PM, Jachym Holecek wrote: > # Lo?c Hoguin 2013-05-14: >> On 05/14/2013 03:07 PM, Robert Virding wrote: >>> Lo?c you and I can meet at dawn like gentlemen to discuss special syntax >>> or not. Swords or pistols? :-) >> >> Pistols. >> >> I don't disagree with most of what you say. The special syntax isn't >> really needed until you get a project where you need to keep really many >> things in the state (hundreds if not thousands of terms per connection) >> and update them many times, in my case many times per second, while >> still striving for low latency. > > Have you considered ETS tables? A private ordered_set variety would seem > to fit your needs rather well, based on what you just said and an example > you gave in an earlier post. You mention keeping large state, be mindful > this may, inherently, be playing against your low latency goal if you keep > all the data on heap. Well, depending on what precisely you mean by "large > state" and "low latency". > > Just briefly -- you mentioned you need to maintain nested objects along > with attributes: > > true = ets:insert_new(Tid, {{Player_id, possesions, Item_id, count}, 1}) > > There's also ets:{update_counter/X|lookup_element/X}, and ets:select/X with > known prefix behaves just nice too. Maybe a list would work better as key > type than tuple. Anyway, you get the idea. Thanks but I already know ets. ets is too verbose, and I can't pattern match against values easily in function clauses. There's been a thread a while back with various "solutions" being given, none coming close to what maps can allow, most particularly none allowing matching in function clauses, which is the other requirement for the code to be tidy. > Also -- you could consider using processes to encapsulate various bits of > state. I'd just be passing the data back to the connection process all the time, it's pointless. I don't control the protocol, and the protocol requires many superfluous values to be sent to the client all the time. The only thing I can control is how I structure things to make this easier for me, and avoid writing too much code to do it. Maps allow me to do that. >> How do you access deep values in JSON in Erlang? > > I don't. Usually it ends up being pretty reasonable to convert them to > Erlang-friendly internal representations on protocol boundaries (good > thing one mostly knows what parts of them are actually of intereset), > these end up being mostly flat -- often tagged-value lists or records. > > For conversion itself -- layer-by-layer, accumulating the result as > you go. But clearly, it all depends on the details here. If you only had objects you could do it in a single line with maps. If you also have arrays, it'll be one list comprehension per array on top of that, with possible operations inside it. You could process your whole JSON with a one liner. No accumulators, not lists:reverse, no need to write any function, and most likely a lot more optimized with less chances of bugs than you'd have done otherwise. >> How do you update a value in JSON? How do you update 20 of them? >> Painfully. > > Convert them to something sensible and only then process them? A few > utility functions on top of tagged-value lists can work miracles for > you, for example. I know what I can do today, you don't need to tell me. I also know what I could do tomorrow. The future looks pretty damn good. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From wallentin.dahlberg@REDACTED Tue May 14 23:04:10 2013 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 14 May 2013 23:04:10 +0200 Subject: [erlang-questions] Maps In-Reply-To: <51929CC9.60003@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> <51922223.3040308@ninenines.eu> <51929CC9.60003@ninenines.eu> Message-ID: 2013/5/14 Lo?c Hoguin > On 05/14/2013 08:21 PM, Bj?rn-Egil Dahlberg wrote: > >> 2013/5/14 Lo?c Hoguin > >> >> I personally would prefer a more powerful: >> M#{ K #{ DK => DV }} >> >> Because it allows you to write things like: >> M#{ K #{ DK => DV }, K2 => V2 } >> >> And update multiple levels at once. >> >> I know K can be a map, but deep update only works on values, the >> same way normal update only works on values, so no confusion is >> possible, we're only updating values, not K itself. If you feel an >> operator is needed it can be introduced between K and # in the two >> examples above. >> >> Programmatically this would translate as extracting the map found at >> key K, and ensuring it's a map, updating this map with DK => DV, >> then placing this map back into the key K in map M along with >> setting V2 in key K2. >> >> It doesn't sound hard to implement in the compiler, it's just >> unrolling things for access and rolling back again for the actual >> update. Exactly what we do manually today. >> >> It should also be easy to compile to an optimized deep update with >> this syntax as you got everything in a single expression. >> >> Note: If you are not interested in it despite how simple it is >> please at least ensure it can be done with a parse transform. >> >> Thoughts? >> >> >> I think we want to be explicit here. Meaning using an operator and >> "seeing" the value in action. (I have bad memories of perl). >> >> Just some first thoughts on deep updates here to see if I get the >> intention right. What I think we need is someway to use an associated >> value within the map. I figure, something like "escaping" the key which >> would fetch the associated value. Let's use '~' in this example. >> >> Say I have a nested map like above, M = #{ a => #{ b => 1 }} and I would >> like to update the inner b value with it self plus four. I could write >> it like this: >> >> 3> M#{ a := ~a#{ b := ~b + 4 }}. >> #{ a => #{ b => 5 }} >> >> Another possibility would perhaps be to bind values first (wrapped in { >> .. } )and then use them in updates, say >> >> M#{ a := M }{ a := M#{ b := V }{ b := V + 4 } } >> >> .. that looks even more horrible .. =) >> >> I haven't given this much thought but I'm trying to get a sense of it. >> Something like this? >> > > That's taking it even further than I thought. I would be perfectly fine > with binding separately from updating, especially because: > > > M#{ a := ~a#{ b := ~b + 4 }}. > > isn't so pretty when done with some key types like: > > M#{ <<"a">> := ~<<"a">>#{ <<"b">> := ~<<"b">> + 4 }}. > Now most of what I would be using would be atoms and integers, but this > still would be the ugliest allowed syntax ever. > I totally agree with you, it certainly lacks beauty. > > It works fine if done in two steps (with the first step most likely done > in the function clause itself). > > doit(M=#{ a := A=#{ b := B }}) -> > M#{ a := A#{ b := B + 4 }}. > > Writing this I realize it's quite possible that you can already do it with > the current EEP. Am I right? > Yes. > > And it also fits: > > doit(M=#{ a := A=#{ b := B }, aa := yes_do_it}) -> > M#{ a := A#{ b := B + 4 }, aa := done}. > > You can already do this with records. Can we do this with maps too? This + > any type as key + dynamic is all I need. > Yes. I had similar examples in the EEP but didn't nest the Maps. > > It would be cool if that could also be optimized at compile time, though > that's more of an edge case so understandable if that comes later. What was it you said to me? "Make it work, make it pretty, make it fast." =) Still on step oneish. > > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Tue May 14 23:21:18 2013 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Tue, 14 May 2013 23:21:18 +0200 Subject: [erlang-questions] Cowboy/giallo {error,badlength} Message-ID: Hi all I have a very small and very stupid question about Cowboy (or, perhaps, giallo? ) If playing around a bit, trying to see what and how to fit in my head, and I constantly run into a problem giallo:post_param(<<"param_name">>, Req) returns {error, badlength} for rather short texts (I really need to post much larger texts). How can I raise the limit on the body length? Thanks! From essen@REDACTED Tue May 14 23:48:38 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 14 May 2013 23:48:38 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> <51922223.3040308@ninenines.eu> <51929CC9.60003@ninenines.eu> Message-ID: <5192B136.5000001@ninenines.eu> On 05/14/2013 11:04 PM, Bj?rn-Egil Dahlberg wrote: > It works fine if done in two steps (with the first step most likely > done in the function clause itself). > > doit(M=#{ a := A=#{ b := B }}) -> > M#{ a := A#{ b := B + 4 }}. > > Writing this I realize it's quite possible that you can already do > it with the current EEP. Am I right? > > > Yes. > > > And it also fits: > > doit(M=#{ a := A=#{ b := B }, aa := yes_do_it}) -> > M#{ a := A#{ b := B + 4 }, aa := done}. > > You can already do this with records. Can we do this with maps too? > This + any type as key + dynamic is all I need. > > > Yes. I had similar examples in the EEP but didn't nest the Maps. Could be worth adding to the EEP so as to not forget to test it. Should I send a PR to your repository? > It would be cool if that could also be optimized at compile time, > though that's more of an edge case so understandable if that comes > later. > > > What was it you said to me? "Make it work, make it pretty, make it > fast." =) Still on step oneish. Yep. It was just a suggestion that heavily hinted at "optimize for that case after everything else is optimized". -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From wallentin.dahlberg@REDACTED Wed May 15 00:05:58 2013 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Wed, 15 May 2013 00:05:58 +0200 Subject: [erlang-questions] Maps In-Reply-To: <5192B136.5000001@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <519130AC.8090303@erlang.org> <51922223.3040308@ninenines.eu> <51929CC9.60003@ninenines.eu> <5192B136.5000001@ninenines.eu> Message-ID: 2013/5/14 Lo?c Hoguin > On 05/14/2013 11:04 PM, Bj?rn-Egil Dahlberg wrote: > >> It works fine if done in two steps (with the first step most likely >> done in the function clause itself). >> >> doit(M=#{ a := A=#{ b := B }}) -> >> M#{ a := A#{ b := B + 4 }}. >> >> Writing this I realize it's quite possible that you can already do >> it with the current EEP. Am I right? >> >> >> Yes. >> >> >> And it also fits: >> >> doit(M=#{ a := A=#{ b := B }, aa := yes_do_it}) -> >> M#{ a := A#{ b := B + 4 }, aa := done}. >> >> You can already do this with records. Can we do this with maps too? >> This + any type as key + dynamic is all I need. >> >> >> Yes. I had similar examples in the EEP but didn't nest the Maps. >> > > Could be worth adding to the EEP so as to not forget to test it. Should I > send a PR to your repository? If you do, make sure to base it of erlang/eep master since Raimo did not use my eep branch for some reason. But hopefully we will not forget to test this anyhow =) > > > It would be cool if that could also be optimized at compile time, >> though that's more of an edge case so understandable if that comes >> later. >> >> >> What was it you said to me? "Make it work, make it pretty, make it >> fast." =) Still on step oneish. >> > > Yep. It was just a suggestion that heavily hinted at "optimize for that > case after everything else is optimized". =) > > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From flw@REDACTED Wed May 15 00:04:48 2013 From: flw@REDACTED (Florian Waas) Date: Tue, 14 May 2013 15:04:48 -0700 Subject: [erlang-questions] Rebar/Cover and leex/yecc Message-ID: Using rebar's reporting of coverage during eunit runs, I get coverage numbers for the generated lexer/parser code which contains a ton of boiler plate code from leexinc.hrl and yeccpre.hrl which is not exercised. As a result, the coverage number is rather meaningless. Anybody have ideas for workarounds? Elegant ways to exclude these files from the report? Thanks, -fl. From ok@REDACTED Wed May 15 00:55:39 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 15 May 2013 10:55:39 +1200 Subject: [erlang-questions] Maps In-Reply-To: <51921CD4.4000706@ninenines.eu> References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> <518CDE83.4090401@ninenines.eu> <518D0592.9000601@ninenines.eu> <29EB3051-9BD6-4310-A4DF-A1E2451F167B@cs.otago.ac.nz> <5190D1F1.7020608@ninenines.eu> <51921CD4.4000706@ninenines.eu> Message-ID: <4BA84299-0137-4491-B2F9-25FD6BC69E64@cs.otago.ac.nz> On 14/05/2013, at 11:15 PM, Lo?c Hoguin wrote: >> Single mechanism. ONE thing to understand. >> Proven technology: people using other languages with this approach >> don't seem to have any problem. > > You don't say what language though, so I'm assuming it's obscure languages and the technology has only been proven with a small population, probably mostly academic. Stop making assumptions. I want the EEP to have my name on it, but the approach is used in a language which has enough users that new books about it or based on it keep rolling off the presses (much faster than Erlang books appear) and for which there are millions of lines of maintained code in the public repositories (dwarfing Erlang). > > If there is a fool proof solution I'm all for it, but I can't really envision a readable universal syntax for something like this. (And this is the part where you say that nobody needs syntax, they only need functions, I assume.) Stop making assumptions. Except perhaps for the spelling of a particular operator, the syntax is familiar to every programmer, including you. > There are details that have to be spelled out clearly, and I'm very busy at the moment, which is why the EEP isn't ready yet. (Actually, the basic idea has been used in four different languages that I've used. Great artists steal. (:-)) From ok@REDACTED Wed May 15 03:15:34 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 15 May 2013 13:15:34 +1200 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: <24037BF3-B33A-4F79-913F-09142A6767E2@cs.otago.ac.nz> On 15/05/2013, at 3:17 AM, Dan Gudmundsson wrote: > Richard A. O'Keefe > > Frames > > > Maps are basically an extension of Frames which allows for arbitrary key terms. And from my perspective that is the major thing WRONG with them. I have never understood why it is so hard to persuade people that unbridled power is a *bad* thing. "Unbridled Power" is the title of a book by a former New Zealand Prime Minister arguing that there were practically no limits on what the New Zealand Government could allow itself to do and that this was a bad thing. When MMP came in he wrote a successor called "Bridled Power" explaining why the limitations this would place on a government would be a good thing. However, actual experience has shown that the party that gets most of the cabinet seats still thinks that getting the votes of 35% of the electors is a licence to do what you please. We definitely need better reins for Government. See the analogy? Frames are valuable because of the things they CAN'T do. Maps will take any old rubbish as a key EVEN IF YOU DIDN'T MEAN THAT. For example, suppose you are dealing with JSON, and you add X=Y to a map, intending this to mirror adding X: Y to the JSON. But in fact you needed to add Y=X. The map doesn't know or care. If you want to model JSON in Erlang, you want a data structure that won't _let_ you use "arbitrary key terms". > Which could use your implementation proposal for small maps defined in source code, > and switches to a tree (or something else) for larger dynamic sized Maps. And frames could also switch to a tree for larger frames. > > >- Frames are limited to atom keys in the interests of > > compile-time error detection (both simply noticing that > > a key is not an atom at all and allowing the possibility > > of Dialyzer support). Frames are limited to atom keys IN THE INTERESTS OF DOING WHAT THE PROGRAMMER SAYS S/HE WANTS. Compile-time is always an attractive option, but run-time will do. > > So use := and you will get a runtime/compile check that the key exists. I used to have a Pogo cartoon on my office door where the punchline was something like "Your mistake is not the right mistake." (:-). The existence of two different notions of equality in Erlang is a problem, and some day it will have to be addressed. Fortunately Prolog was able to distinguish between X < Y (X is numerically less than Y) and X @< Y (X is less than Y as a term) and we were able to say that nan(X) => X = X, X == X, \+ (X =:= Y). (a NaN unifies with itself, is identical to itself, but is not numerically equal to itself.) One of the reasons for sticking with atoms in frames is that the two notions of equality and indeed the two notions of ordering coincide. You could extend that to allow integers as keys, which arguably ML does. You could extend it to allow binaries as keys, which could be a good thing. But you can't extend it to floats without running into trouble. > Robert Virding > > Use only matching > > We would like to but we believe we can not do that because we can't output a defined order, > i.e. you can not sort 1.0 and 1. > We would need to define a new term order in erlang and we need to introduce something > like <:<, >:> =:< and >:=. No, the issue here is that ORDERED dictionaries and UNORDERED dictionaries are different types that have similar but not identical interfaces, constraints, and performance. In particular, "is identical to" is the right comparison for unordered dictionaries and "is numerically equal to" is the right comparison for ordered dictionaries. Just as I think trying to smush records and dictionaries into a single data structure is rather like trying to make a combined motorbike/pogo stick, so I think that trying to smuch ordered dictionaries and unordered dictionaries into a single data structure is rather like trying to make a combined motorbike/tracked vehicle. See http://en.wikipedia.org/wiki/Kettenkrad. If you want a motorbike, you probably _don't_ want a Kettenkrad, even though it looks like a motorbike at the front. In my Smalltalk library, SortedDictionary offers _ first _ first: n _ firstKey _ firstKeyGeq: key [ifNone: aBlock] _ firstKeyGtr: key [ifNone: aBlock] _ removeFirst _ removeFirst: n _ last _ last: n _ lastKey _ lastKeyLeq: key [ifNone: aBlock] _ lastKeyLss: key [ifNone: aBlock] _ removeLast _ removeLast: n _ removeKeysGeq: key [return: aBoolean] _ removeKeysGtr: key [return: aBoolean] _ removeKeysLeq: key [return: aBoolean] _ removeKeysLss: key [return: aBoolean] _ reverse[Keys[AndValues]]Do: aBlock (not all of these are implemented at the moment), as well as the operations suitable for unordered dictionaries. These methods make no sense at all for unordered dictionaries. Providing them offers greater power. There are more things you can do. But the last thing any sane Smalltalk programmer would want is to have them as the *default* kind of dictionary, because this power comes at the cost of increased storage and increased time. I use them unhesitatingly when I *need* the sorted order. But I avoid them at all other times. The existence of two data structures with *different* contracts (sorted dictionaries and unsorted dictionaries) makes it hard to privilege one of them as THE map type to get a syntax. In developing my Smalltalk, I assure you that I have felt the cultural pressure from Python and Perl to have a syntax for dictionaries most keenly. But I have 12 different kinds of dictionary (one of them unfinished) and can foresee a need for two more. No, wait, there's a "ShellEnvironment" class that is also a dictionary. So somewhere between 12 and 15. Let's admit that two of them are for compatibility only, so claim just 10 right now. They differ in - what kinds of keys are allowed - what kind of comparison is done - is sorted traversal easy - do they have a "parent" that is consulted for read access to keys not directly present - what the time and space costs are Erlang doesn't have a huge number of dictionary types, but it should certainly have at least - keys any term, built in hashing and equality - keys any term, programmer-defined hashing and equality - keys any term, built-in term order - keys any term, programmer-defined term order - keys restricted somehow, perhaps a user-defined filter on addition - perhaps some optimisation for binary keys or integer keys or atom keys. Much of the variations could be hidden behind a common interface, but the distinction between ORDERED and UNORDERED cannot be completely hidden. From ok@REDACTED Wed May 15 03:32:57 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 15 May 2013 13:32:57 +1200 Subject: [erlang-questions] Maps In-Reply-To: <5192688E.5090107@ninenines.eu> References: <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <5192688E.5090107@ninenines.eu> Message-ID: <8C7174A8-8514-4C06-9EDD-024FF9EBB751@cs.otago.ac.nz> > > My personal use case is probably not that universal, but I believe it also applies to anyone who has to access or modify JSON or equivalent (not XML). How do you access deep values in JSON in Erlang? How do you update a value in JSON? Not present-day Erlang, but we could do it. (1) Copy Pop-2. Define that E0.[m:]f[([E1,...,En])] is equivalent to [m:]f(E0[,E1,...,En]). This means that X.hd and hd(X) are the same expression, and X.lists:sublist(Y, Z) and lists:sublist(X, Y, Z) are the same expression. This is a trivial source->source transformation that gives functional semantics OO-like syntax. Restriction: the dot must be immediately adjacent to the m or the f. Similar restrictions are found in other languages. For example, in Haskell, m.f is "f in module m" but m . f is "composition of functions m and f". People live with it. (2) With a slight twist. Define that E0.[E1] is equivalent to slot(E0, E1), whatever slot is in scope. So if you do -import(json, [slot/2]). then J.[1].[foo].[3].[oar] would be the functional equivalent of J[1]["foo"][3]["oar"] in Javascript. Again, we have function *semantics* hiding under OO-ish syntax. (1) and (2) can be mixed freely. This is also a pretty trivial source->source transformation. (3) Define update according to the EEP that I haven't finished yet, so that J.[1].[foo].[3].[oar] := Whatever creates a new variable J' which is used downstream in the place of J and does pretty much the right thing. Getting all the details of this right is non-trivial, but definitely doable. All of this is purely at the level of syntax. None of it is specific to records, frames, maps, or Dho-Nha curves. We *DON'T* have this yet, but we *COULD*, and it would require NO changes to the VM or the compiler back-end, so BEAM code generated with a new compiler would be backwards compatible. From daniel.widgren@REDACTED Wed May 15 09:12:53 2013 From: daniel.widgren@REDACTED (Daniel Widgren) Date: Wed, 15 May 2013 09:12:53 +0200 Subject: [erlang-questions] Cowboy/giallo {error,badlength} In-Reply-To: References: Message-ID: Do you have any stacktrace on the error? Would be great. -- Daniel 2013/5/14 Dmitrii Dimandt > Hi all > > I have a very small and very stupid question about Cowboy (or, perhaps, > giallo? ) > > If playing around a bit, trying to see what and how to fit in my head, and > I constantly run into a problem > > giallo:post_param(<<"param_name">>, Req) returns {error, badlength} for > rather short texts (I really need to post much larger texts). > > How can I raise the limit on the body length? > > Thanks! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed May 15 10:20:20 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 15 May 2013 10:20:20 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> Message-ID: <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> On May 14, 2013, at 7:37 PM, Chris King wrote: > Static structure does not preclude subtyping. OCaml's functional > objects capture this notion perfectly -- an object has a set of > methods which are known at compile time (hence static). However any > given function need only be aware of the presence of the subset of > methods in which it is interested. Note that this does not preclude a > function from creating a copy of said object with only those methods > modified! This is often called "structural subtyping" in contrast to "nominal subtyping". The difference is that the structural variant is implicit whereas the nominal variant is explicit. You have to explicitly define the relationsship in the nominal case, which is common in Java, C++, ? Google Go is another language with structural subtyping in its interface model. There is also some structurality in the Standard ML module system w.r.t. signature checks. I agree we need something which is better than records at some point, with static guarantee of the contents. But currently, the thing we are missing is a fast map implementation which is functional, so we can avoid ETS tables for purposes they aren't designed for. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From bip@REDACTED Wed May 15 10:01:48 2013 From: bip@REDACTED (Bip Thelin) Date: Wed, 15 May 2013 10:01:48 +0200 Subject: [erlang-questions] Cowboy/giallo {error,badlength} In-Reply-To: References: Message-ID: Comments interspersed. > I have a very small and very stupid question about Cowboy (or, perhaps, giallo? ) That would be giallo. > giallo:post_param(<<"param_name">>, Req) returns {error, badlength} for rather short texts (I really need to post much larger texts). > How can I raise the limit on the body length? You've hit the default MaxBodyLength of 16kb: https://github.com/extend/cowboy/blob/master/guide/req.md#request-body I've just pushed a new version with a post_param/4 which lets you optionally specify your own MaxBodyLength or the atom infinity to ignore the body_length check. Remember that the body_length check was added to cowboy to mitigate a possible DoS scenario: https://gist.github.com/josevalim/eea34c1c76056d2fdf4f -- Bip Thelin -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed May 15 12:28:25 2013 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 15 May 2013 11:28:25 +0100 (BST) Subject: [erlang-questions] Maps In-Reply-To: Message-ID: <1951928406.113795457.1368613705911.JavaMail.root@erlang-solutions.com> ----- Original Message ----- > From: "Dan Gudmundsson" > Robert Virding > > Use only matching > We would like to but we believe we can not do that because we can't > output a defined order, > i.e. you can not sort 1.0 and 1. > We would need to define a new term order in erlang and we need to > introduce something > like <:<, >:> =:< and >:=. I personally think this would be a good idea to introduce anyway as I think today's comparison ops are not good. So: - introduce a new set of comparison operators that do not convert integers to floats on comparison - fix the term ordering for this. This is actually very easy, just make integers < floats, we are comparing *terms* - modify the libraries which use comparisons to use the new ops - use them in maps if we are going to have sorted maps I know we can't make the existing comparison ops only work numbers, but it would be nice. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed May 15 14:01:24 2013 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 15 May 2013 13:01:24 +0100 (BST) Subject: [erlang-questions] Rebar/Cover and leex/yecc In-Reply-To: Message-ID: <877786169.113884683.1368619284188.JavaMail.root@erlang-solutions.com> You also get a lot of dialyzer warnings as well from the leex boiler plate code. It is correct but irritating to see. Robert ----- Original Message ----- > From: "Florian Waas" > To: "erlang-questions" > Sent: Wednesday, 15 May, 2013 12:04:48 AM > Subject: [erlang-questions] Rebar/Cover and leex/yecc > > Using rebar's reporting of coverage during eunit runs, I get coverage > numbers for the generated lexer/parser code which contains a ton of > boiler plate code from leexinc.hrl and yeccpre.hrl which is not > exercised. As a result, the coverage number is rather meaningless. > > Anybody have ideas for workarounds? Elegant ways to exclude these > files from the report? > > > Thanks, > -fl. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From colanderman@REDACTED Wed May 15 16:27:43 2013 From: colanderman@REDACTED (Chris King) Date: Wed, 15 May 2013 10:27:43 -0400 Subject: [erlang-questions] Maps In-Reply-To: <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> Message-ID: On Wed, 15 May 2013 04:20:20 -0400, Jesper Louis Andersen wrote: > > On May 14, 2013, at 7:37 PM, Chris King wrote: > >> Static structure does not preclude subtyping. OCaml's functional >> objects capture this notion perfectly -- an object has a set of >> methods which are known at compile time (hence static). However any >> given function need only be aware of the presence of the subset of >> methods in which it is interested. Note that this does not preclude a >> function from creating a copy of said object with only those methods >> modified! > > This is often called "structural subtyping" in contrast to "nominal > subtyping". The difference is that the structural variant is implicit > whereas the nominal variant is explicit. You have to explicitly define > the relationsship in the nominal case, which is common in Java, C++, ? > > Google Go is another language with structural subtyping in its interface > model. There is also some structurality in the Standard ML module system > w.r.t. signature checks. Yes, I know. > I agree we need something which is better than records at some point, > with static guarantee of the contents. But currently, the thing we are > missing is a fast map implementation which is functional, so we can > avoid ETS tables for purposes they aren't designed for. I understand that. But why must (a) this new fast map implementation have an associated syntax, and (b) this syntax displace the syntax which *is* needed to implement pattern-matching of frames? If anything you can get an *even faster* map implementation if you ignore the use case of extensible records. From dmitrii@REDACTED Wed May 15 16:56:46 2013 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 15 May 2013 16:56:46 +0200 Subject: [erlang-questions] Cowboy/giallo {error,badlength} In-Reply-To: References: Message-ID: > Comments interspersed. > > I have a very small and very stupid question about Cowboy (or, perhaps, giallo? ) > That would be giallo. > > > giallo:post_param(<<"param_name">>, Req) returns {error, badlength} for rather short texts (I really need to post much larger texts). > > > How can I raise the limit on the body length? > You've hit the default MaxBodyLength of 16kb: https://github.com/extend/cowboy/blob/master/guide/req.md#request-body > > I've just pushed a new version with a post_param/4 which lets you optionally specify your own MaxBodyLength or the atom infinity to ignore the body_length check. Awesome! > Remember that the body_length check was added to cowboy to mitigate a possible DoS scenario: https://gist.github.com/josevalim/eea34c1c76056d2fdf4f > Yes, I saw that one as well. I wonder if this can be governed by a configuration parameter somewhere > -- > Bip Thelin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ola.Backstrom@REDACTED Wed May 15 17:15:10 2013 From: Ola.Backstrom@REDACTED (=?utf-8?B?T2xhICBCw6Rja3N0csO2bQ==?=) Date: Wed, 15 May 2013 15:15:10 +0000 Subject: [erlang-questions] Maps In-Reply-To: References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> Message-ID: <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> > On Wed, 15 May 2013 04:20:20 -0400, Jesper Louis Andersen wrote: [snip] > I understand that. But why must (a) this new fast map implementation have > an associated syntax, and (b) this syntax displace the syntax which *is* > needed to implement pattern-matching of frames? Could the syntax somehow be decoupled from the implementation, allowing for usage in both scenarios? I just played with the idea a little... so assume the map implementation is found in the module 'coolmap' A) To create a map, one would have to point out what implementation module that should do the job. M = coolmap#{foo => 1, bar => 2, baz =>3}. Since coolmap is an atom, the compiler knows this must be creation of a new map... thus translated to M= coolmap:create([{foo, 1},{bar, 2}, {baz, 3}]). The variable M should somewhere contain the module name, so that a call extract_module(M) returns coolmap. B) To update a map, M2 = M#{foo := 12}. Since M isn't an atom the compiler understands that this translates to M2= (extract_module(M)):update([{foo, 12}], M). C) To introduce new keys M3 = M2#{foo:=100, bary => 24} The Since => is used in the expression this will be translated to another function: M2= (extract_module(M2)):edit([{update, foo, 100},{set, foo, 12}], M2). The above would allow anybody to create new implementations, could it be useful to allow Frames and Maps to share syntax? From ingela.andin@REDACTED Wed May 15 18:58:34 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 15 May 2013 18:58:34 +0200 Subject: [erlang-questions] SSL renegotiation In-Reply-To: <51920093.5020704@tail-f.com> References: <51920093.5020704@tail-f.com> Message-ID: Hi Niclas! 2013/5/14 Niclas Eklund > Hi! > > Out of curiosity, why is this timelimit hardcoded and why was 12 choosen? > > To be long enough to mitigate the dos attack but short enough to not cause any disturbance. If you think it would be really useful to you to be able to configure this timeout you are welcome to make a patch suggestion. It is easier to add things than to remove them so it is a question of priorities. > -define(WAIT_TO_ALLOW_**RENEGOTIATION, 12000). > > 42 would have made more sense since it's the meaning of life ;-) > > I do agree that it would have been much more fun to to module the world in terms of The *Hitchhiker's Guide to the Galaxy* ;) Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From flw@REDACTED Wed May 15 22:37:22 2013 From: flw@REDACTED (Florian Waas) Date: Wed, 15 May 2013 13:37:22 -0700 Subject: [erlang-questions] Rebar/Cover and leex/yecc In-Reply-To: <877786169.113884683.1368619284188.JavaMail.root@erlang-solutions.com> References: <877786169.113884683.1368619284188.JavaMail.root@erlang-solutions.com> Message-ID: Quite right, Robert. Since I run dialyzer explicitly controlled by the Makefile I can simply exclude these files. Rationale being that the little bit of not-generated code in there doesn't need a dialyzer scrub as much as "regular" code. Not perfect, I know, but workable. With Cover and its reporting things are a little more complicated though. Rebar separates application from test sources and then reports on each bucket. Since my check-in criteria are based among other things on code coverage being above 90% having 2 files with abysmally low coverage (~65%) makes this criterion a bit hard to enforce automatically. Hence, I was wondering if anybody's gone through similar pains and found a nice way to exclude these or, better still, mapped the coverage numbers back to the original xrl/yrl files? Cheers, -fl. On Wed, May 15, 2013 at 5:01 AM, Robert Virding wrote: > You also get a lot of dialyzer warnings as well from the leex boiler plate code. It is correct but irritating to see. > > Robert > > ----- Original Message ----- >> From: "Florian Waas" >> To: "erlang-questions" >> Sent: Wednesday, 15 May, 2013 12:04:48 AM >> Subject: [erlang-questions] Rebar/Cover and leex/yecc >> >> Using rebar's reporting of coverage during eunit runs, I get coverage >> numbers for the generated lexer/parser code which contains a ton of >> boiler plate code from leexinc.hrl and yeccpre.hrl which is not >> exercised. As a result, the coverage number is rather meaningless. >> >> Anybody have ideas for workarounds? Elegant ways to exclude these >> files from the report? >> >> >> Thanks, >> -fl. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From ok@REDACTED Thu May 16 05:04:23 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 16 May 2013 15:04:23 +1200 Subject: [erlang-questions] Better error messages would be helpful Message-ID: I'm in the middle of writing to someone who asked for help with some Erlang code. With identifiers replaced, he had 21: f(A, B, C) -> 22: D = lists:map(fun (E) -> 23: case proplist:get_value(E, B, error) of 24: {F} -> {E,F}; 25: error -> {error} 26: end, A). erlc reported ./foobar.erl:26: syntax error before: ')' ./foobar.erl:16: function f/3 undefined When there are error messages that prevent any function from being processed, it would be helpful if erlc did *NOT* complain about undefined, because that message is untrue. The function *was* defined, the compiler just gave up on it. But the message that really had him confused was the "syntax error before ')'" one. This is the kind of error message one typically gets from something using Yacc-style parsing. I didn't like his layout, so I fed his file to an Erlang pretty-printer I wrote a couple of years. The code is fast but rather horrible C, so I do not care to share it. It was thrown together as an experiment, with an eye to using it in a text editor. What did that pile of hacks say? Line 26: 'fun' at line 22 closed by ')' (expected 'end'). So a better error message is certainly possible. This is a pretty-printer, not a parser. Almost the only syntax checking it does is to keep track of brackets. But bracket errors are common enough that it's worth while. I don't like the idea of slowing erlc down, but I wonder if decent bracket matching checks could be folded in with macro expansion without too much trouble? From skribent_har@REDACTED Thu May 16 06:04:00 2013 From: skribent_har@REDACTED (Martin Hedberg) Date: Thu, 16 May 2013 06:04:00 +0200 Subject: [erlang-questions] A small question about macros Message-ID: Hi there Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). I read about how to make macros in the language. However I wonder how I do if I want to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) I am very grateful for any help. Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From esumbar@REDACTED Thu May 16 06:05:20 2013 From: esumbar@REDACTED (Edmund Sumbar) Date: Wed, 15 May 2013 22:05:20 -0600 Subject: [erlang-questions] using a registered name with lib_chan fails Message-ID: Hi (from an erlang newbie), I can get the name server example at the end of chapter 14 of Programming Erlang 2nd ed to work perfectly with just the kvs, mod_name_server, and lib_chan* modules compiled in their own directory. The code from the book is in a completely different directory. My search path is the default. When I register the Pid from the lib_chan:connect function and subsequently pass the registered name to the lib_chan:rpc function, the client process hangs. Using the registered name with lib_chan:cast works. This is what I get... <<< server >>> $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Processing $HOME/.erlang... done Eshell V5.10.1 (abort with ^G) 1> kvs:start(). true 2> lib_chan:start_server(). lib_chan starting:"/Users/esumbar/.erlang_config/lib_chan.conf" ConfigData=[{port,1234}, {service,nameServer,password,"ABXy45",mfa,mod_name_server, start_me_up,notUsed}] true <<< client >>> $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Processing $HOME/.erlang... done Eshell V5.10.1 (abort with ^G) 1> {ok, Pid} = lib_chan:connect("localhost", 1234, nameServer, "ABXy45", ""). {ok,<0.34.0>} 2> register(abc, Pid). true 3> lib_chan:cast(abc, {store, joe, "writing a book"}). {send,{store,joe,"writing a book"}} 4> lib_chan:rpc(Pid, {lookup, joe}). {ok,"writing a book"} 5> lib_chan:rpc(abc, {lookup, joe}). <<< no response >>> I looked at the lib_chan:rpc implementation. It's pretty straightforward. I suppose it?s hanging because there?s no matching receive pattern, but I can?t figure out why that would be. Any help would be appreciated. Ed From bengt.kleberg@REDACTED Thu May 16 06:49:11 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 16 May 2013 06:49:11 +0200 Subject: [erlang-questions] A small question about macros In-Reply-To: References: Message-ID: <1368679751.4774.4.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, What commands do you want to have blank spaces in? Erlang shell commands? It is not possible to use macros in the shell. bengt On Thu, 2013-05-16 at 06:04 +0200, Martin Hedberg wrote: > > > Hi there > > Have begun to take my first "baby steps" with Erlang (holding on to > books and tutorials). > I read about how to make macros in the language. However I wonder how > I do if I want > to have blank spaces in my commands? So I just can write, for example: > "repeat 5 times". > > Hope you don't see me as a heretic if I want to HyperTalk-ificat > Erlang a little bit. :-) > > I am very grateful for any help. > > Best regards > > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Thu May 16 07:40:38 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 16 May 2013 17:40:38 +1200 Subject: [erlang-questions] A small question about macros In-Reply-To: References: Message-ID: <70BA3E72-7DDA-4FA9-BA88-CCCCC061FCEE@cs.otago.ac.nz> On 16/05/2013, at 4:04 PM, Martin Hedberg wrote: > I read about how to make macros in the language. However I wonder how I do if I want > to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". What do you mean by "commands"? I'm not sure that Erlang has anything like that. Erlang macros were introduced as a quick and dirty solution to some real problems (giving names to numbers so they can be used in patterns, abstracting patterns the way functions abstract expressions). They are not meant as general purpose language extension mechanism. Nothing other than good taste and good sense will stop you doing -define(repeat, loop((). -define(times, ), fun () ->). -define(end_repeat, end)). loop(N, F) when is_integer(N), N > 0 -> F(), loop(N-1, F); loop(0, _) -> ok. foo(N) -> ?repeat N ?times io:fwrite('Going against the grain of a language is not helpful.~n') ?end_repeat. But please don't do that if you want me to read your code. > Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. Erlang is a _practice_, not a religion. I don't see you as a heretic, more as an isolationist. From ok@REDACTED Thu May 16 07:43:06 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 16 May 2013 17:43:06 +1200 Subject: [erlang-questions] Maps In-Reply-To: <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> Message-ID: <9FAEF070-D021-4943-B46D-4D71721FFA18@cs.otago.ac.nz> Oddly enough, the EEP I've been working on (or at least Part 1, as I've split it into four parts) _is_ syntax decoupled from the implementation. Here's a copy of the current draft. I'm about to go through the torture of adding Markdown, sigh. -------------- next part -------------- A non-text attachment was scrubbed... Name: eep-slot.md Type: application/octet-stream Size: 7351 bytes Desc: not available URL: From Ola.Backstrom@REDACTED Thu May 16 08:30:54 2013 From: Ola.Backstrom@REDACTED (=?iso-8859-1?Q?Ola__B=E4ckstr=F6m?=) Date: Thu, 16 May 2013 06:30:54 +0000 Subject: [erlang-questions] Maps In-Reply-To: <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> Message-ID: <3d962a4a8eaa4f8b9d8a7c1443c4d881@exsrv01.ls.local> Correction: It was Chris King that wrote what I cited below. I cut away the wrong part, sorry for the confusion. After reading the EPP I see that similar functions to B and C are called Update and Put but just acts on one pair at a time. -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Ola B?ckstr?m Sent: den 15 maj 2013 17:15 To: Erlang Subject: Re: [erlang-questions] Maps > On Wed, 15 May 2013 04:20:20 -0400, Jesper Louis Andersen wrote: [snip] > I understand that. But why must (a) this new fast map implementation > have an associated syntax, and (b) this syntax displace the syntax > which *is* needed to implement pattern-matching of frames? Could the syntax somehow be decoupled from the implementation, allowing for usage in both scenarios? I just played with the idea a little... so assume the map implementation is found in the module 'coolmap' A) To create a map, one would have to point out what implementation module that should do the job. M = coolmap#{foo => 1, bar => 2, baz =>3}. Since coolmap is an atom, the compiler knows this must be creation of a new map... thus translated to M= coolmap:create([{foo, 1},{bar, 2}, {baz, 3}]). The variable M should somewhere contain the module name, so that a call extract_module(M) returns coolmap. B) To update a map, M2 = M#{foo := 12}. Since M isn't an atom the compiler understands that this translates to M2= (extract_module(M)):update([{foo, 12}], M). C) To introduce new keys M3 = M2#{foo:=100, bary => 24} The Since => is used in the expression this will be translated to another function: M2= (extract_module(M2)):edit([{update, foo, 100},{set, foo, 12}], M2). The above would allow anybody to create new implementations, could it be useful to allow Frames and Maps to share syntax? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Thu May 16 08:39:40 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 16 May 2013 08:39:40 +0200 Subject: [erlang-questions] Maps In-Reply-To: <3d962a4a8eaa4f8b9d8a7c1443c4d881@exsrv01.ls.local> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> <3d962a4a8eaa4f8b9d8a7c1443c4d881@exsrv01.ls.local> Message-ID: <1368686380.4774.14.camel@sekic1152.rnd.ki.sw.ericsson.se> Please do not cut away parts of an email. Keep all, or remove all and let me look up the things I need in the archive. bengt On Thu, 2013-05-16 at 06:30 +0000, Ola B?ckstr?m wrote: > Correction: It was Chris King that wrote what I cited below. I cut away the wrong part, sorry for the confusion. > > After reading the EPP I see that similar functions to B and C are called Update and Put but just acts on one pair at a time. > > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Ola B?ckstr?m > Sent: den 15 maj 2013 17:15 > To: Erlang > Subject: Re: [erlang-questions] Maps > > > > On Wed, 15 May 2013 04:20:20 -0400, Jesper Louis Andersen wrote: > [snip] > > I understand that. But why must (a) this new fast map implementation > > have an associated syntax, and (b) this syntax displace the syntax > > which *is* needed to implement pattern-matching of frames? > > Could the syntax somehow be decoupled from the implementation, allowing for usage in both scenarios? > I just played with the idea a little... so assume the map implementation is found in the module 'coolmap' > > A) To create a map, one would have to point out what implementation module that should do the job. > > M = coolmap#{foo => 1, bar => 2, baz =>3}. > Since coolmap is an atom, the compiler knows this must be creation of a new map... thus translated to M= coolmap:create([{foo, 1},{bar, 2}, {baz, 3}]). > > The variable M should somewhere contain the module name, so that a call extract_module(M) returns coolmap. > > B) To update a map, > > M2 = M#{foo := 12}. > Since M isn't an atom the compiler understands that this translates to M2= (extract_module(M)):update([{foo, 12}], M). > > C) To introduce new keys > > M3 = M2#{foo:=100, bary => 24} > The Since => is used in the expression this will be translated to another function: > M2= (extract_module(M2)):edit([{update, foo, 100},{set, foo, 12}], M2). > > The above would allow anybody to create new implementations, could it be useful to allow Frames and Maps to share syntax? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Thu May 16 09:41:05 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 16 May 2013 08:41:05 +0100 Subject: [erlang-questions] A small question about macros In-Reply-To: References: Message-ID: <51948D91.7020104@llaisdy.com> Dear Martin If by commands you mean function names, you can have spaces in functions names. In erlang a function name is just an atom. If you want spaces in an atom you enclose it in single quotes: MyAtom = 'this atom is fine'. Example module: -module('te st'). -export(['do this'/0]). 'do this'() -> 'Ol Korekt!'. Usage: 1> c('te st'). {ok,'te st'} 2> 'te st':'do this'(). 'Ol Korekt!' :D Ivan On 16/05/2013 05:04, Martin Hedberg wrote: > > > Hi there > > Have begun to take my first "baby steps" with Erlang (holding on to > books and tutorials). > I read about how to make macros in the language. However I wonder how I > do if I want > to have blank spaces in my commands? So I just can write, for example: > "repeat 5 times". > > Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang > a little bit. :-) > > I am very grateful for any help. > > Best regards > > Martin > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From lukas@REDACTED Thu May 16 09:43:21 2013 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 16 May 2013 09:43:21 +0200 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: <43187.1366743687@snookles.snookles.com> References: <43187.1366743687@snookles.snookles.com> Message-ID: Hello Brian, Just letting you know that I have just merged a fix which changes the rwlock I mentioned before to an rlock. This should reduce the contention which you are seeing if it was caused by many calls to erlang:nodes(). Lukas On Tue, Apr 23, 2013 at 9:01 PM, Scott Lystig Fritchie < fritchie@REDACTED> wrote: > Brian Picciano wrote: > > bp> We have a pool of 3 erlang nodes, all on different servers. Every > bp> afternoon, without fail, we start seeing lots of messages between > bp> the nodes start having really high latency, on the order of tens of > bp> seconds. [...] > > Brian, it's probably worthwhile to continue chasing the 'lcnt' avenue > as you've been corresponding with Lukas... > > ... but at the same time, I also wonder about "tens of seconds". My gut > says that such delays would require some amazingly high lock contention > rates. Something that can cause such messaging delays much more easily > is network congestion/packet loss that triggers TCP slow start. Many > Linux kernels have the RTO_min value at one second, which is the amount > of time to wait before entering slow start state. > > If network packet loss is a problem, this blog posting can explain one > reason why it's happening: > http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/ > > -Scott > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu May 16 10:30:46 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 16 May 2013 10:30:46 +0200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: References: Message-ID: Hello Richard, To silence the undefined error, the parser would need to be able to know it indeed gave up on it; that is not so straightforward. For the syntax error to be more useful, that also would require either heavy yecc surgery or rewritting erl_parse to be an handwritten descent-recursive parser --that is a thing I've always wanted to do. For pairs we can indeed use epp to do it as it already needs to track them for macro arguments. Otherwise we can still improve the situation quite a lot with just column numbers, when they get merged someday. Regards, -- Anthony Ramine Le 16 mai 2013 ? 05:04, Richard A. O'Keefe a ?crit : > I'm in the middle of writing to someone who asked for help with > some Erlang code. With identifiers replaced, he had > > 21: f(A, B, C) -> > 22: D = lists:map(fun (E) -> > 23: case proplist:get_value(E, B, error) of > 24: {F} -> {E,F}; > 25: error -> {error} > 26: end, A). > > erlc reported > > ./foobar.erl:26: syntax error before: ')' > ./foobar.erl:16: function f/3 undefined > > When there are error messages that prevent any function from being processed, > it would be helpful if erlc did *NOT* complain about undefined, because that > message is untrue. The function *was* defined, the compiler just gave up on > it. > > But the message that really had him confused was the "syntax error before ')'" > one. This is the kind of error message one typically gets from something > using Yacc-style parsing. > > I didn't like his layout, so I fed his file to an Erlang pretty-printer I > wrote a couple of years. The code is fast but rather horrible C, so I do > not care to share it. It was thrown together as an experiment, with an > eye to using it in a text editor. What did that pile of hacks say? > > Line 26: 'fun' at line 22 closed by ')' (expected 'end'). > > So a better error message is certainly possible. > > This is a pretty-printer, not a parser. Almost the only syntax checking it > does is to keep track of brackets. But bracket errors are common enough > that it's worth while. > > I don't like the idea of slowing erlc down, but I wonder if decent bracket > matching checks could be folded in with macro expansion without too much > trouble? > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Thu May 16 12:38:27 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 16 May 2013 12:38:27 +0200 Subject: [erlang-questions] Maps In-Reply-To: <9FAEF070-D021-4943-B46D-4D71721FFA18@cs.otago.ac.nz> References: <518D93AF.5030502@gmail.com> <108853326.112583902.1368536829898.JavaMail.root@erlang-solutions.com> <143EC204-A55F-42A7-BD17-81F3D45B3B7A@erlang-solutions.com> <000751f873344714920b9bef44c8ade5@exsrv01.ls.local> <9FAEF070-D021-4943-B46D-4D71721FFA18@cs.otago.ac.nz> Message-ID: <5194B723.6010305@ninenines.eu> On 05/16/2013 07:43 AM, Richard A. O'Keefe wrote: > Oddly enough, the EEP I've been working on (or at least Part 1, as I've > split it into four parts) _is_ syntax decoupled from the implementation. > > Here's a copy of the current draft. I'm about to go through the torture > of adding Markdown, sigh. If I understand right, you need '#[]' defined in the local module, and as such can only have one defined? This is going to be annoying when you want to use this syntax on both records and JSON and some other structure in the same module. Why not just have Erlang define a default function if one isn't available that does what people expect depending on the type? * lists, tuples: integer N fetching nth element * proplists: atom A doing a keyfind * dicts, maps: any A doing an equivalent of a keyfind * etc. You'd still allow overriding it but it would do the sane thing by default. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From egil@REDACTED Thu May 16 15:23:35 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 16 May 2013 15:23:35 +0200 Subject: [erlang-questions] Maps In-Reply-To: References: <518A5ED1.2030507@erlang.org> <518A61B7.8030201@ninenines.eu> <49E3FA0B-C195-4A4A-B766-9FBDA8FB71EC@cs.otago.ac.nz> Message-ID: <5194DDD7.8050601@erlang.org> On 2013-05-10 05:03, Richard A. O'Keefe wrote: > On 9/05/2013, at 8:10 PM, Max Lapshin wrote: > >> Richard. Would you, kindly, tell: where is the difference between maps >> and frames? >> >> As far as I understand, it is not about syntax, it is about internal >> way of storing data? > I thought I already had, but here goes. > > There is one central semantic difference, which leads to two > others. > > (U) What do you want to optimise the design for? > > Frames are optimised (pared to the bone, in fact) for use in > record-like ways. They are somewhere between pathetic and > hopeless as general purpose dictionaries. > > Maps optimised for service as general-purpose dictionaries. > They are somewhere between almost usable and pathetic for > record-like uses. > > I do not believe that it is possible to design a data structure > which is *good* at both tasks. I suspect that it is possible to > prove this, but I must admit I haven't tried. > > THIS is the reason that Bj?rn-Egil Dahlberg was able to sneer at > frames as "just a replacement". I don't try to build perpetual > motion machines. I don't try to square the circle with ruler and > compass. I don't try to design bottle-openers that make good > chisels. And given two sets of conflicting requirements (even if > the sets overlap substantially), I don't try to satisfy them both > with a single data structure. (Using a Java ArrayList as a queue > is *possible*, but stupid.) Richard, I'm sorry that you feel that way. It was not my intention to "sneer" at Frames. I merely stated that Frames is intended as a record replacement whereas Maps is not. I also said: "The one suggestion that stands out is of course the Frames proposal from Richard O'Keefe. It is the most complete proposal I've seen and is very well thought out." .. but I wanted something else, hence the EEP. > As for implementation, Bj?rn-Egil Dahlberg wrote: > >> Current prototypes stems from using sparse tuples in a >> HAMT-like data structure and tuple-like data structures. >> The HAMT-like data structure is discontinued and will be >> replaced by some ordered tree. > The HAMT-like data structure for OCAML that I've looked at > takes *at least* four words for every key/value association. > (I have been able to find plenty of numbers for the *speed* > of HAMTs but nothing for the *space* they need other than a > claim of "small".) That's not necessarily the same thing, > and the "tuple-like" part makes me wonder if something much > cleverer is being done. I don't know how OCAML does it but that word count seems excessive. Now, the HAMT impl. is discontinued because it was decided an ordered structured was going to be used. But, here is what I had i mind: First things first. HAMT is a functional structure, meaning no destructive updates. In my implementation I used the structure of an radix tree. In the tree we have three types of nodes, leaf nodes, tuple nodes and "sparse-tuple" nodes. Tuple nodes are a saturated "sparse-tuple". It is a plain tuple of fixed size. Each element in the tuple is a leaf node, tuple node or "sparse-tuple". Leaf nodes are conses with key values [K|V]. Leaf nodes not are not really necessary since we can save KVs directly in the other nodes but it simplifies implementation and was used in the prototype. Storing KVs directly in tuple and "sparse-tuple" nodes saves one word per KV but it also doubles the number of words that needs copying in each update. This boils down to a trade-off between speed and memory. "Sparse-tuple" nodes is an internal construct, but a real type, only used in this data structure. Elements may be indexed 1 - N but it would *not* take N + 1 words in memory. Instead a mapping function is used to index elements. We also need the information of which elements in the sparse tuple are set. This information takes about one word. The number of words required depends on the node size (which is fixed), but can be placed in the header of this type. So the memory size of a sparse tuple, indexable 1 - N with only two elements, takes 3 - 4 words (3 if the mapping is in the header). N is typically 16 or 32. What this means is that we can take a hashing approach to this but still keep the tree compact. I believe this is meant to be the "small" memory footprint. The key to be used to traverse tree is hashed (using the same algorithm as phash2) which produces an 32 bit integer. We index the tree by using a set of bits from this hash at each level. The number of bits used is dependent on the fixed size of the nodes. In my prototype i used nodes of size 32. For each level in the tree we use 5 bits of the hash to index down in the tree. When the hash is exhausted we produce an new hash but add the level as well, ex. phash([Lvl|Key]). When we update a value in the tree only the nodes in the path are updated. "sparse-tuples" reduces the amount of copying. The space this tree requires depends on the hashing algorithm (and the keys that are hashed). Some "management size" is required ofc. The root of this structure may be implemented in several ways. For instance we can consider a variant of the "sparse-tuple" as the root. The management size requirement would be 2 words, a header defining the type and keeping the mapping, and a Eterm for the number of entries in the tree. If we optimize for space we would require one word header per node in the tree, the rest of the elements are key/values. I have no exact value for words/pair but I think you get it. It also worth noting that, * the tree would be shallow, * only one key comparison is done while traversing the tree (leaf node). It is also worth noting that the mapping function in "sparse-tuples" is an expensive one. We need hamming weight of a bitfield. This can be done in newer hardware using a popcount instruction. Doing popcount in Erlang code is doable but not very efficient unless compiler/runtime aided. This whole explanation serves as one big parenthesis since HAMT will not be used. Space conservation, key-sharing and ordering requirements were added. > However, we're told that's going "and > will be replaced by some ordered tree". Suppose you have a > 2-3-4 tree. We can play SML/Mercury-style games to move the > size of a node into the pointer to it, so we > > 2-node: C1 K1 V1 C2 4 words/pair > 3-node: C1 K1 V1 C2 K2 V2 C3 3.5 words/pair > 4-node: C1 K1 V1 C2 K2 V2 C3 K3 V3 C4 3.3+words/pair > > So it's fair to say that this would take *roughly* 3.5 words > per field. > > Playing similar move-the-balance-info-into-the-pointers games > would let us do red-black trees or AVL trees in 4 words/pair. > > There are other games one can play, at the price of more > complicated code, where we take advantage of the leaves of > a 2-3-4 tree being at the same depth, so that leaf nodes > are K1 V1 > or K1 V1 K2 V2 > or K1 V1 K2 V2 K3 V3 > or K1 V1 K2 V2 K3 V3 K4 V4 > and the tree as a whole is roughly 2.3 words per pair. > > I don't know what exactly Bj?rn-Egil Dahlberg has in mind, > but if we charge 3 words per key/value association, it's > not likely that we're overestimating the space cost. Maps would have a two tier approach. Tier 1 would have an approach much like your frames { {K1, ..., Kn}, V1, ..., Vn }, essentially two tuples where keys are ordered in term order. When the number of entries reaches a certain point the cost of doing business becomes too great. Searching and the garbage generated by updates costs more than space saving earns us. Typically 30 - 50 entries. At this point the structure is converted to some ordered tree, Tier 2. I say "some" ordered tree because it is still undecided what is suitable here. I think it is fair to say that space cost here would be 3 - 4 words per pair. I would view Tier 1 as an optimization of Tier 2 for few entries. I would also like to point out that when this point is reached, space is not our primary concern. Regards, Bj?rn-Egil From ivan@REDACTED Thu May 16 16:29:45 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 16 May 2013 15:29:45 +0100 Subject: [erlang-questions] lager dropping invisible messages Message-ID: <5194ED59.4030202@llaisdy.com> Dear All I'm using lager for logging and I've recently switched to lager 2.*. I occasionally get this warning message: [warning] lager_error_logger_h dropped N messages in the last second that exceeded the limit of 50 messages/sec where N ranges between 35-78. However, nowhere in my logs can I find a bunch of 50 messages written within a second, or anything that might be linked to this warning. I haven't been able to find an erlang lager mailing list. Does anyone know if this lager warning might be warning me about anything important, if so what, if not how do I switch it off? With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From skribent_har@REDACTED Thu May 16 17:29:56 2013 From: skribent_har@REDACTED (Martin Hedberg) Date: Thu, 16 May 2013 17:29:56 +0200 Subject: [erlang-questions] Just want to say "thanks". In-Reply-To: References: Message-ID: Hi again Hope I am not spamming the thread, by not posting a question. I just want to say thank you for your help. It's wonderful to have some people to ask, when you work with a (what I think) difficult language like Erlang. Hope I can repay you all (with answers) when my knowledge of the language gets a bit better. Best regards Martin From: skribent_har@REDACTED To: erlang-questions@REDACTED Subject: A small question about macros Date: Thu, 16 May 2013 06:04:00 +0200 Hi there Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). I read about how to make macros in the language. However I wonder how I do if I want to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) I am very grateful for any help. Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From skribent_har@REDACTED Thu May 16 22:41:43 2013 From: skribent_har@REDACTED (Martin Hedberg) Date: Thu, 16 May 2013 22:41:43 +0200 Subject: [erlang-questions] Basic atom question In-Reply-To: References: , Message-ID: Hello there Newbie wan Kenobi here again with a small question. :-) When the system restarts atoms are cleared but I wonder: Is it enough to restart the process that the atoms were declared in, to clear the memory from it? Best regards Martin From: skribent_har@REDACTED To: erlang-questions@REDACTED Subject: A small question about macros Date: Thu, 16 May 2013 06:04:00 +0200 Hi there Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). I read about how to make macros in the language. However I wonder how I do if I want to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) I am very grateful for any help. Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Thu May 16 23:27:06 2013 From: dmercer@REDACTED (David Mercer) Date: Thu, 16 May 2013 16:27:06 -0500 Subject: [erlang-questions] Basic atom question In-Reply-To: References: , Message-ID: <000c01ce527c$1e0501b0$5a0f0510$@gmail.com> No. After all, the atoms could have been sent to another process. If I understand it correctly, atoms defined in modules would be allocated when the module is loaded. All other atoms allocated when first created. Atoms I don?t think are ever garbage collected. Cheers, DBM From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Martin Hedberg Sent: Thursday, May 16, 2013 15:42 To: erlang-questions@REDACTED Subject: [erlang-questions] Basic atom question Hello there Newbie wan Kenobi here again with a small question. :-) When the system restarts atoms are cleared but I wonder: Is it enough to restart the process that the atoms were declared in, to clear the memory from it? Best regards Martin _____ From: skribent_har@REDACTED To: erlang-questions@REDACTED Subject: A small question about macros Date: Thu, 16 May 2013 06:04:00 +0200 Hi there Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). I read about how to make macros in the language. However I wonder how I do if I want to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) I am very grateful for any help. Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Thu May 16 23:46:29 2013 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 16 May 2013 17:46:29 -0400 Subject: [erlang-questions] lager dropping invisible messages In-Reply-To: <5194ED59.4030202@llaisdy.com> References: <5194ED59.4030202@llaisdy.com> Message-ID: <20130516214629.GI3966@hijacked.us> On Thu, May 16, 2013 at 03:29:45PM +0100, Ivan Uemlianin wrote: > Dear All > > I'm using lager for logging and I've recently switched to lager 2.*. > > I occasionally get this warning message: > > [warning] lager_error_logger_h dropped N messages in the last > second that exceeded the limit of 50 messages/sec > > where N ranges between 35-78. > > However, nowhere in my logs can I find a bunch of 50 messages > written within a second, or anything that might be linked to this > warning. > > I haven't been able to find an erlang lager mailing list. > > Does anyone know if this lager warning might be warning me about > anything important, if so what, if not how do I switch it off? https://github.com/basho/lager#overload-protection https://github.com/basho/lager/pull/116 Lager does not (yet) have a mailing list, but I ususally notive lager related messages here or on riak-users. Andrew From ok@REDACTED Fri May 17 03:08:06 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 17 May 2013 13:08:06 +1200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: References: Message-ID: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> On 16/05/2013, at 8:30 PM, Anthony Ramine wrote: > Hello Richard, > > To silence the undefined error, the parser would need to be able to know it indeed gave up on it; that is not so straightforward. > > For the syntax error to be more useful, that also would require either heavy yecc surgery or rewritting erl_parse to be an handwritten descent-recursive parser --that is a thing I've always wanted to do. It all depends on the structure of the compiler. For example, I have an AWK->Java compiler written in C++ as an exercise; it needs some work on the symbol table and the runtime needs finishing, but then it will be open-sourced. There are three phases, not entirely unlike Erlang. (1) Lexical analysis builds a sequence of token records in memory. (2) Parsing recycles those tokens into an AST, resolves variables, and does some very crude type inference. (3) Code generation emits Java. If a lexical error occurs, no parsing is done. And brackets are checked during lexical analysis: the lexical analyser only needs to keep a simple stack of what brackets are open. My crude Erlang pretty-printer plays the same check-brackets-during- lexical-analysis-with-a-straightforward-stack trick. In the days when compiling meant submitting a deck of punched cards in the morning and getting a listing in the late afternoon, it was important for a compiler to check as much as it could. These days, it is more important that a compiler's messages should not be misleading, and parse errors following lexical errors (including bracket errors as lexical errors) are generally pretty misleading. > Otherwise we can still improve the situation quite a lot with just column numbers, when they get merged someday. In this specific case, it would not have helped. It was perfectly clear exactly where the parser ran into trouble. It just wasn't clear _why_ it ran into trouble. Part of the problem is precisely the fact that the parser is written in Yecc. From yashgt@REDACTED Fri May 17 08:17:54 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Fri, 17 May 2013 06:17:54 +0000 Subject: [erlang-questions] Message delivery in case of process restart Message-ID: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Fri May 17 08:24:06 2013 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 17 May 2013 08:24:06 +0200 Subject: [erlang-questions] Message delivery in case of process restart In-Reply-To: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> References: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> Message-ID: They are lost. New instance of this process will have new Pid. There are no built-in store-and-forward features in Erlang, but they can be implemented by a proxy process. But I never had a need to do that in all of my projects. What is your use case? On 17 May 2013 08:18, wrote: > Hi, > > If a process crashes and is restarted by the supervisor, what happens to > the messages that were sent to the process after it crashed and before it > restarted? Are they collected and stored somewhere and are they sent to the > process when it is ready to receive? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri May 17 08:31:46 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 17 May 2013 12:01:46 +0530 Subject: [erlang-questions] Message delivery in case of process restart In-Reply-To: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> References: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> Message-ID: <20130517063146.GA9295@aluminium.local> On Fri, May 17, 2013 at 06:17:54AM +0000, yashgt@REDACTED wrote: } If a process crashes and is restarted by the supervisor, what } happens to the messages that were sent to the process after it } crashed and before it restarted? http://www.erlang.org/doc/reference_manual/expressions.html#send "Sending a message to a pid never fails, even if the pid identifies a non-existing process." } Are they collected and stored somewhere and are they sent to the } process when it is ready to receive? No. -- -Vance From lee.sylvester@REDACTED Fri May 17 09:27:27 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Fri, 17 May 2013 08:27:27 +0100 Subject: [erlang-questions] Just want to say "thanks". In-Reply-To: References: Message-ID: I concur with Martin. However, I've not just had great help from the Erlang list, but from Erlang project lists, too. Its rare to have such a great community. The only other community I've experienced this with is the Haxe community. It makes me think I've chosen the right language :-) Lee On 16 May 2013, at 16:29, Martin Hedberg wrote: > > > Hi again > > Hope I am not spamming the thread, by not posting a question. I just want to say thank you for your help. > It's wonderful to have some people to ask, when you work with a (what I think) difficult language like Erlang. > > Hope I can repay you all (with answers) when my knowledge of the language gets a bit better. > > Best regards > > Martin > > > > > > From: skribent_har@REDACTED > To: erlang-questions@REDACTED > Subject: A small question about macros > Date: Thu, 16 May 2013 06:04:00 +0200 > > > > Hi there > > Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). > I read about how to make macros in the language. However I wonder how I do if I want > to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". > > Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) > > I am very grateful for any help. > > Best regards > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Fri May 17 10:27:28 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 17 May 2013 10:27:28 +0200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> Message-ID: <9E7445F0-7A53-4AD9-9E71-1BE26D105F86@gmail.com> While I'm absolutely buying your argument about checking brackets during lexical analysis, I'm not sure the OTP team would like that seeing how they are quite frisky with my =<< fix; wouldn't checking them there bring syntax knowledge into the lexer, something they want to avoid? Regards, -- Anthony Ramine Le 17 mai 2013 ? 03:08, Richard A. O'Keefe a ?crit : > > On 16/05/2013, at 8:30 PM, Anthony Ramine wrote: > >> Hello Richard, >> >> To silence the undefined error, the parser would need to be able to know it indeed gave up on it; that is not so straightforward. >> >> For the syntax error to be more useful, that also would require either heavy yecc surgery or rewritting erl_parse to be an handwritten descent-recursive parser --that is a thing I've always wanted to do. > > It all depends on the structure of the compiler. > For example, I have an AWK->Java compiler written in C++ as an exercise; > it needs some work on the symbol table and the runtime needs finishing, > but then it will be open-sourced. > > There are three phases, not entirely unlike Erlang. > (1) Lexical analysis builds a sequence of token records in memory. > (2) Parsing recycles those tokens into an AST, resolves variables, > and does some very crude type inference. > (3) Code generation emits Java. > > If a lexical error occurs, no parsing is done. > And brackets are checked during lexical analysis: > the lexical analyser only needs to keep a simple stack > of what brackets are open. > > My crude Erlang pretty-printer plays the same check-brackets-during- > lexical-analysis-with-a-straightforward-stack trick. > > In the days when compiling meant submitting a deck of punched cards in the > morning and getting a listing in the late afternoon, it was important for > a compiler to check as much as it could. > > These days, it is more important that a compiler's messages should not be > misleading, and parse errors following lexical errors (including bracket > errors as lexical errors) are generally pretty misleading. > >> Otherwise we can still improve the situation quite a lot with just column numbers, when they get merged someday. > > In this specific case, it would not have helped. It was perfectly clear > exactly where the parser ran into trouble. It just wasn't clear _why_ it > ran into trouble. > > Part of the problem is precisely the fact that the parser is written in Yecc. > From ivan@REDACTED Fri May 17 10:39:58 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 17 May 2013 09:39:58 +0100 Subject: [erlang-questions] lager dropping invisible messages In-Reply-To: <20130516214629.GI3966@hijacked.us> References: <5194ED59.4030202@llaisdy.com> <20130516214629.GI3966@hijacked.us> Message-ID: <5195ECDE.1000301@llaisdy.com> Dear Andrew Thanks, I am familiar with those two web pages. The mystery is not "what is it supposed to do?". The mystery is "where are the 50 messages it thinks it's written within the last second?". Often, as far as I can tell it hasn't written any. Ivan On 16/05/2013 22:46, Andrew Thompson wrote: > On Thu, May 16, 2013 at 03:29:45PM +0100, Ivan Uemlianin wrote: >> Dear All >> >> I'm using lager for logging and I've recently switched to lager 2.*. >> >> I occasionally get this warning message: >> >> [warning] lager_error_logger_h dropped N messages in the last >> second that exceeded the limit of 50 messages/sec >> >> where N ranges between 35-78. >> >> However, nowhere in my logs can I find a bunch of 50 messages >> written within a second, or anything that might be linked to this >> warning. >> >> I haven't been able to find an erlang lager mailing list. >> >> Does anyone know if this lager warning might be warning me about >> anything important, if so what, if not how do I switch it off? > > https://github.com/basho/lager#overload-protection > https://github.com/basho/lager/pull/116 > > Lager does not (yet) have a mailing list, but I ususally notive lager > related messages here or on riak-users. > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From vincent.dephily@REDACTED Fri May 17 10:58:53 2013 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 17 May 2013 10:58:53 +0200 Subject: [erlang-questions] Maps In-Reply-To: <5194B723.6010305@ninenines.eu> References: <518D93AF.5030502@gmail.com> <9FAEF070-D021-4943-B46D-4D71721FFA18@cs.otago.ac.nz> <5194B723.6010305@ninenines.eu> Message-ID: <2517790.OTxdF9hIdp@moltowork> On Thursday 16 May 2013 12:38:27 Lo?c Hoguin wrote: > On 05/16/2013 07:43 AM, Richard A. O'Keefe wrote: > > Oddly enough, the EEP I've been working on (or at least Part 1, as I've > > split it into four parts) _is_ syntax decoupled from the implementation. > > > > Here's a copy of the current draft. I'm about to go through the torture > > of adding Markdown, sigh. > > If I understand right, you need '#[]' defined in the local module, and > as such can only have one defined? > > This is going to be annoying when you want to use this syntax on both > records and JSON and some other structure in the same module. That worried me when reading the draft too. I expect that the answer is to define : > '#[]'({my_type, V}, K) -> extract_mytype(K,V); > '#[]'(Other, K) -> erlang_default:'#[]'(Other, K). But this can become ugly and unwieldy quickly. Perhaps we need a standard wraper function : > '#[]'(K,V,[]) -> erlang:error(badarg). > '#[]'(K,V,[M|T]) -> try M:'#[]'(K,V) catch error:badarg -> '#[]'(K,V,T). That we can use in each module as : > '#[]'(K,V) -> '#[]'(K,V,[mytype,json,maps,lists...]). But you need to order your modules very carefully, puting the most selective ones first. Appart from that, while this eep looks interesting, it doesn't seem to address pattern matching ? Or did I read incorrectly and you actually can > case Json of > Json#['class']#['foo'] -> handle_foo(Json); > Json#['class']#['bar'] -> handle_bar(Json); > end. ? -- Vincent de Phily From jon@REDACTED Fri May 17 11:09:16 2013 From: jon@REDACTED (Jonathan Schneider) Date: Fri, 17 May 2013 10:09:16 +0100 Subject: [erlang-questions] Message delivery in case of process restart In-Reply-To: <20130517063146.GA9295@aluminium.local> References: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> <20130517063146.GA9295@aluminium.local> Message-ID: <88ABB713-3492-4EBB-8139-DC75DF50D96F@axismilton.ltd.uk> > "Sending a message to a pid never fails, even if the pid > identifies a non-existing process." > The documentation could be clearer. It means never causes an error for the sender. Jon From wallentin.dahlberg@REDACTED Fri May 17 11:11:25 2013 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Fri, 17 May 2013 11:11:25 +0200 Subject: [erlang-questions] Maps In-Reply-To: <2517790.OTxdF9hIdp@moltowork> References: <518D93AF.5030502@gmail.com> <9FAEF070-D021-4943-B46D-4D71721FFA18@cs.otago.ac.nz> <5194B723.6010305@ninenines.eu> <2517790.OTxdF9hIdp@moltowork> Message-ID: <-3814756186832017250@unknownmsgid> I like this idea of deep update, haven't look too closely though. :) But please, discuss this in a separate thread. // egil Sent from my iPhone 17 maj 2013 kl. 10:58 skrev Vincent de Phily : > On Thursday 16 May 2013 12:38:27 Lo?c Hoguin wrote: >> On 05/16/2013 07:43 AM, Richard A. O'Keefe wrote: >>> Oddly enough, the EEP I've been working on (or at least Part 1, as I've >>> split it into four parts) _is_ syntax decoupled from the implementation. >>> >>> Here's a copy of the current draft. I'm about to go through the torture >>> of adding Markdown, sigh. >> >> If I understand right, you need '#[]' defined in the local module, and >> as such can only have one defined? >> >> This is going to be annoying when you want to use this syntax on both >> records and JSON and some other structure in the same module. > > > That worried me when reading the draft too. I expect that the answer is to > define : >> '#[]'({my_type, V}, K) -> extract_mytype(K,V); >> '#[]'(Other, K) -> erlang_default:'#[]'(Other, K). > > But this can become ugly and unwieldy quickly. Perhaps we need a standard > wraper function : >> '#[]'(K,V,[]) -> erlang:error(badarg). >> '#[]'(K,V,[M|T]) -> try M:'#[]'(K,V) catch error:badarg -> '#[]'(K,V,T). > > That we can use in each module as : >> '#[]'(K,V) -> '#[]'(K,V,[mytype,json,maps,lists...]). > > But you need to order your modules very carefully, puting the most selective > ones first. > > > > Appart from that, while this eep looks interesting, it doesn't seem to address > pattern matching ? Or did I read incorrectly and you actually can >> case Json of >> Json#['class']#['foo'] -> handle_foo(Json); >> Json#['class']#['bar'] -> handle_bar(Json); >> end. > > ? > -- > Vincent de Phily > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pan@REDACTED Fri May 17 11:11:44 2013 From: pan@REDACTED (Patrik Nyblom) Date: Fri, 17 May 2013 11:11:44 +0200 Subject: [erlang-questions] [eeps] Maps In-Reply-To: <518A5ED1.2030507@erlang.org> References: <518A5ED1.2030507@erlang.org> Message-ID: <5195F450.6010409@erlang.org> Hi! I think we need to clarify a few things from the perspective of the technical board that gave Bj?rn-Egil the premises for this EEP. 1) Maps are Maps and Frames are Frames. Maps is not some kind of extension to Frames, as Frames hold properties that Maps never should hold. What was decided, and what Bj?rn-Egil is working on, was to investigate if a clever implementation of a dictionary can be a good enough record replacement to be used in situations like interfaces between modules, instead of records and property lists. As the implementation is not yet done, we only have the interface to Maps to discuss. Even if the EEP mentions complexity, we still have to see if the implementation in a "record-like" situation can be made efficient both when it comes to memory consumption and time complexity. 2) Maps is *not* to be implemented using HAMT's, so discussing the complexity or implementation of HAMT's is interesting, but beside the point. The task is to define an ordered dictionary. Maps have keys in order. How that is implemented is not defined and that job is yet to be done. A maximal O(log N) complexity in look up was the only requirement, but an implementation with low memory consumption for "simple" Maps was *suggested*. Discussing characteristics of an implementation that is no more than a general idea seems futile. There are expectations, but no facts about the performance characteristics. 3) The behavior of float and integer keys that compare equal is the same as the one for ordered_set ETS tables. Even though we don't like it, that behavior is the only consistent we've found for ordered dictionaries. If we want to define two types of comparison in Erlang, that is another discussion. (An EEP for that would be welcome. Do not forget to add suggestions for all current interfaces where order is used, like lists:sort, orddict etc.) 4) Maps is supposed to be general, a general mapping from keys to values. The thinking was that the current data types (at least the compound ones) are very general. Also the current dictionaries have no restrictions on the keys or values. A Map data type should of course not have that either. When used in place of records, the Map gives maybe to much freedom, that is the price to pay when being very general. 5) To be able to differentiate between operations where you possibly add new keys and operations where you replace existing keys was a requirement. The arguments have already been outlined in Joe's earlier posts, so I see no point in repeating that. 6) Imposing an order on the keys was thought to make the data type fit better with the current built in types. Using for example the HAMT implementation to describe the order between Maps, to have a serialization looking different depending on how the map was constructed, or having random order of the keys when serializing or displaying the data, would make the Map an odd creature in the Erlang world. If we for example want to serialize and send two Maps to a program written in another language, we would like to be able to describe the order between the Maps so that comparison could be done in that other language. The order between Maps as they look in this EEP is easily described and comparison is simple to implement. Suggestions like "sorting the MAP before comparing and printing" was rejected. Making a comparison between two elements possibly have the complexity of O(N log N) was thought unacceptable. 7) To publish an EEP even before there was a prototype implementation was a requirement from the technical board, as we wanted feedback on the API, the semantics etc. I think this thread has shown that there is definitely a lot to discuss. It however also shows that as long as there is no description of the actual implementation, there's a lot that cannot be discussed, but we really would want to discuss... So - these were the premises and I think Bj?rn-Egil has done a really good job in writing this EEP. A lot of people has also taken the time to give valuable feedback. I think the EEP and the current feedback gives me reason enough to suggest a prototype implementation. Possibly with a few adjustments to the EEP text, examples etc. Cheers, Patrik On 05/08/2013 04:18 PM, Bj?rn-Egil Dahlberg wrote: > Hi everyone! > > We finally have a Maps EEP for you. This will get you an idea on what > we are working on. Some of the ideas presented here was presented at > Erlang Factory SF Bay Area 2013. > > I am sure that there will be some discussions about the contents of > this EEP and I hope the discussions will be both fruitful and civilized. > > The journey of Maps and this EEP has been long and by no means a > straight-forward or continuous one. I had a crystal clear picture of > what I wanted Maps to be when we first started discussing it within > OTP about two-three years ago. This EEP resembles that vision but it > has had a lot of contributions of other ideas from both within and > outside of OTP. > > The idea was a functional data-type, a syntax aware mapping of > key-value associations with pattern matching. A syntax similar to > records but without the hazzle of compile-time dependency and with > arbitrary terms as keys. Order was not important and it could be > implemented with a Hash-Array-Mapped-Trie with good performance and > memory trade-offs. This was not an approach to replace records. It was > meant to replace records where suitable and in other regards not be a > replacement but its own thing. > > From the community there has been many wishes of a Map like data-type > and a few suggestions. The one suggestion that stands out is of > course the Frames proposal from Richard O'Keefe. It is the most > complete proposal I've seen and is very well thought out. Its goal is > to be a record replacement and the proposal satisfies this goal very > well. > > - If Frames are that good, why a separate EEP? > - It boils down to goals and constraints. > > A record replacement is just that, a replacement. > It's like asking the question, "What do we have?" instead of "What can > we get?" > The instant rebuttal would be "What do we need?" I say Maps. > > Frames has certainly influenced Maps. In many regards Maps also > encompasses Frames but Maps tries to do more. I think the most > significant difference would be, arbitrary terms as keys and how many > different keys we would have in a Map. In the end I believe they are > two different things and have different goals. > > Some Notes and Disclaimers: > > Later iterations of Maps has gone through some changes, most > significantly, > > * From a set of key-values to a ordered set of key-value associations > > I was originally against this change since it forces restrictions on > the implementation and it illuminates the lack of distinction between > arithmetic order and term order, i.e. the problem of mixing integer > and float types as keys in a tree. However, I was later persuaded that > key ordering is necessary. We have to respect the totalitarian order > of terms. > > Considerations has been made on how to, if at all possible, apply > Frames characteristics to Maps. Most significantly memory space and > key-sharing characteristics. This is not detailed in the EEP though, > just mentioned. > > The function interface has had many revisions as well. At some stage > the API was considered to be a drop-in-replacement for `dict` and thus > would have the same function-names. This goal/constraint was dropped > by Technical Board decision recently. > > From the very beginning Maps was envisioned to have the ability to > bind variables derived from the environment. Like this: > > function(K1, #{ K1 := K2, K2 := V }) -> V. > > This feature is a beast. Powerful and scary. It is not confined to > only Maps but should also be applicable to other types as well: > > function(Skip, <<_:Skip/binary, Value:Size, _/bits>>, Size) -> Value. > > It is uncertain how effective such an implementation would be and in > the end we might not want this feature at all. > > In this EEP we will describe syntax and semantics of Maps but very > little is disclosed of its actual implementation. Current prototypes > stems from using sparse tuples in a HAMT-like data structure and > tuple-like data structures. The HAMT-like data structure is > discontinued and will be replaced by some ordered tree. > > The proposal is included as an attachment but can also be viewed at > this git-repository: > https://github.com/psyeugenic/eep/blob/egil/maps/eeps/eep-0043.md > > > Regards, > Bj?rn-Egil Dahlberg > Erlang/OTP > > > _______________________________________________ > eeps mailing list > eeps@REDACTED > http://erlang.org/mailman/listinfo/eeps -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri May 17 12:02:30 2013 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 17 May 2013 12:02:30 +0200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> Message-ID: On Fri, May 17, 2013 at 3:08 AM, Richard A. O'Keefe wrote: > > On 16/05/2013, at 8:30 PM, Anthony Ramine wrote: > > > Hello Richard, > > > > To silence the undefined error, the parser would need to be able to know > it indeed gave up on it; that is not so straightforward. > > > > For the syntax error to be more useful, that also would require either > heavy yecc surgery or rewritting erl_parse to be an handwritten > descent-recursive parser --that is a thing I've always wanted to do. > > It all depends on the structure of the compiler. > For example, I have an AWK->Java compiler written in C++ as an exercise; > it needs some work on the symbol table and the runtime needs finishing, > but then it will be open-sourced. > > There are three phases, not entirely unlike Erlang. > (1) Lexical analysis builds a sequence of token records in memory. > (2) Parsing recycles those tokens into an AST, resolves variables, > and does some very crude type inference. > (3) Code generation emits Java. > > If a lexical error occurs, no parsing is done. > And brackets are checked during lexical analysis: > the lexical analyser only needs to keep a simple stack > of what brackets are open. > That's a nice idea - we could easily stick an extra pass between scanning and parsing to do this /Joe > > My crude Erlang pretty-printer plays the same check-brackets-during- > lexical-analysis-with-a-straightforward-stack trick. > > In the days when compiling meant submitting a deck of punched cards in the > morning and getting a listing in the late afternoon, it was important for > a compiler to check as much as it could. > > These days, it is more important that a compiler's messages should not be > misleading, and parse errors following lexical errors (including bracket > errors as lexical errors) are generally pretty misleading. > > > Otherwise we can still improve the situation quite a lot with just > column numbers, when they get merged someday. > > In this specific case, it would not have helped. It was perfectly clear > exactly where the parser ran into trouble. It just wasn't clear _why_ it > ran into trouble. > > Part of the problem is precisely the fact that the parser is written in > Yecc. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Fri May 17 14:24:32 2013 From: andrew@REDACTED (Andrew Thompson) Date: Fri, 17 May 2013 08:24:32 -0400 Subject: [erlang-questions] lager dropping invisible messages In-Reply-To: <5195ECDE.1000301@llaisdy.com> References: <5194ED59.4030202@llaisdy.com> <20130516214629.GI3966@hijacked.us> <5195ECDE.1000301@llaisdy.com> Message-ID: <20130517122432.GJ3966@hijacked.us> On Fri, May 17, 2013 at 09:39:58AM +0100, Ivan Uemlianin wrote: > Dear Andrew > > Thanks, I am familiar with those two web pages. > > The mystery is not "what is it supposed to do?". The mystery is > "where are the 50 messages it thinks it's written within the last > second?". Often, as far as I can tell it hasn't written any. Sorry, was a little jetlagged when I replied. Perhaps there is a bug? Can you provide a reproducible test case in a github issue so I can look into it next week when I'm back from travelling? Andrew From ivan@REDACTED Fri May 17 14:33:47 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 17 May 2013 13:33:47 +0100 Subject: [erlang-questions] lager dropping invisible messages In-Reply-To: <20130517122432.GJ3966@hijacked.us> References: <5194ED59.4030202@llaisdy.com> <20130516214629.GI3966@hijacked.us> <5195ECDE.1000301@llaisdy.com> <20130517122432.GJ3966@hijacked.us> Message-ID: <519623AB.2000601@llaisdy.com> Dear Andrew It's very occasional. I'll try and observe when it happens and I'll certainly provide a test case if I can. Best wishes Ivan On 17/05/2013 13:24, Andrew Thompson wrote: > On Fri, May 17, 2013 at 09:39:58AM +0100, Ivan Uemlianin wrote: >> Dear Andrew >> >> Thanks, I am familiar with those two web pages. >> >> The mystery is not "what is it supposed to do?". The mystery is >> "where are the 50 messages it thinks it's written within the last >> second?". Often, as far as I can tell it hasn't written any. > > Sorry, was a little jetlagged when I replied. > > Perhaps there is a bug? Can you provide a reproducible test case in a > github issue so I can look into it next week when I'm back from > travelling? > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From erlang@REDACTED Fri May 17 14:46:40 2013 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 17 May 2013 14:46:40 +0200 Subject: [erlang-questions] Message delivery in case of process restart In-Reply-To: <88ABB713-3492-4EBB-8139-DC75DF50D96F@axismilton.ltd.uk> References: <5195cba2.4725310a.168d.ffffa3ef@mx.google.com> <20130517063146.GA9295@aluminium.local> <88ABB713-3492-4EBB-8139-DC75DF50D96F@axismilton.ltd.uk> Message-ID: On Fri, May 17, 2013 at 11:09 AM, Jonathan Schneider wrote: > > "Sending a message to a pid never fails, even if the pid > > identifies a non-existing process." > > > > The documentation could be clearer. It means never causes an error for the > sender. > Yes - *sending* messages always works if you have a valid Pid. This does not mean that *receiving* the message succeeds. Suppose the Pid refers to a process on a remote machine. When you send the message the remote process might exist. But the remote machine might die while the message is in transit between the machines. Failure of a remote process can be detected by linking to the process. If the process you have linked to dies you'll be send an exit signal. Receiving a message is a somewhat vague idea. When you send a message to a process it gets put in the mailbox of the receiving processes (assuming the receiving process has not died). The word "receiving" might mean "put in the receives mailbox" or "evaluates one of the receive clauses in a receive statement." If you want to really know that a message was received and acted upon, you need to send a confirmation message back the the sender, when you get this message you'll know that the message was processed by the receiver. If you also link to the process you'll get an error signal back if the process fails. So basically you link to the remote process, send it a message and you'll either get a reply or an exit signal back. Cheers /Joe > > Jon > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Fri May 17 15:40:13 2013 From: dmercer@REDACTED (David Mercer) Date: Fri, 17 May 2013 08:40:13 -0500 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> Message-ID: <004701ce5304$1017c190$304744b0$@gmail.com> Richard A. O'Keefe wrote: > Part of the problem is precisely the fact that the parser is written in > Yecc. Back in the day, I'd have known the answer to this, but I figure some of you are more knowledgeable than I ever was: What parsing technology besides hand-rolled would produce better error messages? ANTLR? Parsec? PEG? Cheers, DBM From tino.breddin@REDACTED Fri May 17 18:50:42 2013 From: tino.breddin@REDACTED (Tino Breddin) Date: Fri, 17 May 2013 18:50:42 +0200 Subject: [erlang-questions] lager dropping invisible messages In-Reply-To: <519623AB.2000601@llaisdy.com> References: <5194ED59.4030202@llaisdy.com> <20130516214629.GI3966@hijacked.us> <5195ECDE.1000301@llaisdy.com> <20130517122432.GJ3966@hijacked.us> <519623AB.2000601@llaisdy.com> Message-ID: Hi, I was seeing similar messages with lager 2.x under very low load as well. Will try to capture such a case as well. Cheers Tino On Fri, May 17, 2013 at 2:33 PM, Ivan Uemlianin wrote: > Dear Andrew > > It's very occasional. I'll try and observe when it happens and I'll > certainly provide a test case if I can. > > Best wishes > > Ivan > > > > On 17/05/2013 13:24, Andrew Thompson wrote: > >> On Fri, May 17, 2013 at 09:39:58AM +0100, Ivan Uemlianin wrote: >> >>> Dear Andrew >>> >>> Thanks, I am familiar with those two web pages. >>> >>> The mystery is not "what is it supposed to do?". The mystery is >>> "where are the 50 messages it thinks it's written within the last >>> second?". Often, as far as I can tell it hasn't written any. >>> >> >> Sorry, was a little jetlagged when I replied. >> >> Perhaps there is a bug? Can you provide a reproducible test case in a >> github issue so I can look into it next week when I'm back from >> travelling? >> >> Andrew >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > -- > ==============================**============================== > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/**ivanuemlianin > > festina lente > ==============================**============================== > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 17 20:05:28 2013 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 17 May 2013 20:05:28 +0200 Subject: [erlang-questions] Mnesia wildcard query In-Reply-To: <001a01ce507f$43261140$c97233c0$@gmail.com> References: <001a01ce507f$43261140$c97233c0$@gmail.com> Message-ID: Many years ago I wrote rdbms_ms.erl, https://github.com/gebi/jungerl/blob/master/lib/rdbms/src/rdbms_ms.erl which takes a regexp.erl-style expression and converts it to a match spec. It has its limitations, but works pretty well for simple expressions. BR, Ulf On 14 May 2013, at 10:44, Janos Hary wrote: > Hi, > > Is there a way to specify a ?begins with? style query in Mnesia? My keys are strings, and I?d like to get all of them sharing a given prefix. > > Thanks, > Janos > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Sat May 18 03:48:24 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 17 May 2013 21:48:24 -0400 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <004701ce5304$1017c190$304744b0$@gmail.com> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> Message-ID: <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> On May 17, 2013, at 9:40 AM, David Mercer wrote: > Richard A. O'Keefe wrote: > >> Part of the problem is precisely the fact that the parser is written in >> Yecc. > > Back in the day, I'd have known the answer to this, but I figure some of you are more knowledgeable than I ever was: What parsing technology besides hand-rolled would produce better error messages? ANTLR? Parsec? PEG? Menhir http://gallium.inria.fr/~fpottier/menhir/ -- Siraaj Khandkar From ok@REDACTED Sun May 19 01:41:32 2013 From: ok@REDACTED (ok@REDACTED) Date: Sun, 19 May 2013 11:41:32 +1200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> Message-ID: > > On May 17, 2013, at 9:40 AM, David Mercer wrote: > >> Richard A. O'Keefe wrote: >> >>> Part of the problem is precisely the fact that the parser is written in >>> Yecc. >> >> Back in the day, I'd have known the answer to this, but I figure some of >> you are more knowledgeable than I ever was: What parsing technology >> besides hand-rolled would produce better error messages? ANTLR? >> Parsec? PEG? > > > Menhir > > http://gallium.inria.fr/~fpottier/menhir/ Menhir is definitely interesting, and if I had experienced anything other than extreme frustration with Godi, I would have liked to try it. But judging from section 8 of the Menhir manual, a Menhir-built parser would be no better than a Yecc-built one here. It might even be worse: the manual says that the current lookahead token is *replaced* by an ERROR token as part of recovery, and in the example we are discussing, the token in question is ")" and it is correct, and should be retained. What is more, the Menhir manual is quite explicit that error detection may be deferred. What the user needs in cases like this is the information that an earlier token (fun) caused the parser to be looking for a particular token (end) but it encountered a closer (right paren) before finding it. Something basically top-down should have an easier time of it. From robert.virding@REDACTED Sun May 19 20:55:26 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 19 May 2013 19:55:26 +0100 (BST) Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: Message-ID: <221244575.119835004.1368989726455.JavaMail.root@erlang-solutions.com> You can in some cases alleviate the problem by adding explicit grammar rules which generate errors. It is not a good solution as you clutter up the syntax rules. Robert ----- Original Message ----- > From: ok@REDACTED > To: "Siraaj Khandkar" > Cc: "Erlang-Questions Questions" > Sent: Sunday, 19 May, 2013 1:41:32 AM > Subject: Re: [erlang-questions] Better error messages would be helpful > > > > > On May 17, 2013, at 9:40 AM, David Mercer > > wrote: > > > >> Richard A. O'Keefe wrote: > >> > >>> Part of the problem is precisely the fact that the parser is > >>> written in > >>> Yecc. > >> > >> Back in the day, I'd have known the answer to this, but I figure > >> some of > >> you are more knowledgeable than I ever was: What parsing > >> technology > >> besides hand-rolled would produce better error messages? ANTLR? > >> Parsec? PEG? > > > > > > Menhir > > > > http://gallium.inria.fr/~fpottier/menhir/ > > Menhir is definitely interesting, and if I had experienced anything > other than extreme frustration with Godi, I would have liked to > try it. But judging from section 8 of the Menhir manual, a > Menhir-built parser would be no better than a Yecc-built one here. > It might even be worse: the manual says that the current lookahead > token is *replaced* by an ERROR token as part of recovery, and in > the example we are discussing, the token in question is ")" and it > is correct, and should be retained. What is more, the Menhir manual > is quite explicit that error detection may be deferred. > > What the user needs in cases like this is the information that > an earlier token (fun) caused the parser to be looking for a > particular token (end) but it encountered a closer (right paren) > before finding it. Something basically top-down should have an > easier time of it. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From sean@REDACTED Mon May 20 08:00:02 2013 From: sean@REDACTED (Functional Jobs) Date: Mon, 20 May 2013 02:00:02 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <5199bbf11e8bb@functionaljobs.com> Here are some functional programming job opportunities that were posted recently: Test Engineer at Klarna http://functionaljobs.com/jobs/146-test-engineer-at-klarna Senior Developer - Erlang at Klarna http://functionaljobs.com/jobs/145-senior-developer-erlang-at-klarna Cheers, Sean Murphy FunctionalJobs.com From raimo+erlang-questions@REDACTED Mon May 20 09:25:18 2013 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 20 May 2013 09:25:18 +0200 Subject: [erlang-questions] Basic atom question In-Reply-To: References: Message-ID: <20130520072518.GA23035@erix.ericsson.se> On Thu, May 16, 2013 at 10:41:43PM +0200, Martin Hedberg wrote: > > > Hello there > > Newbie wan Kenobi here again with a small question. :-) When the system restarts atoms are cleared but > I wonder: Is it enough to restart the process that the atoms were declared in, to clear the memory from it? As others have answered; atoms are never garbage collected. After system start no atom is ever deleted on that VM. There have been discussions about atom garbage collect but it is a tricky problem since you want to keep the properties of atoms especially size and speed.. Therefore the current recommendation is to not use atoms in excess: http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id61516 The number of atoms allowed is supposed to be enough for any number that will be introduced by loaded code and by communicating with other node's loaded code, but the number can be adjusted if your system requires it. It is when you start creating atoms dynamically you have a disaster waiting to happen... > > Best regards > > Martin > > > > > From: skribent_har@REDACTED > To: erlang-questions@REDACTED > Subject: A small question about macros > Date: Thu, 16 May 2013 06:04:00 +0200 > > > > > > > Hi there > > Have begun to take my first "baby steps" with Erlang (holding on to books and tutorials). > I read about how to make macros in the language. However I wonder how I do if I want > to have blank spaces in my commands? So I just can write, for example: "repeat 5 times". > > Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a little bit. :-) > > I am very grateful for any help. > > Best regards > > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From nieanan3602@REDACTED Mon May 20 16:30:09 2013 From: nieanan3602@REDACTED (=?GBK?B?xPSwsg==?=) Date: Mon, 20 May 2013 22:30:09 +0800 (CST) Subject: [erlang-questions] erlang socket send and rcv, high CPU usage Message-ID: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> hi all, I have a question about the high cpu usage when sending and receiving data via erlang-socket. My program uses a C/S mode. First, the server start a listening socket, and then the client connect to the server by using gen_tcp:connect. When establishing a connection, the client will keep sending "info" package to the server while waiting for a response from the client ; the server will respond an "info_ack" to the client when receiving a "info" package. When there's one client, at most 20000 packages per second are sent betweent C/S, while cpu usage reaches about 90%. My package is very little, at most 10Bytes, so my netcard is idle. When doing the same thing by C language, the cpu usage is just 6% while my netcard turns busy with about 500000 packages per second. I have no idea why erlang gets so cpu-consuming. My program goes as follows: Server config items about socket setting: {tcp_listen_opts, [binary,{packet,0},{reuseaddr,true},{exit_on_close,true},{active,false}]} 30 start_connection(Parent, Sock, SockTransform) -> 31 process_flag(trap_exit, true), 32 Name = 33 case catch ?NET:connection_string(Sock, inbound) of 34 {ok, Str} -> Str; 35 {error, enotcon} -> ?NET:fast_close(Sock),exit(normal); 36 {error, _Reason} -> ?NET:fast_clost(Sock),exit(normal) 37 end, 38 Msg = io_lib:format("accept connection ~p(~p)", [Name, self()]), 39 ?DEBUG(Msg), log:log(notice, ?MODULE, Msg), 40 ClientSock = socket_op(Sock, SockTransform), 41 ClientSock = Sock, 42 {PHost, PPort, Host, Port} = 43 socket_op(Sock, fun(S)->?NET:socket_ends(S,inbound) end), 44 State = #state{ 45 parent = Parent, 46 sock = ClientSock, 47 name = list_to_binary(Name), 48 callback = undefined, 49 host = Host, 50 port = Port, 51 peer_host = PHost, 52 peer_port = PPort 53 }, 54 try 55 %ok = inet_op(), 56 inet:setopts(ClientSock, [{active, true}]), 57 Ret = rcv_loop(State), 58 WMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " 59 "reason: ~p", [Name,self(),Ret])), 60 ?DEBUG(WMsg), log:log(warning, ?MODULE, WMsg) 61 catch 62 E:R -> 63 EMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " 64 "reason: ~p:~p", [Name, self(), E, R])), 65 ?DEBUG(EMsg), log:log(error, ?MODULE, EMsg) 66 after 66 after 67 ?NET:fast_close(ClientSock), 68 Msg2=lists:flatten(io_lib:format("connection ~p(~p) closed", 69 [Name, self()])), 70 ?DEBUG(Msg2), log:log(notice, ?MODULE, Msg2) 71 end, 72 normal. 73 74 rcv_loop(#state{sock=Sock}=State) -> 75 receive 76 {tcp, Sock, Binary} -> 77 NState = report_run(State, Binary),% calls port_command to send ack pkg 78 %inet:setopts(Sock, [{active, once}]), 79 rcv_loop(NState); 80 {tcp_closed, Sock} -> 'peer closed'; 81 _Other -> rcv_loop(State) 82 after ?RCV_TIMEOUT -> 'rcv timeout' 83 end. Client 10 -define(TCP_OPTIONS, [binary, {packet,0}, {active, false}, 11 {exit_on_close,true}, {reuseaddr,true}]). 101 setup_short_connection(Host, Port, #state{id=Id}=State) -> 102 case gen_tcp:connect(Host, Port, ?TCP_OPTIONS) of 103 {ok, Socket} -> 104 ?DEBUG("client(~p) connected to ~p:~p", [Id,Host,Port]), 105 inet:setopts(Socket, [{active, once}]), 106 send_request_and_wait_for_response(State#state{socket=Socket}); 107 {error, Reason} -> 108 EMsg=lists:flatten(io_lib:format("client(~p) connect failed," 109 "reason: ~p", [Id, Reason])), 110 ?DEBUG(EMsg), 111 {error, 'connect failed'} 112 end. 118 send_request_and_wait_for_response(#state{id=Id,requ_cnt=RequCnt}=State) -> 119 try 120 Ref = RequCnt, 121 NState = send_request(State, Ref), 122 wait_for_response(NState, Ref) 123 catch 124 E:R -> 125 EMsg = io:format("client(~p) failed in send_request_and_wait_for_response" 126 ", reason: ~p:~p", [Id, E, R]), 127 ?DEBUG(EMsg), 128 {error, 'error during send or wait'} 129 end. 130 131 send_request(#state{socket=Sock,requ_cnt=RequCnt}=State, Ref) -> 132 Data = {{"business", "aos"}, {"ref", Ref}}, 133 Pkg = {{"type", "key"}, {"data", Data}}, 134 JPkg = term_to_binary(Pkg), 135 true = erlang:port_command(Sock, JPkg, [nosuspend]), 136 State#state{requ_cnt=RequCnt-1}. 137 138 wait_for_response(#state{socket=Sock}=State, Ref) -> 139 receive 140 {tcp, Sock, Binary} -> 141 ok = inet:setopts(Sock, [{active,once}]), 142 case report_run(State, Binary, Ref) of 143 {response, NState} -> send_request_and_wait_for_response(NState); 144 {other, _} -> wait_for_response(State, Ref) 145 end; 146 _Other -> wait_for_response(State, Ref) 147 after 30000 -> {error, 'rcv response timeout'} 148 end. I'm sorry for my poor English. I will be very grateful if someone help me to solve this problem. yours niean -------------- next part -------------- An HTML attachment was scrubbed... URL: From arun11299@REDACTED Mon May 20 17:28:22 2013 From: arun11299@REDACTED (Arun Muralidharan) Date: Mon, 20 May 2013 20:58:22 +0530 Subject: [erlang-questions] erlang socket send and rcv, high CPU usage In-Reply-To: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> References: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> Message-ID: Hello Niean, Mentioned below are few of my suggestions: 1) Disable Nagle in server. 2) Try to use {active, once} in server. {active, true} setting may overflow your TCP buffer. In between, how much have you set the TCP read-write buffer (both in OS and application)? 3) Not sure what you are trying to achieve with port_command. Its not clear to me.. Other design related suggestions would be to: 1) Use async communication model using prim_inets as the listener. 2) Better to use an FSM for managing socket states and flow control. Thanks, Arun -Arun On Mon, May 20, 2013 at 8:00 PM, ?? wrote: > hi all, > I have a question about the high cpu usage when sending and receiving data > via erlang-socket. > My program uses a C/S mode. First, the server start a listening socket, > and then the client connect to the server by using gen_tcp:connect. When > establishing a connection, the client will keep sending "info" package to > the server while waiting for a response from the client ; the server will > respond an "info_ack" to the client when receiving a "info" package. When > there's one client, at most 20000 packages per second are sent betweent > C/S, while cpu usage reaches about 90%. My package is very little, at most > 10Bytes, so my netcard is idle. When doing the same thing by C language, > the cpu usage is just 6% while my netcard turns busy with about 500000 > packa ges per second. > I have no idea why erlang gets so cpu-consuming. My program goes as > follows: > > Server > config items about socket setting: > {tcp_listen_opts, [binary,{packet,0},{reuseaddr,** > true},{exit_on_close,true},{**active,false}****]} > > 30 start_connection(Parent, Sock, SockTransform) -> > 31 process_flag(trap_exit, true), > 32 Name = > 33 case catch ?NET:connection_string(Sock, inbound) of > 34 {ok, Str} -> Str; > 35 {error, enotcon} -> ?NET:fast_close(Sock),exit(**normal);** > 36 {error, _Reason} -> ?NET:fast_clost(Sock),exit(**normal)** > 37 end, > 38 Msg = io_lib:format("accept connection ~p(~p)", [Name, self()]), > 39 ?DEBUG(Msg), log:log(notice, ?MODULE, Msg), > 40 ClientSock = socket_op(Sock, SockTransform), > 41 ClientSock = Sock, > 42 {PHost, PPort, Host, Port} = > 43 socket_op(Sock, fun(S)->?NET:socket_ends(S,**inbound) end),** > 44 State = #state{ > 45 parent = Parent, > 46 sock = ClientSock, > 47 name = list_to_binary(Name), > 48 callback = undefined, > 49 host = Host, > 50 port = Port, > 51 peer_host = PHost, > 52 peer_port = PPort > 53 }, > 54 try > 55 %ok = inet_op(), > 56 inet:setopts(ClientSock, [{active, true}]), > 57 Ret = rcv_loop(State), > 58 WMsg=lists:flatten(io_lib:**format("closing connection > ~p(~p), "** > 59 "reason: ~p", [Name,self(),Ret])), > 60 ?DEBUG(WMsg), log:log(warning, ?MODULE, WMsg) > &nbs p;61 catch > 62 E:R -> > 63 EMsg=lists:flatten(io_lib:**format("closing connection > ~p(~p), "** > 64 "reason: ~p:~p", [Name, self(), E, R])), > 65 ?DEBUG(EMsg), log:log(error, ?MODULE, EMsg) > 66 after > 66 after > 67 ?NET:fast_close(ClientSock), > 68 Msg2=lists:flatten(io_lib:**format("connection ~p(~p) closed", > ** > 69 [Name, self()])), > 70 ?DEBUG(Msg2), log:log(notice, ?MODULE, Msg2) > 71 end, > 72 normal. > 73 > 74 rcv_loop(#state{sock=Sock}=**Sta te) ->** > 75 receive > 76 {tcp, Sock, Binary} -> > 77 NState = report_run(State, Binary),% calls port_command to > send ack pkg > 78 %inet:setopts(Sock, [{active, once}]), > 79 rcv_loop(NState); > 80 {tcp_closed, Sock} -> 'peer closed'; > 81 _Other -> rcv_loop(State) > 82 after ?RCV_TIMEOUT -> 'rcv timeout' > 83 end. > > > > Client > 10 -define(TCP_OPTIONS, [binary, {packet,0}, {active, false}, > 11 &nb sp; {exit_on_close,true}, {reuseaddr,true}]). > 101 setup_short_connection(Host, Port, #state{id=Id}=State) -> > 102 case gen_tcp:connect(Host, Port, ?TCP_OPTIONS) of > 103 {ok, Socket} -> > 104 ?DEBUG("client(~p) connected to ~p:~p", [Id,Host,Port]), > 105 inet:setopts(Socket, [{active, once}]), > 106 send_request_and_wait_for_**response(State#state{socket=** > Socket});**** > 107 {error, Reason} -> > 108 EMsg=lists:flatten(io_lib:**format("client(~p) connect > failed,"** > 109 "reason: ~p", [Id, Reason])), > 110 ?DEBUG(EMsg), > 111 {error, 'connect failed'} > 112 end. > 118 send_request_and_wait_for_**response(#state{id=Id,requ_**cnt=RequCnt}=State) > ->**** > 119 try > 120 Ref = RequCnt, > 121 NState = send_request(State, Ref), > 122 wait_for_response(NState, Ref) > 123 catch > 124 E:R -> > 125 EMsg = io:format("client(~p) failed in > send_request_and_wait_for_**response"** > 126 ", reason: ~p:~p", [Id, E, R]), > 127 ?DEBUG(EMsg), > 128 {error, 'error during send or wait'} > 129 end. > 130 > 131 send_request(#state{socket=**Sock,requ_cnt=RequCnt}=State, Ref) ->** > 132 Data = {{"business", "aos"}, {"ref", Ref}}, > 133 Pkg = {{"type", "key"}, {"data", Data}}, > 134 JPkg = term_to_binary(Pkg), > 135 true = erlang:port_command(Sock, JPkg, [nosuspend]), > 136 State#state{requ_cnt=RequCnt-**1}.** > 137 > 138 wait_for_response(#state{**socket=Sock}=State, Ref) ->** > 139 receive > 140 {tcp, Sock, Binary} -> > 141 ok = inet:setopts(Sock, [{active,once}]), > 142 case report_run(State, Binary, Ref) of > 143 {response, NState} -> send_request_and_wait_for_** > response(NState);** > 144 {other, _} -> wait_for_response(State, Ref) > 145 end; > 146 _Other -> wait_for_response(State, Ref) > 147 after 30000 -> {error, 'rcv response timeout'} > 148 end. > > > I'm sorry for my poor English. I will be very grateful if someone help me > to solve this problem. > > yours > niean > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Mon May 20 17:36:11 2013 From: paulperegud@REDACTED (Paul Peregud) Date: Mon, 20 May 2013 17:36:11 +0200 Subject: [erlang-questions] erlang socket send and rcv, high CPU usage In-Reply-To: References: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> Message-ID: Please also remember that debug printing costs a lot more then sending a packet via socket. If you want to measure you need to make sure that you don't print a message per packet. And for really big values of throughput you need to use {active, false} with gen_tcp:recv. Search information on erlyvideo and etorrent. Both have reached very nice values of thoughput. On Mon, May 20, 2013 at 5:28 PM, Arun Muralidharan wrote: > Hello Niean, > Mentioned below are few of my suggestions: > 1) Disable Nagle in server. > 2) Try to use {active, once} in server. {active, true} setting may overflow > your TCP buffer. In between, how much have you set the TCP read-write buffer > (both in OS and application)? > 3) Not sure what you are trying to achieve with port_command. Its not clear > to me.. > > Other design related suggestions would be to: > 1) Use async communication model using prim_inets as the listener. > 2) Better to use an FSM for managing socket states and flow control. > > Thanks, > Arun > > -Arun > > > On Mon, May 20, 2013 at 8:00 PM, ?? wrote: >> >> hi all, >> I have a question about the high cpu usage when sending and receiving data >> via erlang-socket. >> My program uses a C/S mode. First, the server start a listening socket, >> and then the client connect to the server by using gen_tcp:connect. When >> establishing a connection, the client will keep sending "info" package to >> the server while waiting for a response from the client ; the server will >> respond an "info_ack" to the client when receiving a "info" package. When >> there's one client, at most 20000 packages per second are sent betweent C/S, >> while cpu usage reaches about 90%. My package is very little, at most >> 10Bytes, so my netcard is idle. When doing the same thing by C language, the >> cpu usage is just 6% while my netcard turns busy with about 500000 packa ges >> per second. >> I have no idea why erlang gets so cpu-consuming. My program goes as >> follows: >> >> Server >> config items about socket setting: >> {tcp_listen_opts, >> [binary,{packet,0},{reuseaddr,true},{exit_on_close,true},{active,false}]} >> >> 30 start_connection(Parent, Sock, SockTransform) -> >> 31 process_flag(trap_exit, true), >> 32 Name = >> 33 case catch ?NET:connection_string(Sock, inbound) of >> 34 {ok, Str} -> Str; >> 35 {error, enotcon} -> ?NET:fast_close(Sock),exit(normal); >> 36 {error, _Reason} -> ?NET:fast_clost(Sock),exit(normal) >> 37 end, >> 38 Msg = io_lib:format("accept connection ~p(~p)", [Name, self()]), >> 39 ?DEBUG(Msg), log:log(notice, ?MODULE, Msg), >> 40 ClientSock = socket_op(Sock, SockTransform), >> 41 ClientSock = Sock, >> 42 {PHost, PPort, Host, Port} = >> 43 socket_op(Sock, fun(S)->?NET:socket_ends(S,inbound) end), >> 44 State = #state{ >> 45 parent = Parent, >> 46 sock = ClientSock, >> 47 name = list_to_binary(Name), >> 48 callback = undefined, >> 49 host = Host, >> 50 port = Port, >> 51 peer_host = PHost, >> 52 peer_port = PPort >> 53 }, >> 54 try >> 55 %ok = inet_op(), >> 56 inet:setopts(ClientSock, [{active, true}]), >> 57 Ret = rcv_loop(State), >> 58 WMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " >> 59 "reason: ~p", [Name,self(),Ret])), >> 60 ?DEBUG(WMsg), log:log(warning, ?MODULE, WMsg) >> &nbs p;61 catch >> 62 E:R -> >> 63 EMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " >> 64 "reason: ~p:~p", [Name, self(), E, R])), >> 65 ?DEBUG(EMsg), log:log(error, ?MODULE, EMsg) >> 66 after >> 66 after >> 67 ?NET:fast_close(ClientSock), >> 68 Msg2=lists:flatten(io_lib:format("connection ~p(~p) closed", >> 69 [Name, self()])), >> 70 ?DEBUG(Msg2), log:log(notice, ?MODULE, Msg2) >> 71 end, >> 72 normal. >> 73 >> 74 rcv_loop(#state{sock=Sock}=Sta te) -> >> 75 receive >> 76 {tcp, Sock, Binary} -> >> 77 NState = report_run(State, Binary),% calls port_command to >> send ack pkg >> 78 %inet:setopts(Sock, [{active, once}]), >> 79 rcv_loop(NState); >> 80 {tcp_closed, Sock} -> 'peer closed'; >> 81 _Other -> rcv_loop(State) >> 82 after ?RCV_TIMEOUT -> 'rcv timeout' >> 83 end. >> >> >> >> Client >> 10 -define(TCP_OPTIONS, [binary, {packet,0}, {active, false}, >> 11 &nb sp; {exit_on_close,true}, {reuseaddr,true}]). >> 101 setup_short_connection(Host, Port, #state{id=Id}=State) -> >> 102 case gen_tcp:connect(Host, Port, ?TCP_OPTIONS) of >> 103 {ok, Socket} -> >> 104 ?DEBUG("client(~p) connected to ~p:~p", [Id,Host,Port]), >> 105 inet:setopts(Socket, [{active, once}]), >> 106 >> send_request_and_wait_for_response(State#state{socket=Socket}); >> 107 {error, Reason} -> >> 108 EMsg=lists:flatten(io_lib:format("client(~p) connect failed," >> 109 "reason: ~p", [Id, Reason])), >> 110 ?DEBUG(EMsg), >> 111 {error, 'connect failed'} >> 112 end. >> 118 >> send_request_and_wait_for_response(#state{id=Id,requ_cnt=RequCnt}=State) -> >> 119 try >> 120 Ref = RequCnt, >> 121 NState = send_request(State, Ref), >> 122 wait_for_response(NState, Ref) >> 123 catch >> 124 E:R -> >> 125 EMsg = io:format("client(~p) failed in >> send_request_and_wait_for_response" >> 126 ", reason: ~p:~p", [Id, E, R]), >> 127 ?DEBUG(EMsg), >> 128 {error, 'error during send or wait'} >> 129 end. >> 130 >> 131 send_request(#state{socket=Sock,requ_cnt=RequCnt}=State, Ref) -> >> 132 Data = {{"business", "aos"}, {"ref", Ref}}, >> 133 Pkg = {{"type", "key"}, {"data", Data}}, >> 134 JPkg = term_to_binary(Pkg), >> 135 true = erlang:port_command(Sock, JPkg, [nosuspend]), >> 136 State#state{requ_cnt=RequCnt-1}. >> 137 >> 138 wait_for_response(#state{socket=Sock}=State, Ref) -> >> 139 receive >> 140 {tcp, Sock, Binary} -> >> 141 ok = inet:setopts(Sock, [{active,once}]), >> 142 case report_run(State, Binary, Ref) of >> 143 {response, NState} -> >> send_request_and_wait_for_response(NState); >> 144 {other, _} -> wait_for_response(State, Ref) >> 145 end; >> 146 _Other -> wait_for_response(State, Ref) >> 147 after 30000 -> {error, 'rcv response timeout'} >> 148 end. >> >> >> I'm sorry for my poor English. I will be very grateful if someone help me >> to solve this problem. >> >> yours >> niean >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Paul Peregud +48602112091 From dmkolesnikov@REDACTED Mon May 20 17:39:20 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 20 May 2013 18:39:20 +0300 Subject: [erlang-questions] erlang socket send and rcv, high CPU usage In-Reply-To: References: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> Message-ID: <050E1238-4C76-4CB7-805E-3809C5B7F9E9@gmail.com> Hello, ... and use kernel poll. Put flag +K true to vm. Best Regards, Dmitry >-|-|-*> On 20.5.2013, at 18.28, Arun Muralidharan wrote: > Hello Niean, > Mentioned below are few of my suggestions: > 1) Disable Nagle in server. > 2) Try to use {active, once} in server. {active, true} setting may overflow your TCP buffer. In between, how much have you set the TCP read-write buffer (both in OS and application)? > 3) Not sure what you are trying to achieve with port_command. Its not clear to me.. > > Other design related suggestions would be to: > 1) Use async communication model using prim_inets as the listener. > 2) Better to use an FSM for managing socket states and flow control. > > Thanks, > Arun > > -Arun > > > On Mon, May 20, 2013 at 8:00 PM, ?? wrote: >> hi all, >> I have a question about the high cpu usage when sending and receiving data via erlang-socket. >> My program uses a C/S mode. First, the server start a listening socket, and then the client connect to the server by using gen_tcp:connect. When establishing a connection, the client will keep sending "info" package to the server while waiting for a response from the client ; the server will respond an "info_ack" to the client when receiving a "info" package. When there's one client, at most 20000 packages per second are sent betweent C/S, while cpu usage reaches about 90%. My package is very little, at most 10Bytes, so my netcard is idle. When doing the same thing by C language, the cpu usage is just 6% while my netcard turns busy with about 500000 packa ges per second. >> I have no idea why erlang gets so cpu-consuming. My program goes as follows: >> >> Server >> config items about socket setting: >> {tcp_listen_opts, [binary,{packet,0},{reuseaddr,true},{exit_on_close,true},{active,false}]} >> >> 30 start_connection(Parent, Sock, SockTransform) -> >> 31 process_flag(trap_exit, true), >> 32 Name = >> 33 case catch ?NET:connection_string(Sock, inbound) of >> 34 {ok, Str} -> Str; >> 35 {error, enotcon} -> ?NET:fast_close(Sock),exit(normal); >> 36 {error, _Reason} -> ?NET:fast_clost(Sock),exit(normal) >> 37 end, >> 38 Msg = io_lib:format("accept connection ~p(~p)", [Name, self()]), >> 39 ?DEBUG(Msg), log:log(notice, ?MODULE, Msg), >> 40 ClientSock = socket_op(Sock, SockTransform), >> 41 ClientSock = Sock, >> 42 {PHost, PPort, Host, Port} = >> 43 socket_op(Sock, fun(S)->?NET:socket_ends(S,inbound) end), >> 44 State = #state{ >> 45 parent = Parent, >> 46 sock = ClientSock, >> 47 name = list_to_binary(Name), >> 48 callback = undefined, >> 49 host = Host, >> 50 port = Port, >> 51 peer_host = PHost, >> 52 peer_port = PPort >> 53 }, >> 54 try >> 55 %ok = inet_op(), >> 56 inet:setopts(ClientSock, [{active, true}]), >> 57 Ret = rcv_loop(State), >> 58 WMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " >> 59 "reason: ~p", [Name,self(),Ret])), >> 60 ?DEBUG(WMsg), log:log(warning, ?MODULE, WMsg) >> &nbs p;61 catch >> 62 E:R -> >> 63 EMsg=lists:flatten(io_lib:format("closing connection ~p(~p), " >> 64 "reason: ~p:~p", [Name, self(), E, R])), >> 65 ?DEBUG(EMsg), log:log(error, ?MODULE, EMsg) >> 66 after >> 66 after >> 67 ?NET:fast_close(ClientSock), >> 68 Msg2=lists:flatten(io_lib:format("connection ~p(~p) closed", >> 69 [Name, self()])), >> 70 ?DEBUG(Msg2), log:log(notice, ?MODULE, Msg2) >> 71 end, >> 72 normal. >> 73 >> 74 rcv_loop(#state{sock=Sock}=Sta te) -> >> 75 receive >> 76 {tcp, Sock, Binary} -> >> 77 NState = report_run(State, Binary),% calls port_command to send ack pkg >> 78 %inet:setopts(Sock, [{active, once}]), >> 79 rcv_loop(NState); >> 80 {tcp_closed, Sock} -> 'peer closed'; >> 81 _Other -> rcv_loop(State) >> 82 after ?RCV_TIMEOUT -> 'rcv timeout' >> 83 end. >> >> >> >> Client >> 10 -define(TCP_OPTIONS, [binary, {packet,0}, {active, false}, >> 11 &nb sp; {exit_on_close,true}, {reuseaddr,true}]). >> 101 setup_short_connection(Host, Port, #state{id=Id}=State) -> >> 102 case gen_tcp:connect(Host, Port, ?TCP_OPTIONS) of >> 103 {ok, Socket} -> >> 104 ?DEBUG("client(~p) connected to ~p:~p", [Id,Host,Port]), >> 105 inet:setopts(Socket, [{active, once}]), >> 106 send_request_and_wait_for_response(State#state{socket=Socket}); >> 107 {error, Reason} -> >> 108 EMsg=lists:flatten(io_lib:format("client(~p) connect failed," >> 109 "reason: ~p", [Id, Reason])), >> 110 ?DEBUG(EMsg), >> 111 {error, 'connect failed'} >> 112 end. >> 118 send_request_and_wait_for_response(#state{id=Id,requ_cnt=RequCnt}=State) -> >> 119 try >> 120 Ref = RequCnt, >> 121 NState = send_request(State, Ref), >> 122 wait_for_response(NState, Ref) >> 123 catch >> 124 E:R -> >> 125 EMsg = io:format("client(~p) failed in send_request_and_wait_for_response" >> 126 ", reason: ~p:~p", [Id, E, R]), >> 127 ?DEBUG(EMsg), >> 128 {error, 'error during send or wait'} >> 129 end. >> 130 >> 131 send_request(#state{socket=Sock,requ_cnt=RequCnt}=State, Ref) -> >> 132 Data = {{"business", "aos"}, {"ref", Ref}}, >> 133 Pkg = {{"type", "key"}, {"data", Data}}, >> 134 JPkg = term_to_binary(Pkg), >> 135 true = erlang:port_command(Sock, JPkg, [nosuspend]), >> 136 State#state{requ_cnt=RequCnt-1}. >> 137 >> 138 wait_for_response(#state{socket=Sock}=State, Ref) -> >> 139 receive >> 140 {tcp, Sock, Binary} -> >> 141 ok = inet:setopts(Sock, [{active,once}]), >> 142 case report_run(State, Binary, Ref) of >> 143 {response, NState} -> send_request_and_wait_for_response(NState); >> 144 {other, _} -> wait_for_response(State, Ref) >> 145 end; >> 146 _Other -> wait_for_response(State, Ref) >> 147 after 30000 -> {error, 'rcv response timeout'} >> 148 end. >> >> >> I'm sorry for my poor English. I will be very grateful if someone help me to solve this problem. >> >> yours >> niean >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 20 19:15:20 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 20 May 2013 21:15:20 +0400 Subject: [erlang-questions] erlang socket send and rcv, high CPU usage In-Reply-To: <050E1238-4C76-4CB7-805E-3809C5B7F9E9@gmail.com> References: <7d6b0565.17dbf.13ec258f1f7.Coremail.nieanan3602@163.com> <050E1238-4C76-4CB7-805E-3809C5B7F9E9@gmail.com> Message-ID: erlyvideo can send about 5-6 gigabits per second via one network interface. If you want to reach such amounts, use blocking gen_tcp:recv with big buffer. Not gen_tcp:recv(Socket, 100), but gen_tcp:recv(Socket, 1000000) From esumbar@REDACTED Mon May 20 19:12:22 2013 From: esumbar@REDACTED (Edmund Sumbar) Date: Mon, 20 May 2013 11:12:22 -0600 Subject: [erlang-questions] pattern matching registered name Message-ID: Hello, I'm new to erlang and am perplexed by the use of registered names when they are passed as function parameters and subsequently used in pattern matching. For example (taken from Programming Erlang 2/ed by Joe Armstrong), -module(kvs). -export([start/0, rpc/1]). start() -> register(kvs, spawn(fun() -> loop() end)). rpc(Q) -> kvs ! {self(), Q}, receive {kvs, Reply} -> Reply end. loop() -> receive {From, {store, Key, Value}} -> put(Key, {ok, Value}), From ! {kvs, true}, loop(); {From, {lookup, Key}} -> From ! {kvs, get(Key)}, loop() end. This module works as expected. $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V5.10.1 (abort with ^G) 1> c(kvs). {ok,kvs} 2> kvs:start(). true 3> kvs:rpc({store, name, "Joe"}). true 4> kvs:rpc({lookup, name}). {ok,"Joe"} Then, I modify the code to remove the registered name. -module(kvsx). -export([start/0, rpc/2]). start() -> spawn(fun() -> loop() end). rpc(Pid, Q) -> Pid ! {self(), Q}, receive {Pid, Reply} -> Reply end. loop() -> receive {From, {store, Key, Value}} -> put(Key, {ok, Value}), From ! {self(), true}, loop(); {From, {lookup, Key}} -> From ! {self(), get(Key)}, loop() end. The modified module also works as expected. $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V5.10.1 (abort with ^G) 1> c(kvsx). {ok,kvsx} 2> Pid = kvsx:start(). <0.40.0> 3> kvsx:rpc(Pid, {store, name, "Joe"}). true 4> kvsx:rpc(Pid, {lookup, name}). {ok,"Joe"} However, when I register a name for Pid and pass it to the function, the process hangs. 5> register(abc, Pid). true 6> whereis(abc). <0.40.0> 7> kvsx:rpc(abc, {lookup, name}). ^G User switch command --> q As a newcomer to the language (but with decades of programming experience), I'm confused. Can someone explain registered names and when/where they can be used. Ed -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Mon May 20 19:47:30 2013 From: james@REDACTED (James Aimonetti) Date: Mon, 20 May 2013 10:47:30 -0700 Subject: [erlang-questions] pattern matching registered name In-Reply-To: References: Message-ID: <519A61B2.4020502@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/20/2013 10:12 AM, Edmund Sumbar wrote: > Hello, > > I'm new to erlang and am perplexed by the use of registered names > when they are passed as function parameters and subsequently used > in pattern matching. > > For example (taken from Programming Erlang 2/ed by Joe Armstrong), > > -module(kvs). -export([start/0, rpc/1]). > > start() -> register(kvs, spawn(fun() -> loop() end)). > > rpc(Q) -> kvs ! {self(), Q}, receive {kvs, Reply} -> Reply end. > > loop() -> receive {From, {store, Key, Value}} -> put(Key, {ok, > Value}), From ! {kvs, true}, loop(); {From, {lookup, Key}} -> From > ! {kvs, get(Key)}, loop() end. > > This module works as expected. > > $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] > [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.10.1 (abort with ^G) 1> c(kvs). {ok,kvs} 2> > kvs:start(). true 3> kvs:rpc({store, name, "Joe"}). true 4> > kvs:rpc({lookup, name}). {ok,"Joe"} > > Then, I modify the code to remove the registered name. > > -module(kvsx). -export([start/0, rpc/2]). > > start() -> spawn(fun() -> loop() end). > > rpc(Pid, Q) -> Pid ! {self(), Q}, receive {Pid, Reply} -> Reply > end. > > loop() -> receive {From, {store, Key, Value}} -> put(Key, {ok, > Value}), From ! {self(), true}, loop(); {From, {lookup, Key}} -> > From ! {self(), get(Key)}, loop() end. > > The modified module also works as expected. > > $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] > [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.10.1 (abort with ^G) 1> c(kvsx). {ok,kvsx} 2> Pid = > kvsx:start(). <0.40.0> 3> kvsx:rpc(Pid, {store, name, "Joe"}). > true 4> kvsx:rpc(Pid, {lookup, name}). {ok,"Joe"} > > However, when I register a name for Pid and pass it to the > function, the process hangs. > > 5> register(abc, Pid). true 6> whereis(abc). <0.40.0> 7> > kvsx:rpc(abc, {lookup, name}). ^G User switch command --> q > > As a newcomer to the language (but with decades of programming > experience), I'm confused. Can someone explain registered names and > when/where they can be used. > > Ed > > > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > rpc/2 expects the first arg into the function to be the first element in the response tuple. The loop/0 uses self(), which always returns a pid(), not the registered name. So you call rpc with an atom, and receive a tuple with a pid(). Those don't match. Your call to rpc/2 hangs because the selective receive in rpc/2 never matches (and you have no timeout clause). - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRmmGyAAoJENc77s1OYoGgjYkH/jT0bUtnghdmbNo8w0wQnqhL sRSVuh9wKRLTuPf4x/fyCBHBEC7g6Ii+i9Xdnyc7EnaCfI7vEAx8IRmNJiMnh/Rx 9P1P+48hQmkTjRqj1aBfPR5O0EqezGB/72oPxFK5lN8hLVg0+FOjvRjF2FVezQEt 9gv5kbu7ORla9NJK+9CzGj8pUlx0CQ5ba7pXIU0ib00hf6DA3g8rJmpbS1Fe7+6i 5eYJLvh0/m6F8YjGtrdYIEfCfey7assHMgOIZ+iC2QjcbnqWPZHCbuhsPUz0LbMu BQOOn3/8NAbstOOMzW7LHjCJ80yLJX9HMHn+bvORTaFVIDy1kUxhDJSUDDP7mSs= =iAES -----END PGP SIGNATURE----- From yogishb@REDACTED Mon May 20 19:48:51 2013 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 20 May 2013 10:48:51 -0700 Subject: [erlang-questions] pattern matching registered name In-Reply-To: References: Message-ID: When you call rpc(abc, blah), Pid = abc. In your rpc(...) code you are waiting for {abc, Resp} in receive loop. On Mon, May 20, 2013 at 10:12 AM, Edmund Sumbar wrote: > Hello, > > I'm new to erlang and am perplexed by the use of registered names when > they are passed as function parameters and subsequently used in pattern > matching. > > For example (taken from Programming Erlang 2/ed by Joe Armstrong), > > -module(kvs). > -export([start/0, rpc/1]). > > start() -> register(kvs, spawn(fun() -> loop() end)). > > rpc(Q) -> > kvs ! {self(), Q}, > receive > {kvs, Reply} -> > Reply > end. > > loop() -> > receive > {From, {store, Key, Value}} -> > put(Key, {ok, Value}), > From ! {kvs, true}, > loop(); > {From, {lookup, Key}} -> > From ! {kvs, get(Key)}, > loop() > end. > > This module works as expected. > > $ erl > Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.10.1 (abort with ^G) > 1> c(kvs). > {ok,kvs} > 2> kvs:start(). > true > 3> kvs:rpc({store, name, "Joe"}). > true > 4> kvs:rpc({lookup, name}). > {ok,"Joe"} > > Then, I modify the code to remove the registered name. > > -module(kvsx). > -export([start/0, rpc/2]). > > start() -> spawn(fun() -> loop() end). > > rpc(Pid, Q) -> > Pid ! {self(), Q}, > receive > {Pid, Reply} -> > Reply > end. > > loop() -> > receive > {From, {store, Key, Value}} -> > put(Key, {ok, Value}), > From ! {self(), true}, > loop(); > {From, {lookup, Key}} -> > From ! {self(), get(Key)}, > loop() > end. > > The modified module also works as expected. > > $ erl > Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.10.1 (abort with ^G) > 1> c(kvsx). > {ok,kvsx} > 2> Pid = kvsx:start(). > <0.40.0> > 3> kvsx:rpc(Pid, {store, name, "Joe"}). > true > 4> kvsx:rpc(Pid, {lookup, name}). > {ok,"Joe"} > > However, when I register a name for Pid and pass it to the function, the > process hangs. > > 5> register(abc, Pid). > true > 6> whereis(abc). > <0.40.0> > 7> kvsx:rpc(abc, {lookup, name}). > ^G > User switch command > --> q > > As a newcomer to the language (but with decades of programming > experience), I'm confused. Can someone explain registered names and > when/where they can be used. > > Ed > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From esumbar@REDACTED Mon May 20 20:47:05 2013 From: esumbar@REDACTED (Edmund Sumbar) Date: Mon, 20 May 2013 12:47:05 -0600 Subject: [erlang-questions] pattern matching registered name In-Reply-To: <519A61B2.4020502@2600hz.com> References: <519A61B2.4020502@2600hz.com> Message-ID: On Mon, May 20, 2013 at 11:47 AM, James Aimonetti wrote: > rpc/2 expects the first arg into the function to be the first element > in the response tuple. > The loop/0 uses self(), which always returns a pid(), not the > registered name. > So you call rpc with an atom, and receive a tuple with a pid(). Those > don't match. > Your call to rpc/2 hangs because the selective receive in rpc/2 never > matches (and you have no timeout clause). > What's confusing is that the atom apparently resolves into a pid() in the send expression of kvsx:rpc/2 (adding io:format("reg name = ~p~n", [whereis(abc)]) to the function returns the pid()), but not in the pattern tuple of the receive expression. Anyway, thanks for the replies James and Yogish. Ed -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 20 21:27:29 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 20 May 2013 23:27:29 +0400 Subject: [erlang-questions] supervisor with ets stored children: gen_tracker Message-ID: Hi, everyone. gen_tracker is a library that implements subset of supervisor functionality with storing children and their metadata in public named ets table. https://github.com/erlyvideo/gen_tracker MIT license. I've been using this code for a while in erlyvideo and I've extracted it as a library. What problems are solved: 1) supervisor doesn't have supervisor:find_child_by_name API, only which_children that returns full list of children 2) supervisor library doesn't have supervisor:find_or_start API that atomically either return existing child, either starts new. The only way is to handle two different {already_started,Pid} errors 3) gproc library is ortogonal to supervisor tree, so it is impossible to introspect all gproc children via appmon and do other cool supervisor things like autostopping all children 4) there is no guaranteed way to call some function with supervisor or gproc after process is dead, passing some metadata to this function. Module:terminate can be not called if reason is kill. This function is required for accounting need to calculate how many bytes have this user session transferred or something else. How gen_tracker solves this problem? Each gen_tracker instance is a singleton that registers under some atom in system, it creates ets table with the same name and second table NAME_attrs for children metadata. gen_tracker instance behaves like a supervisor. Usage You can start it as a regular child in your supervision tree: init([]) -> Supervisors = [{streams, {gen_tracker, start_link, [streams]}, permanent, infinity, supervisor, []}], {ok, { {one_for_one, 5, 10}, Supervisors} }. Mention that gen_tracker process is a supervisor. Now let's start a child of this gen_tracke instance. Define a supervisor child: Stream = {<<"tv1">>, {stream, start_link, [<<"tv1">>, Options]}, temporary, infinity, supervisor, []}, and start it or get existing: {ok, Pid} = gen_tracker:find_or_open(streams, Stream). Mention that find_or_open first checks in ets table and if it doesn't exists, it goes to process. If you don't want to make a gen_server:call, than you can just: {ok, Pid} = gen_tracker:find(streams, <<"tv1">>). to find _existing_ process (or get undefined if there is no child with such ID) Now, metadata. Erlyvideo streams stores hundreds of megabytes of data in memory. It is a bad idea to store this data in process dictionary, ets is a better place: gen_tracker:setattr(streams, Name, [{hds,true},{bytes_in,0},{bytes_out,0}]), gen_tracker:increment(streams, Name, bytes_in, 1000), {ok, BytesIn} = gen_tracker:getattr(streams, Name, bytes_in). Wow, several _nonblocking_ calls made to modify child metadata. No single message is sent here. Now it is time to kill annoying child: supervisor:which_children(streams), supervisor:delete_child(streams, <<"tv1">>). gen_tracker implements subset of supervisor API. So, I hope that it would be useful for someone else. From garry@REDACTED Mon May 20 21:29:41 2013 From: garry@REDACTED (Garry Watkins) Date: Mon, 20 May 2013 15:29:41 -0400 Subject: [erlang-questions] Embed Erlang VM into an OS/X XPC Service Message-ID: I am interested in developing a OS/X XPC Service that has a erlang vm running in it. The main OS/X program would send XPC messages to the ERL XPC Service. The service would then process that message and do the appropriate thing. Is there a way that I can build erlang as a shared library that I can call from C. I need it to run within the XPC container so it will stay sandboxed correctly. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Mon May 20 21:34:52 2013 From: james@REDACTED (James Aimonetti) Date: Mon, 20 May 2013 12:34:52 -0700 Subject: [erlang-questions] pattern matching registered name In-Reply-To: References: <519A61B2.4020502@2600hz.com> Message-ID: <519A7ADC.8050401@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/20/2013 11:47 AM, Edmund Sumbar wrote: > On Mon, May 20, 2013 at 11:47 AM, James Aimonetti > wrote: > >> rpc/2 expects the first arg into the function to be the first >> element in the response tuple. The loop/0 uses self(), which >> always returns a pid(), not the registered name. So you call rpc >> with an atom, and receive a tuple with a pid(). Those don't >> match. Your call to rpc/2 hangs because the selective receive in >> rpc/2 never matches (and you have no timeout clause). >> > > What's confusing is that the atom apparently resolves into a pid() > in the send expression of kvsx:rpc/2 (adding io:format("reg name = > ~p~n", [whereis(abc)]) to the function returns the pid()), but not > in the pattern tuple of the receive expression. > > Anyway, thanks for the replies James and Yogish. > > Ed > > > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > You could change rpc/2: rpc(Name, Q) when is_atom(Name) -> rpc(whereis(Name), Q); rpc(Pid, Q) when is_pid(Pid) -> Pid ! {self(), Q}, receive {Pid, Reply} -> Reply end. This will crash the caller if Name isn't registered, FYI. - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRmnrcAAoJENc77s1OYoGgF1EH/Ajv0qTcrUEK+NL4kNJ7w8rQ G6hdnnpm6231XA6h9MbmW0jph5nYdAo5PBeF0QBK3XyuVOUw1AvZv8PcqTTHmsKc K0RbZFip7mfNWXFa3wPufg1K7cZ8F5yG4k5BB+PpHSRfnh+UVs7z+qVNeXKXBWp2 OClbohj6BqzuRR8udb+k73Bz1uwsIqDDWTSEtBpJ4YQJwSmTBjlHKIJ6wnEi5yl3 VXFdM+/KKBN0coO3TOWOvTt4soN2AqLKSKL7XfNs91FyJvAL/nJ1YderAY2Q9MY8 MZCLxF/Nsjjm4OxlabTEyMP/wnKwmNNvKmjx7Adx0ve+SoZbjayD32zl5goIQU8= =IwqX -----END PGP SIGNATURE----- From yogishb@REDACTED Mon May 20 21:55:03 2013 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 20 May 2013 12:55:03 -0700 Subject: [erlang-questions] Embed Erlang VM into an OS/X XPC Service In-Reply-To: References: Message-ID: Hello Garry, You can develop a C-node that can communicate with Erlang node OR you can listen on a TCP port in Erlang node, client can send message to Erlang node over TCP port. AFAIK, you cannot embed Erlang VM in your C code. -- baliga On Mon, May 20, 2013 at 12:29 PM, Garry Watkins wrote: > I am interested in developing a OS/X XPC Service that has a erlang vm > running in it. > > The main OS/X program would send XPC messages to the ERL XPC Service. The > service would then process that message and do the appropriate thing. > > Is there a way that I can build erlang as a shared library that I can call > from C. I need it to run within the XPC container so it will stay > sandboxed correctly. > > Thanks > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Mon May 20 21:55:35 2013 From: bob@REDACTED (Bob Ippolito) Date: Mon, 20 May 2013 12:55:35 -0700 Subject: [erlang-questions] pattern matching registered name In-Reply-To: <519A7ADC.8050401@2600hz.com> References: <519A61B2.4020502@2600hz.com> <519A7ADC.8050401@2600hz.com> Message-ID: This strategy is also used by OTP in gen.erl (the plumbing for gen_server, gen_fsm, gen_event, ?). https://github.com/erlang/otp/blob/maint/lib/stdlib/src/gen.erl#L152 On Mon, May 20, 2013 at 12:34 PM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 05/20/2013 11:47 AM, Edmund Sumbar wrote: > > On Mon, May 20, 2013 at 11:47 AM, James Aimonetti > > wrote: > > > >> rpc/2 expects the first arg into the function to be the first > >> element in the response tuple. The loop/0 uses self(), which > >> always returns a pid(), not the registered name. So you call rpc > >> with an atom, and receive a tuple with a pid(). Those don't > >> match. Your call to rpc/2 hangs because the selective receive in > >> rpc/2 never matches (and you have no timeout clause). > >> > > > > What's confusing is that the atom apparently resolves into a pid() > > in the send expression of kvsx:rpc/2 (adding io:format("reg name = > > ~p~n", [whereis(abc)]) to the function returns the pid()), but not > > in the pattern tuple of the receive expression. > > > > Anyway, thanks for the replies James and Yogish. > > > > Ed > > > > > > > > _______________________________________________ erlang-questions > > mailing list erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > You could change rpc/2: > > rpc(Name, Q) when is_atom(Name) -> > rpc(whereis(Name), Q); > rpc(Pid, Q) when is_pid(Pid) -> > Pid ! {self(), Q}, > receive > {Pid, Reply} -> > Reply > end. > > This will crash the caller if Name isn't registered, FYI. > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQEcBAEBAgAGBQJRmnrcAAoJENc77s1OYoGgF1EH/Ajv0qTcrUEK+NL4kNJ7w8rQ > G6hdnnpm6231XA6h9MbmW0jph5nYdAo5PBeF0QBK3XyuVOUw1AvZv8PcqTTHmsKc > K0RbZFip7mfNWXFa3wPufg1K7cZ8F5yG4k5BB+PpHSRfnh+UVs7z+qVNeXKXBWp2 > OClbohj6BqzuRR8udb+k73Bz1uwsIqDDWTSEtBpJ4YQJwSmTBjlHKIJ6wnEi5yl3 > VXFdM+/KKBN0coO3TOWOvTt4soN2AqLKSKL7XfNs91FyJvAL/nJ1YderAY2Q9MY8 > MZCLxF/Nsjjm4OxlabTEyMP/wnKwmNNvKmjx7Adx0ve+SoZbjayD32zl5goIQU8= > =IwqX > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 21 03:31:17 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 21 May 2013 13:31:17 +1200 Subject: [erlang-questions] Basic atom question In-Reply-To: <20130520072518.GA23035@erix.ericsson.se> References: <20130520072518.GA23035@erix.ericsson.se> Message-ID: <0C21A2B8-D6F2-4D94-A63D-032ED4C2894A@cs.otago.ac.nz> It's really time something was done about Erlang atoms. I note that SWI Prolog has threads, uses atoms a _lot_ (more than I would recommend) and has atom garbage collection. From ok@REDACTED Tue May 21 03:47:33 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 21 May 2013 13:47:33 +1200 Subject: [erlang-questions] pattern matching registered name In-Reply-To: References: Message-ID: <2912A149-12A6-4919-A19C-550CC9128B89@cs.otago.ac.nz> In a Who ! What send, Who may be a pid or a registered name. In a pattern match, registered names do not match pids; they are just literal atoms. From wckbluesky@REDACTED Tue May 21 04:14:24 2013 From: wckbluesky@REDACTED (=?utf-8?B?6Z+m5bSH54Sc?=) Date: Tue, 21 May 2013 10:14:24 +0800 (CST) Subject: [erlang-questions] nif driver c code must be reentrant? Message-ID: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> hi~ ? I try to design some nif codes recently and I reading a book?" Erlang and OTP in Action" now. In this book, chapter 12, "Integrating with foreign code using ports and NIFs", page 313, section "Making a linked driver".? ? I found some words: " Because?all?these?instances?execute within?the?memory?of?the?Erlang?VM,?possibly?running?as?different?threads,?the?driver code?must?be?designed?to?be?reentrant?executable?by?multiple?simultaneous?callers? and?must?not?depend?on?global?variables?or?locks. " ? and then, this book give some methods to avoid this problem, something like not to use global variables, and use driver_alloc, driver_free to allocate memory dynamically. ? of course, these words were placed into the section "Making a linked driver" ? My questions is: ? 1) Is nif driver c code has the same problem? That is, nif driver c code must be designed to reentrant? 2) I know that, nif calling will block the VM until the driver c code returns, Is that other problems we must notice when using nifs? ? And if nif driver c code must be designed to reentrant, then I will have anther question. I read this words from erlang.org ? " NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns " ? That is, calling scheduler is driven by the same thread, so the calling will be blocked. Then why should we designed our nif code to be reentrant? ? ? The e-mail is little long, thanks for reading, and I do hope to receive any helpful messages. Thanks! ? ? ? ? Best Regards ckwei From skribent_har@REDACTED Tue May 21 04:35:17 2013 From: skribent_har@REDACTED (Martin Hedberg) Date: Tue, 21 May 2013 04:35:17 +0200 Subject: [erlang-questions] Luerl and dynamically created lists In-Reply-To: References: , Message-ID: Hi again I know that it is a big "no no" to create variables dynamically in Erlang. However I am planning to use Mr Virdings "Luerl" modules so Lua-engienes (on top of Erlang processes) can handle some of the more simple programming tasks. While the Erlang processes handle the communication between them. When the Lua-processes produce variables dynamically I thought it could just pass it via the Erlang layer . However as I have understood it Erlang is just sending copies. Which makes me a bit worried about the memory. Is there some way to erase The Lua-content from an Erlang process after it have been sent? Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue May 21 05:33:42 2013 From: vances@REDACTED (Vance Shipley) Date: Tue, 21 May 2013 09:03:42 +0530 Subject: [erlang-questions] Basic atom question In-Reply-To: References: Message-ID: No, there is a one atom table for the entire emulator. If you create an atom it is added to the table. In the current implementation the atom table is not garbage collected. Normally atoms are created by loading a module which references them and normally there is a fixed set of modules in use. Using erlang:list_to_atom/1 is where the trouble starts. You may use erlang:list_to_existing_atom/1 to keep from adding new atoms if you must. Dynamically loading a large number of modules over time would also be a problem. On May 17, 2013 2:42 AM, "Martin Hedberg" wrote: > > > Hello there > > Newbie wan Kenobi here again with a small question. :-) When the system > restarts atoms are cleared but > I wonder: Is it enough to restart the process that the atoms were declared > in, to clear the memory from it? > > Best regards > > Martin > > > > > ------------------------------ > From: skribent_har@REDACTED > To: erlang-questions@REDACTED > Subject: A small question about macros > Date: Thu, 16 May 2013 06:04:00 +0200 > > > > Hi there > > Have begun to take my first "baby steps" with Erlang (holding on to books > and tutorials). > I read about how to make macros in the language. However I wonder how I do > if I want > to have blank spaces in my commands? So I just can write, for example: > "repeat 5 times". > > Hope you don't see me as a heretic if I want to HyperTalk-ificat Erlang a > little bit. :-) > > I am very grateful for any help. > > Best regards > > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajmanoj.bottle@REDACTED Tue May 21 03:09:23 2013 From: rajmanoj.bottle@REDACTED (Manoj Raj) Date: Tue, 21 May 2013 06:39:23 +0530 Subject: [erlang-questions] Mnesia - Check for a value in either of the keys in the table Message-ID: Hi, I have a mnesia table "users" created with the following record -record(users,{username,nickname,age}) I have a name now to input...I want to check whether name is present in "username" or "nickname"....If someone can suggest me a way, i will really happy... Thanks... -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Tue May 21 09:50:13 2013 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 21 May 2013 09:50:13 +0200 Subject: [erlang-questions] Mnesia - Check for a value in either of the keys in the table In-Reply-To: References: Message-ID: I assume that username is a primary key column, hence you can just use: mnesia:transaction(fun() -> case mnesia:read(users, UsernameOrNickname) of [] -> {error, "Username not found"}; [User] -> {ok, User} end) But nickname column is not a key, hence you have to add an index for it first: mnesia:add_table_index(users, nickname) and later you can use the following code to fetch user by username or nickname: mnesia:transaction(fun() -> case mnesia:read(users, UsernameOrNickname) of [] -> case mnesia:index_read(users, UsernameOrNickname, #users.nickname) of [] -> {error, "User not found"}; [User1] -> {ok, User1} end; [User2] -> {ok, User2} end) Code written from top of my head without testing, take with a grain of salt. Cheers, Gleb On Tue, May 21, 2013 at 3:09 AM, Manoj Raj wrote: > Hi, > > I have a mnesia table "users" created with the following record > > -record(users,{username,nickname,age}) > > I have a name now to input...I want to check whether name is present in > "username" or "nickname"....If someone can suggest me a way, i will really > happy... > > > Thanks... > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gleber.p@REDACTED Tue May 21 09:54:31 2013 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 21 May 2013 09:54:31 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: I believe there is no API nor technical mean to make NIFs reentrant currently. Hence you should refrain from making NIFs which can run for a long time. But if you want to make NIFs more friendly to the scheduler, you can use erlang:bump_reductions/1 from Erlang wrapper of a NIF call. On Tue, May 21, 2013 at 4:14 AM, ??? wrote: > hi~ > > I try to design some nif codes recently and I reading a book " Erlang and > OTP in Action" now. In this book, chapter 12, "Integrating with foreign code > using ports and NIFs", page 313, section "Making a linked driver". > > I found some words: > " > Because all these instances execute > within the memory of the Erlang VM, possibly running as different threads, the driver > code must be designed to be reentrant?executable by multiple simultaneous callers? > and must not depend on global variables or locks. > " > > and then, this book give some methods to avoid this problem, something like > not to use global variables, and use driver_alloc, driver_free to allocate > memory dynamically. > > of course, these words were placed into the section "Making a linked > driver" > > My questions is: > > 1) Is nif driver c code has the same problem? That is, nif driver c code > must be designed to reentrant? > 2) I know that, nif calling will block the VM until the driver c code > returns, Is that other problems we must notice when using nifs? > > And if nif driver c code must be designed to reentrant, then I will have > anther question. > I read this words from erlang.org > > " > NIFs are called directly by the same scheduler thread that executed the > calling Erlang code. The calling scheduler will thus be blocked from doing any > other work until the NIF returns > " > > That is, calling scheduler is driven by the same thread, so the calling > will be blocked. > Then why should we designed our nif code to be reentrant? > > > The e-mail is little long, thanks for reading, and I do hope to receive any > helpful messages. > Thanks! > > > > > Best Regards > ckwei > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rapsey@REDACTED Tue May 21 10:33:20 2013 From: rapsey@REDACTED (Rapsey) Date: Tue, 21 May 2013 10:33:20 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: My experience with R16B and frequent NIF calls with bump_reductions is very negative. Without bump_reductions even worse. At the moment I would not use NIFs for anything. Everything will look ok, but once server starts getting busy it will completely stop working. Sergej On Tue, May 21, 2013 at 9:54 AM, Gleb Peregud wrote: > I believe there is no API nor technical mean to make NIFs reentrant > currently. Hence you should refrain from making NIFs which can run for > a long time. > > But if you want to make NIFs more friendly to the scheduler, you can > use erlang:bump_reductions/1 from Erlang wrapper of a NIF call. > > On Tue, May 21, 2013 at 4:14 AM, ?????j wrote: >> hi~ >> >> I try to design some nif codes recently and I reading a book " Erlang and >> OTP in Action" now. In this book, chapter 12, "Integrating with foreign code >> using ports and NIFs", page 313, section "Making a linked driver". >> >> I found some words: >> " >> Because all these instances execute >> within the memory of the Erlang VM, possibly running as different threads, the driver >> code must be designed to be reentrant??executable by multiple simultaneous callers?? >> and must not depend on global variables or locks. >> " >> >> and then, this book give some methods to avoid this problem, something like >> not to use global variables, and use driver_alloc, driver_free to allocate >> memory dynamically. >> >> of course, these words were placed into the section "Making a linked >> driver" >> >> My questions is: >> >> 1) Is nif driver c code has the same problem? That is, nif driver c code >> must be designed to reentrant? >> 2) I know that, nif calling will block the VM until the driver c code >> returns, Is that other problems we must notice when using nifs? >> >> And if nif driver c code must be designed to reentrant, then I will have >> anther question. >> I read this words from erlang.org >> >> " >> NIFs are called directly by the same scheduler thread that executed the >> calling Erlang code. The calling scheduler will thus be blocked from doing any >> other work until the NIF returns >> " >> >> That is, calling scheduler is driven by the same thread, so the calling >> will be blocked. >> Then why should we designed our nif code to be reentrant? >> >> >> The e-mail is little long, thanks for reading, and I do hope to receive any >> helpful messages. >> Thanks! >> >> >> >> >> Best Regards >> ckwei >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From attila.r.nohl@REDACTED Tue May 21 10:39:16 2013 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 21 May 2013 10:39:16 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: 2013/5/21 ?????j : [...] > " > NIFs are called directly by the same scheduler thread that executed the > calling Erlang code. The calling scheduler will thus be blocked from doing any > other work until the NIF returns > " > > That is, calling scheduler is driven by the same thread, so the calling > will be blocked. > Then why should we designed our nif code to be reentrant? I think on SMP systems you can have multiple schedulers. From max.lapshin@REDACTED Tue May 21 10:53:01 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 21 May 2013 12:53:01 +0400 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: Rapsey: I remember that you do a lot of nif work, you are delivering large amount of video in nif thread. How do you solve it now? From rapsey@REDACTED Tue May 21 12:08:09 2013 From: rapsey@REDACTED (Rapsey) Date: Tue, 21 May 2013 12:08:09 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: I keep my streaming servers in R14 where it works great. Calls generally scan a buffer and do a write to a non blocking socket. I don't know why R16 has such an issue with it. Sergej On Tue, May 21, 2013 at 10:53 AM, Max Lapshin wrote: > Rapsey: I remember that you do a lot of nif work, you are delivering > large amount of video in nif thread. > > How do you solve it now? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dangud@REDACTED Tue May 21 15:31:01 2013 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 21 May 2013 15:31:01 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: 1) Is nif driver c code has the same problem? That is, nif driver c code must be designed to reentrant? Yes 2) I know that, nif calling will block the VM until the driver c code returns, Is that other problems we must notice when using nifs? See http://www.erlang.org/doc/man/erl_nif.html#WARNING /Dan On Tue, May 21, 2013 at 4:14 AM, ?????j wrote: > hi~ > > I try to design some nif codes recently and I reading a book " Erlang and > OTP in Action" now. In this book, chapter 12, "Integrating with foreign > code > using ports and NIFs", page 313, section "Making a linked driver". > > I found some words: > " > Because all these instances execute > > within the memory of the Erlang VM, possibly running as different threads, the driver > > code must be designed to be reentrant??executable by multiple simultaneous callers?? > and must not depend on global variables or locks. > " > > and then, this book give some methods to avoid this problem, something like > not to use global variables, and use driver_alloc, driver_free to allocate > memory dynamically. > > of course, these words were placed into the section "Making a linked > driver" > > My questions is: > > 1) Is nif driver c code has the same problem? That is, nif driver c code > must be designed to reentrant? > 2) I know that, nif calling will block the VM until the driver c code > returns, Is that other problems we must notice when using nifs? > > And if nif driver c code must be designed to reentrant, then I will have > anther question. > I read this words from erlang.org > > " > NIFs are called directly by the same scheduler thread that executed the > calling Erlang code. The calling scheduler will thus be blocked from doing > any > other work until the NIF returns > " > > That is, calling scheduler is driven by the same thread, so the calling > will be blocked. > Then why should we designed our nif code to be reentrant? > > > The e-mail is little long, thanks for reading, and I do hope to receive any > helpful messages. > Thanks! > > > > > Best Regards > ckwei > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Tue May 21 15:34:46 2013 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 21 May 2013 15:34:46 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: References: <1369102464.57871.YahooMailNeo@web15604.mail.cnb.yahoo.com> Message-ID: If you have lengthy work that you can break it in several parts, you can do easily like this: lengthy_work(Data) -> case lengthy_work_impl(Data) of Cont = {continue, _State} -> erlang:bump_reductions(1000000), lengthy_work_impl(Cont); {finished, Result} -> Result end. lengthy_work_impl(_) -> erlang:nif_error(undefined). On Tue, May 21, 2013 at 9:54 AM, Gleb Peregud wrote: > I believe there is no API nor technical mean to make NIFs reentrant > currently. Hence you should refrain from making NIFs which can run for > a long time. > > But if you want to make NIFs more friendly to the scheduler, you can > use erlang:bump_reductions/1 from Erlang wrapper of a NIF call. > > On Tue, May 21, 2013 at 4:14 AM, ?????j wrote: > > hi~ > > > > I try to design some nif codes recently and I reading a book " Erlang and > > OTP in Action" now. In this book, chapter 12, "Integrating with foreign > code > > using ports and NIFs", page 313, section "Making a linked driver". > > > > I found some words: > > " > > Because all these instances execute > > within the memory of the Erlang VM, possibly running as different > threads, the driver > > code must be designed to be reentrant??executable by multiple > simultaneous callers?? > > and must not depend on global variables or locks. > > " > > > > and then, this book give some methods to avoid this problem, something > like > > not to use global variables, and use driver_alloc, driver_free to > allocate > > memory dynamically. > > > > of course, these words were placed into the section "Making a linked > > driver" > > > > My questions is: > > > > 1) Is nif driver c code has the same problem? That is, nif driver c code > > must be designed to reentrant? > > 2) I know that, nif calling will block the VM until the driver c code > > returns, Is that other problems we must notice when using nifs? > > > > And if nif driver c code must be designed to reentrant, then I will have > > anther question. > > I read this words from erlang.org > > > > " > > NIFs are called directly by the same scheduler thread that executed the > > calling Erlang code. The calling scheduler will thus be blocked from > doing any > > other work until the NIF returns > > " > > > > That is, calling scheduler is driven by the same thread, so the calling > > will be blocked. > > Then why should we designed our nif code to be reentrant? > > > > > > The e-mail is little long, thanks for reading, and I do hope to receive > any > > helpful messages. > > Thanks! > > > > > > > > > > Best Regards > > ckwei > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Tue May 21 17:56:59 2013 From: dmercer@REDACTED (David Mercer) Date: Tue, 21 May 2013 10:56:59 -0500 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> Message-ID: <003b01ce563b$d48ad2d0$7da07870$@gmail.com> Richard A. O'Keefe wrote: > > On May 17, 2013, at 9:40 AM, David Mercer wrote: > > > >> Richard A. O'Keefe wrote: > >> > >>> Part of the problem is precisely the fact that the parser is > written > >>> in Yecc. > >> > >> Back in the day, I'd have known the answer to this, but I figure > some > >> of you are more knowledgeable than I ever was: What parsing > >> technology besides hand-rolled would produce better error messages? > ANTLR? > >> Parsec? PEG? > > > > > > Menhir > > > > http://gallium.inria.fr/~fpottier/menhir/ > > Menhir is definitely interesting, and if I had experienced anything > other than extreme frustration with Godi, I would have liked to try it. > But judging from section 8 of the Menhir manual, a Menhir-built parser > would be no better than a Yecc-built one here. > It might even be worse: the manual says that the current lookahead > token is *replaced* by an ERROR token as part of recovery, and in the > example we are discussing, the token in question is ")" and it is > correct, and should be retained. What is more, the Menhir manual is > quite explicit that error detection may be deferred. > > What the user needs in cases like this is the information that an > earlier token (fun) caused the parser to be looking for a particular > token (end) but it encountered a closer (right paren) before finding > it. Something basically top-down should have an easier time of it. Is there a grammar language/parsing technology that you could use for this, or would it all pretty much be hand-roll-your-own? From robert.virding@REDACTED Tue May 21 20:05:47 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 21 May 2013 19:05:47 +0100 (BST) Subject: [erlang-questions] Luerl and dynamically created lists In-Reply-To: Message-ID: <1922507690.122459233.1369159547177.JavaMail.root@erlang-solutions.com> Variables in Erlang are no problem. In fact in one sense they don't even exist. An Erlang variable is not like a variable in imperative or OO languages, a "hole" where you can store data, but more just a reference to data which the compiler uses. Atoms, however, definitely do exist. :-) They are literal constants with a name. And you should definitely avoid dynamically creating atoms in an uncontrolled fashion or you will eventually fill the atom table and crash the system. With Luerl this is no problem as it never creates or uses atoms. Well, to be honest it uses 3: true, false and nil. All Luerl state is kept in a big data structure (a deep nested structure of tuples, lists, records, dicts and arrays) which is passed in when you "call" Luerl, modified as execution proceeds and finally returned to be used in the next call to Luerl. As this is a normal Erlang data structure so the Erlang GC will cleanup and reclaim unused data. You should, however, call the Luerl GC occasionally, either from Erlang by doing: NewState = luerl:gc(State), and then continue with NewState, or from inside Lua with "collectgarbage('collect')". Each Luerl invocation has its own separate state with no sharing. Note that this state contains everything, all tables, stack data, local variables everything. Whether this is a good or bad thing can be discussed. This means that if you want to send Lua data, for example a table, from one process to another you must first decode into an Erlang representation, send it, then encode into Lua data in the new state. After the data has been sent there is no need to worry about erasing or reclaiming the data in the sending process, the Erlang GC will take care of it. As Luerl state is just normal Erlang data there is no need to erase or reclaim its data as it too will be reclaimed by the Erlang GC when there are no references to it. It is actually impossible to explicitly erase or reclaim data in Erlang. This is common to most languages which have automatic memory management, like Java or Ruby, or Lua for that matter. Robert ----- Original Message ----- > From: "Martin Hedberg" > To: erlang-questions@REDACTED > Sent: Tuesday, 21 May, 2013 4:35:17 AM > Subject: [erlang-questions] Luerl and dynamically created lists > Hi again > I know that it is a big "no no" to create variables dynamically in > Erlang. However I am planning to use Mr Virdings "Luerl" > modules so Lua-engienes (on top of Erlang processes) can handle some > of the more simple programming tasks. While the > Erlang processes handle the communication between them. > When the Lua-processes produce variables dynamically I thought it > could just pass it via the Erlang layer . However as I have > understood it Erlang is just sending copies. Which makes me a bit > worried about the memory. > Is there some way to erase The Lua-content from an Erlang process > after it have been sent? > Best regards > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Tue May 21 20:13:57 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 21 May 2013 13:13:57 -0500 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: Message of "Tue, 21 May 2013 15:34:46 +0200." Message-ID: <67065.1369160037@snookles.snookles.com> Dan Gudmundsson wrote: dg> [...] dg> erlang:bump_reductions(1000000) dg> [...] Dan, that prompts a good question that has been bugging several people (including me). Does this 1 million reduction count bump have an effect more than a 2000 reduction bump? IIRC, a process is scheduled out after 2000 reductions. And my (poor) memory says that the reduction count starts at 2000, that count is decremented, and a scheduling decision is made when the reduction count goes negative. Does it really make a difference if the count goes negative by a little (e.g., only one) or by a lot (e.g., 999,990)? -Scott From robert.virding@REDACTED Tue May 21 21:59:13 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 21 May 2013 20:59:13 +0100 (BST) Subject: [erlang-questions] pattern matching registered name In-Reply-To: <2912A149-12A6-4919-A19C-550CC9128B89@cs.otago.ac.nz> Message-ID: <683039329.122556136.1369166353714.JavaMail.root@erlang-solutions.com> Yes, the registered name is just an atom, it is the '!' which interprets this atom as the registered name of a process, if it exists. It is still just a normal atom, the difference is in the interpretation. ----- Original Message ----- > From: "Richard A. O'Keefe" > To: "Edmund Sumbar" > Cc: erlang-questions@REDACTED > Sent: Tuesday, 21 May, 2013 3:47:33 AM > Subject: Re: [erlang-questions] pattern matching registered name > > > In a Who ! What send, Who may be a pid or a registered name. > In a pattern match, registered names do not match pids; they are > just literal atoms. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From n.oxyde@REDACTED Tue May 21 22:52:02 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 21 May 2013 22:52:02 +0200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <003b01ce563b$d48ad2d0$7da07870$@gmail.com> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> <003b01ce563b$d48ad2d0$7da07870$@gmail.com> Message-ID: I don't think there is any parsing technology that does all the things Clang do, and Clang is definitely the way to go with regard to compiler diagnostics. I would love to be wrong on this. -- Anthony Ramine Le 21 mai 2013 ? 17:56, David Mercer a ?crit : > Richard A. O'Keefe wrote: > >>> On May 17, 2013, at 9:40 AM, David Mercer wrote: >>> >>>> Richard A. O'Keefe wrote: >>>> >>>>> Part of the problem is precisely the fact that the parser is >> written >>>>> in Yecc. >>>> >>>> Back in the day, I'd have known the answer to this, but I figure >> some >>>> of you are more knowledgeable than I ever was: What parsing >>>> technology besides hand-rolled would produce better error messages? >> ANTLR? >>>> Parsec? PEG? >>> >>> >>> Menhir >>> >>> http://gallium.inria.fr/~fpottier/menhir/ >> >> Menhir is definitely interesting, and if I had experienced anything >> other than extreme frustration with Godi, I would have liked to try it. >> But judging from section 8 of the Menhir manual, a Menhir-built parser >> would be no better than a Yecc-built one here. >> It might even be worse: the manual says that the current lookahead >> token is *replaced* by an ERROR token as part of recovery, and in the >> example we are discussing, the token in question is ")" and it is >> correct, and should be retained. What is more, the Menhir manual is >> quite explicit that error detection may be deferred. >> >> What the user needs in cases like this is the information that an >> earlier token (fun) caused the parser to be looking for a particular >> token (end) but it encountered a closer (right paren) before finding >> it. Something basically top-down should have an easier time of it. > > Is there a grammar language/parsing technology that you could use for this, or would it all pretty much be hand-roll-your-own? > From thomasl_erlang@REDACTED Wed May 22 00:02:45 2013 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 21 May 2013 15:02:45 -0700 (PDT) Subject: [erlang-questions] Luerl and dynamically created lists In-Reply-To: <1922507690.122459233.1369159547177.JavaMail.root@erlang-solutions.com> References: <1922507690.122459233.1369159547177.JavaMail.root@erlang-solutions.com> Message-ID: <1369173765.28164.YahooMailNeo@web140106.mail.bf1.yahoo.com> I think the erlang community has gotten wedged in a needlessly unconstructive state regarding atoms. Not sure why? Best, Thomas >________________________________ > From: Robert Virding > > > >Atoms, however, definitely do exist. :-) They are literal constants with a name. And you should definitely avoid dynamically creating atoms in an uncontrolled fashion or you will eventually fill the atom table and crash the system. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Wed May 22 00:51:40 2013 From: dmercer@REDACTED (David Mercer) Date: Tue, 21 May 2013 17:51:40 -0500 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> <003b01ce563b$d48ad2d0$7da07870$@gmail.com> Message-ID: <006c01ce5675$c2516680$46f43380$@gmail.com> Isn't Clang and example of roll-your-own parser? I thought it was developed to parse C and C only (and C derivatives). Cheers, DBM > -----Original Message----- > From: Anthony Ramine [mailto:n.oxyde@REDACTED] > Sent: Tuesday, May 21, 2013 15:52 > To: David Mercer > Cc: ok@REDACTED; 'Siraaj Khandkar'; 'Erlang-Questions Questions' > Subject: Re: [erlang-questions] Better error messages would be helpful > > I don't think there is any parsing technology that does all the things > Clang do, and Clang is definitely the way to go with regard to compiler > diagnostics. I would love to be wrong on this. > > -- > Anthony Ramine > > Le 21 mai 2013 ? 17:56, David Mercer a ?crit : > > > Richard A. O'Keefe wrote: > > > >>> On May 17, 2013, at 9:40 AM, David Mercer > wrote: > >>> > >>>> Richard A. O'Keefe wrote: > >>>> > >>>>> Part of the problem is precisely the fact that the parser is > >> written > >>>>> in Yecc. > >>>> > >>>> Back in the day, I'd have known the answer to this, but I figure > >> some > >>>> of you are more knowledgeable than I ever was: What parsing > >>>> technology besides hand-rolled would produce better error > messages? > >> ANTLR? > >>>> Parsec? PEG? > >>> > >>> > >>> Menhir > >>> > >>> http://gallium.inria.fr/~fpottier/menhir/ > >> > >> Menhir is definitely interesting, and if I had experienced anything > >> other than extreme frustration with Godi, I would have liked to try > it. > >> But judging from section 8 of the Menhir manual, a Menhir-built > >> parser would be no better than a Yecc-built one here. > >> It might even be worse: the manual says that the current lookahead > >> token is *replaced* by an ERROR token as part of recovery, and in > the > >> example we are discussing, the token in question is ")" and it is > >> correct, and should be retained. What is more, the Menhir manual is > >> quite explicit that error detection may be deferred. > >> > >> What the user needs in cases like this is the information that an > >> earlier token (fun) caused the parser to be looking for a particular > >> token (end) but it encountered a closer (right paren) before finding > >> it. Something basically top-down should have an easier time of it. > > > > Is there a grammar language/parsing technology that you could use for > this, or would it all pretty much be hand-roll-your-own? > > From n.oxyde@REDACTED Wed May 22 03:06:44 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 22 May 2013 03:06:44 +0200 Subject: [erlang-questions] Better error messages would be helpful In-Reply-To: <006c01ce5675$c2516680$46f43380$@gmail.com> References: <1D26BCB7-A062-4445-99D6-25B0C2E60C5D@cs.otago.ac.nz> <004701ce5304$1017c190$304744b0$@gmail.com> <1C9EF4BE-E598-4E91-9C68-EA3B1C1B3F74@khandkar.net> <003b01ce563b$d48ad2d0$7da07870$@gmail.com> <006c01ce5675$c2516680$46f43380$@gmail.com> Message-ID: Yes it is a roll-your-own parser. -- Anthony Ramine Le 22 mai 2013 ? 00:51, David Mercer a ?crit : > Isn't Clang and example of roll-your-own parser? I thought it was developed to parse C and C only (and C derivatives). > > Cheers, > > DBM > > >> -----Original Message----- >> From: Anthony Ramine [mailto:n.oxyde@REDACTED] >> Sent: Tuesday, May 21, 2013 15:52 >> To: David Mercer >> Cc: ok@REDACTED; 'Siraaj Khandkar'; 'Erlang-Questions Questions' >> Subject: Re: [erlang-questions] Better error messages would be helpful >> >> I don't think there is any parsing technology that does all the things >> Clang do, and Clang is definitely the way to go with regard to compiler >> diagnostics. I would love to be wrong on this. >> >> -- >> Anthony Ramine >> >> Le 21 mai 2013 ? 17:56, David Mercer a ?crit : >> >>> Richard A. O'Keefe wrote: >>> >>>>> On May 17, 2013, at 9:40 AM, David Mercer >> wrote: >>>>> >>>>>> Richard A. O'Keefe wrote: >>>>>> >>>>>>> Part of the problem is precisely the fact that the parser is >>>> written >>>>>>> in Yecc. >>>>>> >>>>>> Back in the day, I'd have known the answer to this, but I figure >>>> some >>>>>> of you are more knowledgeable than I ever was: What parsing >>>>>> technology besides hand-rolled would produce better error >> messages? >>>> ANTLR? >>>>>> Parsec? PEG? >>>>> >>>>> >>>>> Menhir >>>>> >>>>> http://gallium.inria.fr/~fpottier/menhir/ >>>> >>>> Menhir is definitely interesting, and if I had experienced anything >>>> other than extreme frustration with Godi, I would have liked to try >> it. >>>> But judging from section 8 of the Menhir manual, a Menhir-built >>>> parser would be no better than a Yecc-built one here. >>>> It might even be worse: the manual says that the current lookahead >>>> token is *replaced* by an ERROR token as part of recovery, and in >> the >>>> example we are discussing, the token in question is ")" and it is >>>> correct, and should be retained. What is more, the Menhir manual is >>>> quite explicit that error detection may be deferred. >>>> >>>> What the user needs in cases like this is the information that an >>>> earlier token (fun) caused the parser to be looking for a particular >>>> token (end) but it encountered a closer (right paren) before finding >>>> it. Something basically top-down should have an easier time of it. >>> >>> Is there a grammar language/parsing technology that you could use for >> this, or would it all pretty much be hand-roll-your-own? >>> > > From bog495@REDACTED Wed May 22 09:00:12 2013 From: bog495@REDACTED (Bogdan Andu) Date: Wed, 22 May 2013 10:00:12 +0300 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c Message-ID: Hello, I recently compiled otp_src_R14B04 with OpenSSL version 1.0.1c. It is detected ok, but when I try to start crypto app I get the follwing error: Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> crypto:start(). /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' =ERROR REPORT==== 21-May-2013::15:19:12 === Unable to load crypto library. Failed with error: "load_failed, Failed to load NIF library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load specified object'" OpenSSL might not be installed on this system. =ERROR REPORT==== 21-May-2013::15:19:12 === The on_load function for module crypto returned {error, {load_failed, "Failed to load NIF library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load specified object'"}} ** exception error: undefined function crypto:start/0 2> It seems that /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so library tries to access the symbol '__guard_local' that no longer seems to exists in libssl.so.19.0 . On OpenSSL verion 1.0.0f there is no problem, only in version 1.0.1c . What can be done to have ssl functionality as I need it. Thank you in advanced, Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From stas@REDACTED Wed May 22 09:09:16 2013 From: stas@REDACTED (Stanislav Sedov) Date: Wed, 22 May 2013 00:09:16 -0700 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: References: Message-ID: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> On May 22, 2013, at 12:00 AM, Bogdan Andu wrote: > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: undefined symbol '__guard_local' > > =ERROR REPORT==== 21-May-2013::15:19:12 === > Unable to load crypto library. Failed with error: > "load_failed, Failed to load NIF library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load specified object'" > OpenSSL might not be installed on this system. > > =ERROR REPORT==== 21-May-2013::15:19:12 === > The on_load function for module crypto returned {error, > {load_failed, > "Failed to load NIF library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load specified object'"}} > ** exception error: undefined function crypto:start/0 > 2> > > It seems that /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so library tries to access the symbol '__guard_local' that no longer seems to exists in libssl.so.19.0 . > > On OpenSSL verion 1.0.0f there is no problem, only in version 1.0.1c . > > What can be done to have ssl functionality as I need it. > What OS is it? Do you know where this symbol is coming from? I don't think it comes from Erlang -- most likely openssl and erlang was built agains different headers/libraries. -- ST4096-RIPE From bog495@REDACTED Wed May 22 09:18:06 2013 From: bog495@REDACTED (bog495@REDACTED) Date: Wed, 22 May 2013 00:18:06 -0700 (PDT) Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: References: Message-ID: <7f900962-d58d-44ce-8454-48d2e8b05de4@googlegroups.com> The OS is OpenBSD 5.3 amd64 with OpenSSL version 1.0.1c. On OpenBSD 5.2 amd64 with OpenSSL version 1.0.0f everything work fine. Openssl 1.0.1f came preinstalled with the OS, and this is the only openssl library in the system. Erlang was built against this library that came with the OS. On Wednesday, May 22, 2013 10:00:12 AM UTC+3, Bogdan Andu wrote: > > Hello, > > I recently compiled otp_src_R14B04 with OpenSSL version 1.0.1c. > > It is detected ok, but when I try to start crypto app I get the follwing > error: > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [kernel-poll:false] > > Eshell V5.8.5 (abort with ^G) > 1> crypto:start(). > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > =ERROR REPORT==== 21-May-2013::15:19:12 === > Unable to load crypto library. Failed with error: > "load_failed, Failed to load NIF library > /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load > specified object'" > OpenSSL might not be installed on this system. > > =ERROR REPORT==== 21-May-2013::15:19:12 === > The on_load function for module crypto returned {error, > {load_failed, > "Failed to load NIF > library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot > load specified object'"}} > ** exception error: undefined function crypto:start/0 > 2> > > It seems that > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so > library tries to access the symbol '__guard_local' that no longer seems > to exists in libssl.so.19.0 . > > On OpenSSL verion 1.0.0f there is no problem, only in version 1.0.1c . > > What can be done to have ssl functionality as I need it. > > Thank you in advanced, > > Bogdan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Wed May 22 09:31:32 2013 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 22 May 2013 09:31:32 +0200 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: <67065.1369160037@snookles.snookles.com> References: <67065.1369160037@snookles.snookles.com> Message-ID: I believe you are correct, I just grabbed large enough number that will work in the example even if things are changed. Even better would be to bump accordingly in the nif with the enif_consume_timeslice, but then it wouldn't be a short example anymore. /Dan On Tue, May 21, 2013 at 8:13 PM, Scott Lystig Fritchie < fritchie@REDACTED> wrote: > Dan Gudmundsson wrote: > > dg> [...] > dg> erlang:bump_reductions(1000000) > dg> [...] > > Dan, that prompts a good question that has been bugging several people > (including me). Does this 1 million reduction count bump have an effect > more than a 2000 reduction bump? IIRC, a process is scheduled out after > 2000 reductions. And my (poor) memory says that the reduction count > starts at 2000, that count is decremented, and a scheduling decision is > made when the reduction count goes negative. > > Does it really make a difference if the count goes negative by a little > (e.g., only one) or by a lot (e.g., 999,990)? > > -Scott > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Wed May 22 09:34:18 2013 From: bog495@REDACTED (Bogdan Andu) Date: Wed, 22 May 2013 10:34:18 +0300 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> Message-ID: The os is OpenBSD 5.3 amd64 with preinstalled opensssl library /usr/lib/libssl.so.19.0 $ /usr/sbin/openssl OpenSSL> version OpenSSL 1.0.1c 10 May 2012 OpenSSL> ^D This is the only ssl library in the system, and erlang was configured with this ssl library and no complaints at ./configure time. On the other hand: On OpenBSD 5.2 amd64 with preinstalled opensssl library /usr/lib/libssl.so.18.0 $ /usr/sbin/openssl OpenSSL> version OpenSSL 1.0.0f 4 Jan 2012 OpenSSL> ^D everything works perfect, and Erlang was also configured againsat this preinstalled library On Wed, May 22, 2013 at 10:09 AM, Stanislav Sedov wrote: > > On May 22, 2013, at 12:00 AM, Bogdan Andu wrote: > > > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so: > undefined symbol '__guard_local' > > > > =ERROR REPORT==== 21-May-2013::15:19:12 === > > Unable to load crypto library. Failed with error: > > "load_failed, Failed to load NIF library > /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot load > specified object'" > > OpenSSL might not be installed on this system. > > > > =ERROR REPORT==== 21-May-2013::15:19:12 === > > The on_load function for module crypto returned {error, > > {load_failed, > > "Failed to load NIF > library /usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto: 'Cannot > load specified object'"}} > > ** exception error: undefined function crypto:start/0 > > 2> > > > > It seems that > /usr/local/lib/erlang/erts-5.8.5/bin/beam.smp:/usr/local/lib/erlang/lib/crypto-2.0.4/priv/lib/crypto.so > library tries to access the symbol '__guard_local' that no longer seems > to exists in libssl.so.19.0 . > > > > On OpenSSL verion 1.0.0f there is no problem, only in version 1.0.1c . > > > > What can be done to have ssl functionality as I need it. > > > > What OS is it? Do you know where this symbol is coming from? > I don't think it comes from Erlang -- most likely openssl and erlang was > built agains different > headers/libraries. > > -- > ST4096-RIPE > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stas@REDACTED Wed May 22 10:46:45 2013 From: stas@REDACTED (Stanislav Sedov) Date: Wed, 22 May 2013 01:46:45 -0700 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> Message-ID: <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> On May 22, 2013, at 12:34 AM, Bogdan Andu wrote: > The os is OpenBSD 5.3 amd64 with preinstalled opensssl library /usr/lib/libssl.so.19.0 > > $ /usr/sbin/openssl > OpenSSL> version > OpenSSL 1.0.1c 10 May 2012 > OpenSSL> > ^D > > This is the only ssl library in the system, and erlang was configured with this ssl library > and no complaints at ./configure time. > > On the other hand: > > On OpenBSD 5.2 amd64 with preinstalled opensssl library /usr/lib/libssl.so.18.0 > > $ /usr/sbin/openssl > OpenSSL> version > OpenSSL 1.0.0f 4 Jan 2012 > OpenSSL> > ^D > > everything works perfect, and Erlang was also configured againsat this preinstalled library > I found this message on the mailing list, which seems to be relevant: http://www.mail-archive.com/cdesktopenv-devel@REDACTED/msg00598.html It looks like the __guard_local symbol is generated by new stack_protector code, and according to the patch it seems that linking against libcrtbeginS.o and libcrtendS.o fixes the issue for CDE. It might be worth trying to modify the erlang linker flags (or at least crypto linking flags) to link against these objects. However, I don't really understand what's going on here. According to [1], the new __guard_local symbol should be present in crtbegin.o. But maybe crypto.so is being linked manually by erlang build system (i.e. not via cc(1)), in which case applying the same fix as was provided for CDE might help. [1]: http://openbsd.7691.n7.nabble.com/GCC-diff-needs-testing-on-multiple-arches-td167891.html -- ST4096-RIPE From bog495@REDACTED Wed May 22 11:23:41 2013 From: bog495@REDACTED (Bogdan Andu) Date: Wed, 22 May 2013 12:23:41 +0300 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> Message-ID: Thank you for the pointers. I'll give it a try to see what happens. It seems that linking erlang against a previous, working ssl library would give the same error. Bogdan On Wed, May 22, 2013 at 11:46 AM, Stanislav Sedov wrote: > > On May 22, 2013, at 12:34 AM, Bogdan Andu wrote: > > > The os is OpenBSD 5.3 amd64 with preinstalled opensssl library > /usr/lib/libssl.so.19.0 > > > > $ /usr/sbin/openssl > > OpenSSL> version > > OpenSSL 1.0.1c 10 May 2012 > > OpenSSL> > > ^D > > > > This is the only ssl library in the system, and erlang was configured > with this ssl library > > and no complaints at ./configure time. > > > > On the other hand: > > > > On OpenBSD 5.2 amd64 with preinstalled opensssl library > /usr/lib/libssl.so.18.0 > > > > $ /usr/sbin/openssl > > OpenSSL> version > > OpenSSL 1.0.0f 4 Jan 2012 > > OpenSSL> > > ^D > > > > everything works perfect, and Erlang was also configured againsat this > preinstalled library > > > > I found this message on the mailing list, which seems to be relevant: > > http://www.mail-archive.com/cdesktopenv-devel@REDACTED/msg00598.html > > It looks like the __guard_local symbol is generated by new stack_protector > code, > and according to the patch it seems that linking against libcrtbeginS.o and > libcrtendS.o fixes the issue for CDE. It might be worth trying to modify > the > erlang linker flags (or at least crypto linking flags) to link against > these > objects. > > However, I don't really understand what's going on here. According to [1], > the new __guard_local symbol should be present in crtbegin.o. But maybe > crypto.so > is being linked manually by erlang build system (i.e. not via cc(1)), in > which case > applying the same fix as was provided for CDE might help. > > [1]: > http://openbsd.7691.n7.nabble.com/GCC-diff-needs-testing-on-multiple-arches-td167891.html > > -- > ST4096-RIPE > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jann@REDACTED Wed May 22 13:09:28 2013 From: jann@REDACTED (Jann Horn) Date: Wed, 22 May 2013 13:09:28 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? Message-ID: <20130522110928.GP15261@debjann.fritz.box> This is strace output from a totally idle couchdb process: [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 EAGAIN (Resource temporarily unavailable) [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 This seems to be, at least partly, intentional ? erts/emulator/beam/erl_process.c contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is set to 10. Can anyone tell me what the rationale behind this excessive busylooping is? A few dozen syscalls per second for nothing seems a bit weird to me. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From bchesneau@REDACTED Wed May 22 15:46:17 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 22 May 2013 15:46:17 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: <20130522110928.GP15261@debjann.fritz.box> References: <20130522110928.GP15261@debjann.fritz.box> Message-ID: How would it work to receive an event to wake up if doesn't listen on them? - benoit On Wed, May 22, 2013 at 1:09 PM, Jann Horn wrote: > This is strace output from a totally idle couchdb process: > > [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 > [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 > [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 > [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 > [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 > [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 EAGAIN (Resource temporarily unavailable) > [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 > [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 > [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 > [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 > > This seems to be, at least partly, intentional ? erts/emulator/beam/erl_process.c > contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is set to 10. > > Can anyone tell me what the rationale behind this excessive busylooping is? A few > dozen syscalls per second for nothing seems a bit weird to me. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jann@REDACTED Wed May 22 15:56:38 2013 From: jann@REDACTED (Jann Horn) Date: Wed, 22 May 2013 15:56:38 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: References: <20130522110928.GP15261@debjann.fritz.box> Message-ID: <20130522135638.GQ15261@debjann.fritz.box> On Wed, May 22, 2013 at 03:46:17PM +0200, Benoit Chesneau wrote: > How would it work to receive an event to wake up if doesn't listen on them? Uh... register all fds you're interested in using epoll, then do epoll_wait(fd, events, maxevents, -1)? Isn't that actually quite normal? The syscall will return as soon as something happens, but not earlier. You don't have to poll different event sources, there are many facilities that can multiplex those event sources for you and will wake you up as soon as something interesting happens. E.g. select, poll, epoll, kqueue and event ports (the last two aren't available on linux). I can't imagine a reason why you'd have to poll events like this in any OS. > On Wed, May 22, 2013 at 1:09 PM, Jann Horn wrote: > > This is strace output from a totally idle couchdb process: > > > > [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 > > [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 > > [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 > > [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 > > [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 > > [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 EAGAIN (Resource temporarily unavailable) > > [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 > > [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN, {u32=10, u64=73199780460757002}}) = 0 > > [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 > > [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 > > > > This seems to be, at least partly, intentional ? erts/emulator/beam/erl_process.c > > contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is set to 10. > > > > Can anyone tell me what the rationale behind this excessive busylooping is? A few > > dozen syscalls per second for nothing seems a bit weird to me. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From garazdawi@REDACTED Wed May 22 18:42:59 2013 From: garazdawi@REDACTED (Lukas Larsson) Date: Wed, 22 May 2013 18:42:59 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: <20130522135638.GQ15261@debjann.fritz.box> References: <20130522110928.GP15261@debjann.fritz.box> <20130522135638.GQ15261@debjann.fritz.box> Message-ID: You have to poll like that because fd's are not the only thing which can trigger load on the system. Timeouts for instance are triggered by calling gettimeofday which means you have to break out of epoll_wait before the next timeout happens. Also the spinning is done to make the system respond faster to events by delaying sleeping in the kernel. Lukas On Wed, May 22, 2013 at 3:56 PM, Jann Horn wrote: > On Wed, May 22, 2013 at 03:46:17PM +0200, Benoit Chesneau wrote: > > How would it work to receive an event to wake up if doesn't listen on > them? > > Uh... register all fds you're interested in using epoll, then do > epoll_wait(fd, events, maxevents, -1)? Isn't that actually quite > normal? The syscall will return as soon as something happens, but > not earlier. > > You don't have to poll different event sources, there > are many facilities that can multiplex those event sources for > you and will wake you up as soon as something interesting happens. > E.g. select, poll, epoll, kqueue and event ports (the last two aren't > available on linux). I can't imagine a reason why you'd have to > poll events like this in any OS. > > > > On Wed, May 22, 2013 at 1:09 PM, Jann Horn wrote: > > > This is strace output from a totally idle couchdb process: > > > > > > [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 > > > [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 > > > [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 > > > [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 > > > [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 > > > [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 > EAGAIN (Resource temporarily unavailable) > > > [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, {EPOLLIN, > {u32=10, u64=73199780460757002}}) = 0 > > > [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN, > {u32=10, u64=73199780460757002}}) = 0 > > > [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 > > > [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 > > > > > > This seems to be, at least partly, intentional ? > erts/emulator/beam/erl_process.c > > > contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is > set to 10. > > > > > > Can anyone tell me what the rationale behind this excessive > busylooping is? A few > > > dozen syscalls per second for nothing seems a bit weird to me. > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Wed May 22 23:49:14 2013 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Wed, 22 May 2013 14:49:14 -0700 Subject: [erlang-questions] [ANN] erlasticsearch - A thrift-based ElasticSearch client Message-ID: <5721EF26-BEE2-4DD0-BE79-02875B1E6728@dieswaytoofast.com> As the subject-line sez., its a thrift-based erlang client for ElasticSearch, and is available here --> https://github.com/dieswaytoofast/erlasticsearch There is a fairly ridiculous amount of documentation in the associated README, I won't bother to repeat most (or, for that matter, any) of it, but the highlights are * It incorporates connection pooling (via poolboy), which is pretty much the recommended way to use it * For those who won't to get at the individual thrift connections, you can do that too Its fairly comprehensive, but you might find something you really want to be missing - but thats pretty much the way of the world :-( In any case, if you've got any questions, comments or issues, shoot me an email (or github/issue it)? Cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Google+ | Blog | Twitter | LinkedIn -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 04:00:07 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 04:00:07 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers Message-ID: Hello everyone, I'm having a bit of an issue with my production servers. At some point, it seems to enter into an infinite loop that I can't find, or reproduce by myself on the tests servers. The bug appear completely random, 1 hour, or 10 hour after restarting the Erlang node. The loop will eat up all my server's memory in no time, and freeze completely the Erlang node without crashing it. (most of the time) One time I got an crash dump, and tried to investigate it with cdv, but I didn't get much informations about which process or module was eating up all the memory. I just know that it crashed because of the crash message : "eheap_alloc: Cannot allocate 6801972448 bytes of memory (of type "heap")." I'm surely too new to Erlang to investigate something like this with cdv, I really would like some pointers on how I can understand this problem and fix it asap. If you need any informations about the crash dump, let me know what you need, I'll copy/paste? I'm using Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:true] Thank you all for your help ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu May 23 04:21:44 2013 From: bob@REDACTED (Bob Ippolito) Date: Wed, 22 May 2013 19:21:44 -0700 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: Message-ID: This kind of thing tends to happen when you continuously send messages to a process faster than it can handle them. The most common case that I've seen this is where you have a lot of processes communicating with a single gen_server process. If your server has swap enabled, this may appear to make the node "freeze completely but not crash". In the past I've diagnosed this by monitoring the message_queue_len of registered processes, but I'm sure there are tools that can help do this for you. On Wed, May 22, 2013 at 7:00 PM, Morgan Segalis wrote: > Hello everyone, > > I'm having a bit of an issue with my production servers. > > At some point, it seems to enter into an infinite loop that I can't find, > or reproduce by myself on the tests servers. > > The bug appear completely random, 1 hour, or 10 hour after restarting the > Erlang node. > The loop will eat up all my server's memory in no time, and freeze > completely the Erlang node without crashing it. (most of the time) > > One time I got an crash dump, and tried to investigate it with cdv, but I > didn't get much informations about which process or module was eating up > all the memory. > I just know that it crashed because of the crash message : "eheap_alloc: > Cannot allocate 6801972448 bytes of memory (of type "heap")." > > I'm surely too new to Erlang to investigate something like this with cdv, > I really would like some pointers on how I can understand this problem and > fix it asap. > > If you need any informations about the crash dump, let me know what you > need, I'll copy/paste? > > I'm using Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] > [async-threads:10] [kernel-poll:true] > > Thank you all for your help ! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 04:41:50 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 04:41:50 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: Message-ID: <3839B533-728C-4BB9-A5FF-3A39C2EA96E2@gmail.com> Hi, Generally, when a module is critical, and a lot solicited, I create a pg2 "pool" of supervised gen_server that will join the group, and get the pid using get_closest_pid in order to have multiple process. Further more, The server has 16GB of RAM, and when the server is starting to get crazy, it only has 1,5 GB tops of RAM used. it really needs to go crazy a long time before using swap I guess, but I don't see it until another node cluster is telling me that the freeze node is timeout. However, since we are at it? I may have find something really weird looking at my crash dump. I'm using the emysql application. My initialization of the emysql application is pretty basic : application:start(emysql), emysql:add_pool(my_db, 30, "login", "password", "my.db-host.com", 3306, "table", latin1) Has you can see, I only have 30 connections asked in the pool. However in the crash dump here's what I have found in the fun table : Module Uniq Index Address Native_address Refc emysql_util 8432855 1 0x00007f1d4f9f6f00 3476 emysql_util 8432855 0 0x00007f1d4f9f7218 3476 emysql_util 8432855 3 0x00007f1d4f9f6e48 2 emysql_util 8432855 2 0x00007f1d4f9f6ea8 1 emysql 79898780 0 0x00007f1d4f9b56f8 841 Is that something normal to have with only 30 connections in one pool ? Thank you all. Le 23 mai 2013 ? 04:21, Bob Ippolito a ?crit : > This kind of thing tends to happen when you continuously send messages to a process faster than it can handle them. The most common case that I've seen this is where you have a lot of processes communicating with a single gen_server process. If your server has swap enabled, this may appear to make the node "freeze completely but not crash". > > In the past I've diagnosed this by monitoring the message_queue_len of registered processes, but I'm sure there are tools that can help do this for you. > > > On Wed, May 22, 2013 at 7:00 PM, Morgan Segalis wrote: > Hello everyone, > > I'm having a bit of an issue with my production servers. > > At some point, it seems to enter into an infinite loop that I can't find, or reproduce by myself on the tests servers. > > The bug appear completely random, 1 hour, or 10 hour after restarting the Erlang node. > The loop will eat up all my server's memory in no time, and freeze completely the Erlang node without crashing it. (most of the time) > > One time I got an crash dump, and tried to investigate it with cdv, but I didn't get much informations about which process or module was eating up all the memory. > I just know that it crashed because of the crash message : "eheap_alloc: Cannot allocate 6801972448 bytes of memory (of type "heap")." > > I'm surely too new to Erlang to investigate something like this with cdv, I really would like some pointers on how I can understand this problem and fix it asap. > > If you need any informations about the crash dump, let me know what you need, I'll copy/paste? > > I'm using Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:true] > > Thank you all for your help ! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 05:26:03 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 05:26:03 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <77D42475-EB25-4125-8775-08D153BF0F2C@gmail.com> References: <77D42475-EB25-4125-8775-08D153BF0F2C@gmail.com> Message-ID: <624FD540-FF7C-41FE-9A7E-23B62E2D7C6B@gmail.com> Hi, Unfortunately, I'm not using RabbitMQ. as for the garbage collection, Since the server is having 16GB of RAM, and the erlang node is using 1,5 GB when everything works fine, garbage collection should not be here an issue? Le 23 mai 2013 ? 05:14, Yogish Baliga a ?crit : > I saw this message in our RabbitMQ server. Investigation lead me to the garbage collection. It happened only once. After restart everything seems to be fine. > > > > On May 22, 2013, at 19:00, Morgan Segalis wrote: > >> Hello everyone, >> >> I'm having a bit of an issue with my production servers. >> >> At some point, it seems to enter into an infinite loop that I can't find, or reproduce by myself on the tests servers. >> >> The bug appear completely random, 1 hour, or 10 hour after restarting the Erlang node. >> The loop will eat up all my server's memory in no time, and freeze completely the Erlang node without crashing it. (most of the time) >> >> One time I got an crash dump, and tried to investigate it with cdv, but I didn't get much informations about which process or module was eating up all the memory. >> I just know that it crashed because of the crash message : "eheap_alloc: Cannot allocate 6801972448 bytes of memory (of type "heap")." >> >> I'm surely too new to Erlang to investigate something like this with cdv, I really would like some pointers on how I can understand this problem and fix it asap. >> >> If you need any informations about the crash dump, let me know what you need, I'll copy/paste? >> >> I'm using Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:true] >> >> Thank you all for your help ! >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu May 23 07:13:44 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 23 May 2013 08:13:44 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: Message-ID: Hello, I would aree with Bob about most probable root cause. You can use entop to check message queue length and used memory per-process. Best Regards, Dmitry >-|-|-*> On 23.5.2013, at 5.21, Bob Ippolito wrote: > This kind of thing tends to happen when you continuously send messages to a process faster than it can handle them. The most common case that I've seen this is where you have a lot of processes communicating with a single gen_server process. If your server has swap enabled, this may appear to make the node "freeze completely but not crash". > > In the past I've diagnosed this by monitoring the message_queue_len of registered processes, but I'm sure there are tools that can help do this for you. > > > On Wed, May 22, 2013 at 7:00 PM, Morgan Segalis wrote: >> Hello everyone, >> >> I'm having a bit of an issue with my production servers. >> >> At some point, it seems to enter into an infinite loop that I can't find, or reproduce by myself on the tests servers. >> >> The bug appear completely random, 1 hour, or 10 hour after restarting the Erlang node. >> The loop will eat up all my server's memory in no time, and freeze completely the Erlang node without crashing it. (most of the time) >> >> One time I got an crash dump, and tried to investigate it with cdv, but I didn't get much informations about which process or module was eating up all the memory. >> I just know that it crashed because of the crash message : "eheap_alloc: Cannot allocate 6801972448 bytes of memory (of type "heap")." >> >> I'm surely too new to Erlang to investigate something like this with cdv, I really would like some pointers on how I can understand this problem and fix it asap. >> >> If you need any informations about the crash dump, let me know what you need, I'll copy/paste? >> >> I'm using Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:true] >> >> Thank you all for your help ! >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Thu May 23 07:38:34 2013 From: vances@REDACTED (Vance Shipley) Date: Thu, 23 May 2013 11:08:34 +0530 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: Message-ID: <20130523053834.GC428@aluminium.local> On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: } I'm having a bit of an issue with my production servers. You will find that etop is your friend: http://www.erlang.org/doc/apps/observer/etop_ug.html Run etop from the command line and sort on the column you're interested in. To watch memory usage: etop -node tiger@REDACTED -sort memory This will list the processes by memory size in decreasing order. This shows you the memory hogs. Watch it as it starts to get into trouble and you should see where the memory is getting used. As Bob points out the most common problem is that a process's inbox will start to fill up. Once this starts happening it's the beginning of the end. Another process may start eating up memory and the node may crash because it has requested more than is available bt the root cause was that one process not having time to service the messages at the rate they are received. To watch for message queue lengths: etop -node tiger@REDACTED -sort msg_q The above will list the processes in decreasing order of inbox size. They should all be zero, and sometimes one, normally. If you have a problem you'll see one process stay at the top and it's message queue length will start to grow over time. -- -Vance From jon@REDACTED Thu May 23 11:02:43 2013 From: jon@REDACTED (Jon Schneider) Date: Thu, 23 May 2013 10:02:43 +0100 Subject: [erlang-questions] Observer processes tab does not make sense Message-ID: <16879d5ed6e332b3fcdff0836165d3aa.squirrel@mail.jschneider.net> I'm looking at the output of i() and observer's processes tab and am really struggling to make sense especially of the latter. Most processes have zero Reds if I believe observer. Then a couple of seconds later they don't. Or is it telling me Reds since I started looking ? i() output certainly makes more sense. Do they use different units for heap and stack ? Should I see anything that nearly agrees (taking into account the fact that many processes are always changing) ? Or is it that there is room for better documentation (otherwise this tool seems somewhat untrustworthy) ? This is emulator 5.9.3.1 on XP. Jon From garazdawi@REDACTED Thu May 23 11:13:15 2013 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 23 May 2013 11:13:15 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: <20130523085656.GT15261@debjann.fritz.box> References: <20130522110928.GP15261@debjann.fritz.box> <20130522135638.GQ15261@debjann.fritz.box> <20130523085656.GT15261@debjann.fritz.box> Message-ID: On Thu, May 23, 2013 at 10:56 AM, Jann Horn wrote: > On Wed, May 22, 2013 at 06:42:59PM +0200, Lukas Larsson wrote: > > You have to poll like that because fd's are not the only thing which can > > trigger load on the system. Timeouts for instance are triggered by > calling > > gettimeofday which means you have to break out of epoll_wait before the > > next timeout happens. > > Why not by specifying a timeout in the epoll_wait call? > If you look closely to the strace you can see that for every n:th call there is a small timeout given to epoll_wait. This timeout is calculated by looking at the next timeout and a number of other factors. > > > Also the spinning is done to make the system respond > > faster to events by delaying sleeping in the kernel. > > Ah, ok... and that brings a performance gain? > It brings a latency gain, which in turn can bring a performance gain. Sleeping in the kernel is a (relatively) expensive thing to do and by spinning before sleeping the schedulers stay more responsive. You can configure this behaviour through the runtime flags +sbwt, +sws, +swt. See http://www.erlang.org/doc/man/erl.html for some more details. > > > > On Wed, May 22, 2013 at 3:56 PM, Jann Horn wrote: > > > > > On Wed, May 22, 2013 at 03:46:17PM +0200, Benoit Chesneau wrote: > > > > How would it work to receive an event to wake up if doesn't listen on > > > them? > > > > > > Uh... register all fds you're interested in using epoll, then do > > > epoll_wait(fd, events, maxevents, -1)? Isn't that actually quite > > > normal? The syscall will return as soon as something happens, but > > > not earlier. > > > > > > You don't have to poll different event sources, there > > > are many facilities that can multiplex those event sources for > > > you and will wake you up as soon as something interesting happens. > > > E.g. select, poll, epoll, kqueue and event ports (the last two aren't > > > available on linux). I can't imagine a reason why you'd have to > > > poll events like this in any OS. > > > > > > > > > > On Wed, May 22, 2013 at 1:09 PM, Jann Horn wrote: > > > > > This is strace output from a totally idle couchdb process: > > > > > > > > > > [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 > > > > > [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 > > > > > [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 > > > > > [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 > > > > > [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 > > > > > [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 > > > EAGAIN (Resource temporarily unavailable) > > > > > [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, > {EPOLLIN, > > > {u32=10, u64=73199780460757002}}) = 0 > > > > > [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, > {EPOLLIN, > > > {u32=10, u64=73199780460757002}}) = 0 > > > > > [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 > > > > > [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 > > > > > > > > > > This seems to be, at least partly, intentional ? > > > erts/emulator/beam/erl_process.c > > > > > contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is > > > set to 10. > > > > > > > > > > Can anyone tell me what the rationale behind this excessive > > > busylooping is? A few > > > > > dozen syscalls per second for nothing seems a bit weird to me. > > > > > > > > > > _______________________________________________ > > > > > erlang-questions mailing list > > > > > erlang-questions@REDACTED > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 11:13:19 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 11:13:19 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <20130523053834.GC428@aluminium.local> References: <20130523053834.GC428@aluminium.local> Message-ID: <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> I have launch the etop on my computer monitoring the production server? hoping that I will see something wrong ! Thank you for your help so far (to All). I'll come back to you as soon as I have more information with etop. Morgan. Le 23 mai 2013 ? 07:38, Vance Shipley a ?crit : > On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: > } I'm having a bit of an issue with my production servers. > > You will find that etop is your friend: > > http://www.erlang.org/doc/apps/observer/etop_ug.html > > Run etop from the command line and sort on the column you're > interested in. To watch memory usage: > > etop -node tiger@REDACTED -sort memory > > This will list the processes by memory size in decreasing order. > This shows you the memory hogs. Watch it as it starts to get > into trouble and you should see where the memory is getting used. > > As Bob points out the most common problem is that a process's > inbox will start to fill up. Once this starts happening it's > the beginning of the end. Another process may start eating up > memory and the node may crash because it has requested more than > is available bt the root cause was that one process not having > time to service the messages at the rate they are received. > > To watch for message queue lengths: > > etop -node tiger@REDACTED -sort msg_q > > The above will list the processes in decreasing order of inbox > size. They should all be zero, and sometimes one, normally. If > you have a problem you'll see one process stay at the top and it's > message queue length will start to grow over time. > > -- > -Vance From msegalis@REDACTED Thu May 23 11:23:04 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 11:23:04 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> Message-ID: <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> Apparently I'm monitoring my own node? Does someone know how to monitor a external cluster node with etop ? Le 23 mai 2013 ? 11:13, Morgan Segalis a ?crit : > I have launch the etop on my computer monitoring the production server? hoping that I will see something wrong ! > > Thank you for your help so far (to All). > > I'll come back to you as soon as I have more information with etop. > > Morgan. > > Le 23 mai 2013 ? 07:38, Vance Shipley a ?crit : > >> On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: >> } I'm having a bit of an issue with my production servers. >> >> You will find that etop is your friend: >> >> http://www.erlang.org/doc/apps/observer/etop_ug.html >> >> Run etop from the command line and sort on the column you're >> interested in. To watch memory usage: >> >> etop -node tiger@REDACTED -sort memory >> >> This will list the processes by memory size in decreasing order. >> This shows you the memory hogs. Watch it as it starts to get >> into trouble and you should see where the memory is getting used. >> >> As Bob points out the most common problem is that a process's >> inbox will start to fill up. Once this starts happening it's >> the beginning of the end. Another process may start eating up >> memory and the node may crash because it has requested more than >> is available bt the root cause was that one process not having >> time to service the messages at the rate they are received. >> >> To watch for message queue lengths: >> >> etop -node tiger@REDACTED -sort msg_q >> >> The above will list the processes in decreasing order of inbox >> size. They should all be zero, and sometimes one, normally. If >> you have a problem you'll see one process stay at the top and it's >> message queue length will start to grow over time. >> >> -- >> -Vance > From msegalis@REDACTED Thu May 23 11:35:45 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 11:35:45 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> Message-ID: <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> Nevermind I got it? However I do not get a lot of information? most of process is proc_lib:ini_p/5 Le 23 mai 2013 ? 11:23, Morgan Segalis a ?crit : > Apparently I'm monitoring my own node? > > Does someone know how to monitor a external cluster node with etop ? > > Le 23 mai 2013 ? 11:13, Morgan Segalis a ?crit : > >> I have launch the etop on my computer monitoring the production server? hoping that I will see something wrong ! >> >> Thank you for your help so far (to All). >> >> I'll come back to you as soon as I have more information with etop. >> >> Morgan. >> >> Le 23 mai 2013 ? 07:38, Vance Shipley a ?crit : >> >>> On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: >>> } I'm having a bit of an issue with my production servers. >>> >>> You will find that etop is your friend: >>> >>> http://www.erlang.org/doc/apps/observer/etop_ug.html >>> >>> Run etop from the command line and sort on the column you're >>> interested in. To watch memory usage: >>> >>> etop -node tiger@REDACTED -sort memory >>> >>> This will list the processes by memory size in decreasing order. >>> This shows you the memory hogs. Watch it as it starts to get >>> into trouble and you should see where the memory is getting used. >>> >>> As Bob points out the most common problem is that a process's >>> inbox will start to fill up. Once this starts happening it's >>> the beginning of the end. Another process may start eating up >>> memory and the node may crash because it has requested more than >>> is available bt the root cause was that one process not having >>> time to service the messages at the rate they are received. >>> >>> To watch for message queue lengths: >>> >>> etop -node tiger@REDACTED -sort msg_q >>> >>> The above will list the processes in decreasing order of inbox >>> size. They should all be zero, and sometimes one, normally. If >>> you have a problem you'll see one process stay at the top and it's >>> message queue length will start to grow over time. >>> >>> -- >>> -Vance >> > From bog495@REDACTED Thu May 23 11:46:15 2013 From: bog495@REDACTED (Bogdan Andu) Date: Thu, 23 May 2013 12:46:15 +0300 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> Message-ID: I have applied the patches: cd $OTP_SRC_DIR_R14B04 patch -p0 < /usr/ports/lang/erlang/patches/patch-erts_configure_in patch -p0 < /usr/ports/lang/erlang/patches/patch-erts_emulator_Makefile_in patch -p0 < /usr/ports/lang/erlang/patches/patch-erts_lib_src_Makefile_in patch -p0 < /usr/ports/lang/erlang/patches/patch-lib_tools_c_src_Makefile_in patch -p0 < /usr/ports/lang/erlang/patches/patch-make_target_mk *all patches succedded, and then: ./configure sudo gmake install the same error I modified erts/configure.in: from: openbsd*) DED_LD="$CC" DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" DED_LDFLAGS="-shared" ;; in: openbsd*) DED_LD="$CC" DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" DED_LDFLAGS="-shared /usr/lib/crtbeginS.o /usr/lib/crtbegin.o" ;; because: $ nm /usr/lib/crtbegin.o U _GLOBAL_OFFSET_TABLE_ U _Jv_RegisterClasses 00000000 D __dso_handle 00000000 T __fini 00000000 D __guard_local 00000000 T __init 00000000 W __register_frame_info U atexit $ nm /usr/lib/crtbeginS.o U _GLOBAL_OFFSET_TABLE_ U _Jv_RegisterClasses U __cxa_finalize 00000000 D __dso_handle 00000000 D __guard_local 00000000 T _fini 00000000 T _init gmake clean ./configure sudo gmake install the same error Does anyone has an idea of how to solve this issue, because I really need R14B04. Please help. I am really out of ideas. Maybe I should link against other libraries but I do not know what they are. How can I dicover these libraries that I should link erlang against? Bogdan On Wed, May 22, 2013 at 12:23 PM, Bogdan Andu wrote: > Thank you for the pointers. > > I'll give it a try to see what happens. > > It seems that linking erlang against a previous, working ssl library would > give the same error. > > Bogdan > > > On Wed, May 22, 2013 at 11:46 AM, Stanislav Sedov wrote: > >> >> On May 22, 2013, at 12:34 AM, Bogdan Andu wrote: >> >> > The os is OpenBSD 5.3 amd64 with preinstalled opensssl library >> /usr/lib/libssl.so.19.0 >> > >> > $ /usr/sbin/openssl >> > OpenSSL> version >> > OpenSSL 1.0.1c 10 May 2012 >> > OpenSSL> >> > ^D >> > >> > This is the only ssl library in the system, and erlang was configured >> with this ssl library >> > and no complaints at ./configure time. >> > >> > On the other hand: >> > >> > On OpenBSD 5.2 amd64 with preinstalled opensssl library >> /usr/lib/libssl.so.18.0 >> > >> > $ /usr/sbin/openssl >> > OpenSSL> version >> > OpenSSL 1.0.0f 4 Jan 2012 >> > OpenSSL> >> > ^D >> > >> > everything works perfect, and Erlang was also configured againsat this >> preinstalled library >> > >> >> I found this message on the mailing list, which seems to be relevant: >> >> http://www.mail-archive.com/cdesktopenv-devel@REDACTED/msg00598.html >> >> It looks like the __guard_local symbol is generated by new >> stack_protector code, >> and according to the patch it seems that linking against libcrtbeginS.o >> and >> libcrtendS.o fixes the issue for CDE. It might be worth trying to modify >> the >> erlang linker flags (or at least crypto linking flags) to link against >> these >> objects. >> >> However, I don't really understand what's going on here. According to >> [1], >> the new __guard_local symbol should be present in crtbegin.o. But maybe >> crypto.so >> is being linked manually by erlang build system (i.e. not via cc(1)), in >> which case >> applying the same fix as was provided for CDE might help. >> >> [1]: >> http://openbsd.7691.n7.nabble.com/GCC-diff-needs-testing-on-multiple-arches-td167891.html >> >> -- >> ST4096-RIPE >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu May 23 11:50:11 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 23 May 2013 12:50:11 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> Message-ID: <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> which means that you are using proc_lib heavily... Are those top process with reductions, message queue size or heap? Try to connect into node and gather more info about those processes using erlang:process_info(?) or sys:get_status(?) - Dmitry On May 23, 2013, at 12:35 PM, Morgan Segalis wrote: > Nevermind I got it? > > However I do not get a lot of information? > > most of process is proc_lib:ini_p/5 > > Le 23 mai 2013 ? 11:23, Morgan Segalis a ?crit : > >> Apparently I'm monitoring my own node? >> >> Does someone know how to monitor a external cluster node with etop ? >> >> Le 23 mai 2013 ? 11:13, Morgan Segalis a ?crit : >> >>> I have launch the etop on my computer monitoring the production server? hoping that I will see something wrong ! >>> >>> Thank you for your help so far (to All). >>> >>> I'll come back to you as soon as I have more information with etop. >>> >>> Morgan. >>> >>> Le 23 mai 2013 ? 07:38, Vance Shipley a ?crit : >>> >>>> On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: >>>> } I'm having a bit of an issue with my production servers. >>>> >>>> You will find that etop is your friend: >>>> >>>> http://www.erlang.org/doc/apps/observer/etop_ug.html >>>> >>>> Run etop from the command line and sort on the column you're >>>> interested in. To watch memory usage: >>>> >>>> etop -node tiger@REDACTED -sort memory >>>> >>>> This will list the processes by memory size in decreasing order. >>>> This shows you the memory hogs. Watch it as it starts to get >>>> into trouble and you should see where the memory is getting used. >>>> >>>> As Bob points out the most common problem is that a process's >>>> inbox will start to fill up. Once this starts happening it's >>>> the beginning of the end. Another process may start eating up >>>> memory and the node may crash because it has requested more than >>>> is available bt the root cause was that one process not having >>>> time to service the messages at the rate they are received. >>>> >>>> To watch for message queue lengths: >>>> >>>> etop -node tiger@REDACTED -sort msg_q >>>> >>>> The above will list the processes in decreasing order of inbox >>>> size. They should all be zero, and sometimes one, normally. If >>>> you have a problem you'll see one process stay at the top and it's >>>> message queue length will start to grow over time. >>>> >>>> -- >>>> -Vance >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dangud@REDACTED Thu May 23 11:50:54 2013 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 23 May 2013 11:50:54 +0200 Subject: [erlang-questions] Observer processes tab does not make sense In-Reply-To: <16879d5ed6e332b3fcdff0836165d3aa.squirrel@mail.jschneider.net> References: <16879d5ed6e332b3fcdff0836165d3aa.squirrel@mail.jschneider.net> Message-ID: Observer reports (by default) the reductions since last update, you can change that in the View menu, with accumulate. And you can change the refresh rate. The memory comes from process_info(Pid, memory) see doc on that. /Dan On Thu, May 23, 2013 at 11:02 AM, Jon Schneider wrote: > I'm looking at the output of i() and observer's processes tab and am > really struggling to make sense especially of the latter. > > Most processes have zero Reds if I believe observer. Then a couple of > seconds later they don't. Or is it telling me Reds since I started looking > ? i() output certainly makes more sense. > > Do they use different units for heap and stack ? > > Should I see anything that nearly agrees (taking into account the fact > that many processes are always changing) ? > > Or is it that there is room for better documentation (otherwise this tool > seems somewhat untrustworthy) ? > > This is emulator 5.9.3.1 on XP. > > Jon > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jann@REDACTED Thu May 23 10:56:56 2013 From: jann@REDACTED (Jann Horn) Date: Thu, 23 May 2013 10:56:56 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: References: <20130522110928.GP15261@debjann.fritz.box> <20130522135638.GQ15261@debjann.fritz.box> Message-ID: <20130523085656.GT15261@debjann.fritz.box> On Wed, May 22, 2013 at 06:42:59PM +0200, Lukas Larsson wrote: > You have to poll like that because fd's are not the only thing which can > trigger load on the system. Timeouts for instance are triggered by calling > gettimeofday which means you have to break out of epoll_wait before the > next timeout happens. Why not by specifying a timeout in the epoll_wait call? > Also the spinning is done to make the system respond > faster to events by delaying sleeping in the kernel. Ah, ok... and that brings a performance gain? > On Wed, May 22, 2013 at 3:56 PM, Jann Horn wrote: > > > On Wed, May 22, 2013 at 03:46:17PM +0200, Benoit Chesneau wrote: > > > How would it work to receive an event to wake up if doesn't listen on > > them? > > > > Uh... register all fds you're interested in using epoll, then do > > epoll_wait(fd, events, maxevents, -1)? Isn't that actually quite > > normal? The syscall will return as soon as something happens, but > > not earlier. > > > > You don't have to poll different event sources, there > > are many facilities that can multiplex those event sources for > > you and will wake you up as soon as something interesting happens. > > E.g. select, poll, epoll, kqueue and event ports (the last two aren't > > available on linux). I can't imagine a reason why you'd have to > > poll events like this in any OS. > > > > > > > On Wed, May 22, 2013 at 1:09 PM, Jann Horn wrote: > > > > This is strace output from a totally idle couchdb process: > > > > > > > > [pid 18350] 17:40:15.086577 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086610 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086643 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086675 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086736 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086771 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.086940 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.087180 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.087219 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.087251 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.087287 epoll_wait(3, {}, 256, 228) = 0 > > > > [pid 18350] 17:40:15.315646 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315718 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315751 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315800 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315833 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315865 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315900 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315953 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.315992 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.316081 epoll_wait(3, {}, 256, 651) = 0 > > > > [pid 18350] 17:40:15.967979 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.969211 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.969926 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970484 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970616 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970717 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970806 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970895 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.970983 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.971071 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.971158 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.971243 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:15.971343 epoll_wait(3, {}, 256, 345) = 0 > > > > [pid 18350] 17:40:16.316908 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.317266 epoll_wait(3, {}, 256, 650) = 0 > > > > [pid 18350] 17:40:16.969333 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.970057 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.970580 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.970736 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.970827 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.970917 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971011 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971118 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971206 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971300 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971386 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:16.971485 epoll_wait(3, {}, 256, 115) = 0 > > > > [pid 18350] 17:40:17.087042 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.087421 accept(10, 0x7fb208dbfba0, [28]) = -1 > > EAGAIN (Resource temporarily unavailable) > > > > [pid 18350] 17:40:17.087915 epoll_ctl(3, EPOLL_CTL_DEL, 10, {EPOLLIN, > > {u32=10, u64=73199780460757002}}) = 0 > > > > [pid 18350] 17:40:17.088035 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN, > > {u32=10, u64=73199780460757002}}) = 0 > > > > [pid 18350] 17:40:17.088139 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088231 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088327 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088477 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088565 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088660 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088746 epoll_wait(3, {}, 256, 0) = 0 > > > > [pid 18350] 17:40:17.088833 epoll_wait(3, {}, 256, 0) = 0 > > > > > > > > This seems to be, at least partly, intentional ? > > erts/emulator/beam/erl_process.c > > > > contains a constant named "ERTS_SCHED_SYS_SLEEP_SPINCOUNT" which is > > set to 10. > > > > > > > > Can anyone tell me what the rationale behind this excessive > > busylooping is? A few > > > > dozen syscalls per second for nothing seems a bit weird to me. > > > > > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From msegalis@REDACTED Thu May 23 12:04:11 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 12:04:11 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> Message-ID: <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> For more information here's what my erlang node is doing : It is an instant messaging server, each client connected is a process spawn automatically by a supervisor? Every process spawned is monitored and started by a supervisor? I have made a little function a while back, getting all processes and removing the processes inited at the beginning? Here's what it gives me when everything works fine : Dict: {dict,16,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], [], [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], [], [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], [undefined|4]], [], [[{{supervisor,connector,1},[<0.42.0>]}|1], [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], [],[],[], [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], [{{ssl_connection,init,1}, [ssl_connection_sup,ssl_sup,<0.51.0>]}| 1366], [{unknown,unknown}|3]], [],[], [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], [], [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} ok Le 23 mai 2013 ? 11:50, Dmitry Kolesnikov a ?crit : > which means that you are using proc_lib heavily... > Are those top process with reductions, message queue size or heap? > > Try to connect into node and gather more info about those processes using > erlang:process_info(?) or sys:get_status(?) > > - Dmitry > > On May 23, 2013, at 12:35 PM, Morgan Segalis wrote: > >> Nevermind I got it? >> >> However I do not get a lot of information? >> >> most of process is proc_lib:ini_p/5 >> >> Le 23 mai 2013 ? 11:23, Morgan Segalis a ?crit : >> >>> Apparently I'm monitoring my own node? >>> >>> Does someone know how to monitor a external cluster node with etop ? >>> >>> Le 23 mai 2013 ? 11:13, Morgan Segalis a ?crit : >>> >>>> I have launch the etop on my computer monitoring the production server? hoping that I will see something wrong ! >>>> >>>> Thank you for your help so far (to All). >>>> >>>> I'll come back to you as soon as I have more information with etop. >>>> >>>> Morgan. >>>> >>>> Le 23 mai 2013 ? 07:38, Vance Shipley a ?crit : >>>> >>>>> On Thu, May 23, 2013 at 04:00:07AM +0200, Morgan Segalis wrote: >>>>> } I'm having a bit of an issue with my production servers. >>>>> >>>>> You will find that etop is your friend: >>>>> >>>>> http://www.erlang.org/doc/apps/observer/etop_ug.html >>>>> >>>>> Run etop from the command line and sort on the column you're >>>>> interested in. To watch memory usage: >>>>> >>>>> etop -node tiger@REDACTED -sort memory >>>>> >>>>> This will list the processes by memory size in decreasing order. >>>>> This shows you the memory hogs. Watch it as it starts to get >>>>> into trouble and you should see where the memory is getting used. >>>>> >>>>> As Bob points out the most common problem is that a process's >>>>> inbox will start to fill up. Once this starts happening it's >>>>> the beginning of the end. Another process may start eating up >>>>> memory and the node may crash because it has requested more than >>>>> is available bt the root cause was that one process not having >>>>> time to service the messages at the rate they are received. >>>>> >>>>> To watch for message queue lengths: >>>>> >>>>> etop -node tiger@REDACTED -sort msg_q >>>>> >>>>> The above will list the processes in decreasing order of inbox >>>>> size. They should all be zero, and sometimes one, normally. If >>>>> you have a problem you'll see one process stay at the top and it's >>>>> message queue length will start to grow over time. >>>>> >>>>> -- >>>>> -Vance >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From msegalis@REDACTED Thu May 23 12:24:47 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 12:24:47 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> Message-ID: <9628A348-B7A3-4D10-98B0-DCBC03594858@gmail.com> When a client is connecting, yes, I'm using start_child. The supervising strategy is : simple_one_for_one, temporary, worker Morgan. Le 23 mai 2013 ? 12:13, Dmitry Kolesnikov a ?crit : > > On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: > >> Every process spawned is monitored and started by a supervisor? > > Do you use start_child to spawn a new process? If so, do you clean it up? > What is supervising strategy? > > - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu May 23 14:19:40 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 23 May 2013 15:19:40 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> Message-ID: On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: > I have made a little function a while back, getting all processes and removing the processes inited at the beginning? Could you please elaborate on that? Why you are not satisfied with supervisor? - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 14:25:22 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 14:25:22 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> Message-ID: <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : Dict: {dict,16,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], [], [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], [], [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], [undefined|4]], [], [[{{supervisor,connector,1},[<0.42.0>]}|1], [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], [],[],[], [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], [{{ssl_connection,init,1}, [ssl_connection_sup,ssl_sup,<0.51.0>]}| 1366], [{unknown,unknown}|3]], [],[], [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], [], [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} ok I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : > > On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: > >> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? > > Could you please elaborate on that? Why you are not satisfied with supervisor? > > - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu May 23 14:39:06 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 23 May 2013 15:39:06 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> Message-ID: <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> Right, you do not have many processes. Same time you goes out of memory? Unfortunately, I had no time play around with R16B at production? Could it be some issue with SSL, I re-call there was some complains in the list? I would use entop to spot the process that has either too much reductions, queue len or heap. Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) BTW, What erlang:memory() says on you production node? - Dmitry On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: > No, I was talking about the function I made to investigate which processes I have created, which gives me this output : > > Dict: {dict,16,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], > [], > [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], > [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], > [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], > [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], > [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], > [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], > [], > [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], > [undefined|4]], > [], > [[{{supervisor,connector,1},[<0.42.0>]}|1], > [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], > [],[],[], > [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], > [{{ssl_connection,init,1}, > [ssl_connection_sup,ssl_sup,<0.51.0>]}| > 1366], > [{unknown,unknown}|3]], > [],[], > [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], > [], > [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} > ok > > I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... > > Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : > >> >> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >> >>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >> >> Could you please elaborate on that? Why you are not satisfied with supervisor? >> >> - Dmitry > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 14:46:08 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 14:46:08 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> Message-ID: <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> So I should go back to R15B ? erlang:memory() gives me [{total,1525779584}, {processes,1272881427}, {processes_used,1272789743}, {system,252898157}, {atom,372217}, {atom_used,346096}, {binary,148093608}, {code,8274446}, {ets,1546832}] But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : > Right, you do not have many processes. Same time you goes out of memory? > > Unfortunately, I had no time play around with R16B at production? > Could it be some issue with SSL, I re-call there was some complains in the list? > > I would use entop to spot the process that has either too much reductions, queue len or heap. > Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) > > BTW, What erlang:memory() says on you production node? > > - Dmitry > > On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: > >> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >> >> Dict: {dict,16,16,16,8,80,48, >> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >> [], >> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >> [], >> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >> [undefined|4]], >> [], >> [[{{supervisor,connector,1},[<0.42.0>]}|1], >> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >> [],[],[], >> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >> [{{ssl_connection,init,1}, >> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >> 1366], >> [{unknown,unknown}|3]], >> [],[], >> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >> [], >> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >> ok >> >> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >> >> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >> >>> >>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>> >>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>> >>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>> >>> - Dmitry >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Thu May 23 16:30:23 2013 From: vances@REDACTED (Vance Shipley) Date: Thu, 23 May 2013 20:00:23 +0530 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> Message-ID: Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: > So I should go back to R15B ? > > erlang:memory() gives me > > [{total,1525779584}, > {processes,1272881427}, > {processes_used,1272789743}, > {system,252898157}, > {atom,372217}, > {atom_used,346096}, > {binary,148093608}, > {code,8274446}, > {ets,1546832}] > > > But keep in mind that right now, there is no infinite loop, or memory > issue at this exact time? > It will be more interesting to have that when the VM is asking for 14GB of > memory, but when it does, the console is unresponsive, so I can't get > anything then. > > Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a > ?crit : > > Right, you do not have many processes. Same time you goes out of memory? > > Unfortunately, I had no time play around with R16B at production? > Could it be some issue with SSL, I re-call there was some complains in the > list? > > I would use entop to spot the process that has either too much reductions, > queue len or heap. > Once you know they pid you can dig more info about them using > erlang:process_info(?) and/or sys:get:status(?) > > BTW, What erlang:memory() says on you production node? > > - Dmitry > > On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: > > No, I was talking about the function I made to investigate which processes > I have created, which gives me this output : > > Dict: {dict,16,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > > {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], > [], > [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], > > [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], > [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], > [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], > > [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], > [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], > [], > > [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], > [undefined|4]], > [], > [[{{supervisor,connector,1},[<0.42.0>]}|1], > > [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], > [],[],[], > > [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], > [{{ssl_connection,init,1}, > [ssl_connection_sup,ssl_sup,<0.51.0>]}| > 1366], > [{unknown,unknown}|3]], > [],[], > > [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], > [], > [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} > ok > > I'm very satisfied with supervisor, and I don't think to have the > expertise tweaking it... > > Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a > ?crit : > > > On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: > > I have made a little function a while back, getting all processes and > removing the processes inited at the beginning? > > > Could you please elaborate on that? Why you are not satisfied with > supervisor? > > - Dmitry > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 16:34:24 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 16:34:24 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> Message-ID: <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! I get back to the Erlang list as soon as I have more informations about this. Thank you all ! Morgan. Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : > Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. > On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: > So I should go back to R15B ? > > erlang:memory() gives me > > [{total,1525779584}, > {processes,1272881427}, > {processes_used,1272789743}, > {system,252898157}, > {atom,372217}, > {atom_used,346096}, > {binary,148093608}, > {code,8274446}, > {ets,1546832}] > > > But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? > It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. > > Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : > >> Right, you do not have many processes. Same time you goes out of memory? >> >> Unfortunately, I had no time play around with R16B at production? >> Could it be some issue with SSL, I re-call there was some complains in the list? >> >> I would use entop to spot the process that has either too much reductions, queue len or heap. >> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >> >> BTW, What erlang:memory() says on you production node? >> >> - Dmitry >> >> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >> >>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>> >>> Dict: {dict,16,16,16,8,80,48, >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>> [], >>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>> [], >>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>> [undefined|4]], >>> [], >>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>> [],[],[], >>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>> [{{ssl_connection,init,1}, >>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>> 1366], >>> [{unknown,unknown}|3]], >>> [],[], >>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>> [], >>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>> ok >>> >>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>> >>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>> >>>> >>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>> >>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>> >>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>> >>>> - Dmitry >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu May 23 22:25:55 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 23 May 2013 13:25:55 -0700 (PDT) Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <45099.1366745814@snookles.snookles.com> References: <45099.1366745814@snookles.snookles.com> Message-ID: <3cb7dde5-cd5e-4a7e-861d-e1ab68e2baee@googlegroups.com> I'm deploying some of erlang servers with capistrano, so I had to add: default_run_options[:pty] = true in deploy.rb to enable everything working as usual. It really looks like a black magic for me: why does erlang behaviour differs if it is ssh or ssh -t =( -------------- next part -------------- An HTML attachment was scrubbed... URL: From stas@REDACTED Thu May 23 22:50:39 2013 From: stas@REDACTED (Stanislav Sedov) Date: Thu, 23 May 2013 13:50:39 -0700 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> Message-ID: <88DD116E-E734-462C-87A6-42D8A69C9E4F@freebsd.org> On May 23, 2013, at 2:46 AM, Bogdan Andu wrote: > I have applied the patches: > cd $OTP_SRC_DIR_R14B04 > patch -p0 < > /usr/ports/lang/erlang/patches/patch-erts_configure_in > patch -p0 < > /usr/ports/lang/erlang/patches/patch-erts_emulator_Makefile_in > patch -p0 < > /usr/ports/lang/erlang/patches/patch-erts_lib_src_Makefile_in > patch -p0 < > /usr/ports/lang/erlang/patches/patch-lib_tools_c_src_Makefile_in > patch -p0 < > /usr/ports/lang/erlang/patches/patch-make_target_mk > > *all patches succedded, > and then: > > ./configure > sudo gmake install > the same error > > I modified > erts/configure.in: > from: > openbsd*) > DED_LD="$CC" > DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" > DED_LDFLAGS="-shared" > ;; > in: > openbsd*) > > DED_LD="$CC" > > DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" > > DED_LDFLAGS="-shared /usr/lib/crtbeginS.o /usr/lib/crtbegin.o" > ;; > because: > $ nm /usr/lib/crtbegin.o > U _GLOBAL_OFFSET_TABLE_ > U > _Jv_RegisterClasses > 00000000 D __dso_handle > 00000000 T __fini > 00000000 D > __guard_local > 00000000 T __init > 00000000 W __register_frame_info > U > atexit > $ nm /usr/lib/crtbeginS.o > U _GLOBAL_OFFSET_TABLE_ > U > _Jv_RegisterClasses > U __cxa_finalize > 00000000 D __dso_handle > 00000000 > D __guard_local > 00000000 T _fini > 00000000 T _init > > gmake clean > ./configure > sudo gmake install > the same error > > Does anyone has an idea of how to solve this > issue, because I really need R14B04. > > Please help. I am really out of ideas. > Maybe I should link against other libraries but I do not know what they are. > How can I dicover these libraries that I should link erlang against? > Did you try to link agains libcrtendS.o as well like openbsd guys do for CDE? Also, you need to make sure that crypto.o linking actually uses these flags you changes in erts/configure.in (also, btw, you need to rebuild configure if you changes configure.in). I don't have openbsd installed anywhere, so cannot provide more detailed help, unfortunately. I'd recommend you to play with the linking flags for crypto.o: log the entire build, find where it links crypto.o and use the same command from the shell. It will be easier to try different flags combinations that way. -- ST4096-RIPE From msegalis@REDACTED Thu May 23 23:13:29 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 23:13:29 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> Message-ID: <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> Ok, it finally got into the infinite loop? And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. So back to square one? no way to investigate correctly so far :-/ Morgan. Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : > Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! > > I get back to the Erlang list as soon as I have more informations about this. > > Thank you all ! > > Morgan. > > Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : > >> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >> So I should go back to R15B ? >> >> erlang:memory() gives me >> >> [{total,1525779584}, >> {processes,1272881427}, >> {processes_used,1272789743}, >> {system,252898157}, >> {atom,372217}, >> {atom_used,346096}, >> {binary,148093608}, >> {code,8274446}, >> {ets,1546832}] >> >> >> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >> >> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >> >>> Right, you do not have many processes. Same time you goes out of memory? >>> >>> Unfortunately, I had no time play around with R16B at production? >>> Could it be some issue with SSL, I re-call there was some complains in the list? >>> >>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>> >>> BTW, What erlang:memory() says on you production node? >>> >>> - Dmitry >>> >>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>> >>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>> >>>> Dict: {dict,16,16,16,8,80,48, >>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>> [], >>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>> [], >>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>> [undefined|4]], >>>> [], >>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>> [],[],[], >>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>> [{{ssl_connection,init,1}, >>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>> 1366], >>>> [{unknown,unknown}|3]], >>>> [],[], >>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>> [], >>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>> ok >>>> >>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>> >>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>> >>>>> >>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>> >>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>> >>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>> >>>>> - Dmitry >>>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu May 23 23:20:30 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 24 May 2013 00:20:30 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> Message-ID: <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> You system definitely leaking some resources :-/ - Check number of used FD(s) may be you exceeded limit there - What was overall system memory / cpu utilisation before crash? - Check how many connections you got before crash, may be you can reproduce it at dev - Dmitry On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: > Ok, it finally got into the infinite loop? > > And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. > > So back to square one? no way to investigate correctly so far :-/ > > Morgan. > > Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : > >> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >> >> I get back to the Erlang list as soon as I have more informations about this. >> >> Thank you all ! >> >> Morgan. >> >> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >> >>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>> So I should go back to R15B ? >>> >>> erlang:memory() gives me >>> >>> [{total,1525779584}, >>> {processes,1272881427}, >>> {processes_used,1272789743}, >>> {system,252898157}, >>> {atom,372217}, >>> {atom_used,346096}, >>> {binary,148093608}, >>> {code,8274446}, >>> {ets,1546832}] >>> >>> >>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>> >>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>> >>>> Right, you do not have many processes. Same time you goes out of memory? >>>> >>>> Unfortunately, I had no time play around with R16B at production? >>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>> >>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>> >>>> BTW, What erlang:memory() says on you production node? >>>> >>>> - Dmitry >>>> >>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>> >>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>> >>>>> Dict: {dict,16,16,16,8,80,48, >>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>> [], >>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>> [], >>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>> [undefined|4]], >>>>> [], >>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>> [],[],[], >>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>> [{{ssl_connection,init,1}, >>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>> 1366], >>>>> [{unknown,unknown}|3]], >>>>> [],[], >>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>> [], >>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>> ok >>>>> >>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>> >>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>> >>>>>> >>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>> >>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>> >>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>> >>>>>> - Dmitry >>>>> >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 23 23:30:10 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 23 May 2013 23:30:10 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> Message-ID: <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> Yeah you got that right ! leaking at a huge rate at some point ! - The number of Fd - I don't get close to the max # cat /proc/sys/fs/file-nr 3264 0 6455368 - On the production server there is only the erlang node, no other service? The beam.smp was through the roof at 300% CPU and 97% RAM The weird thing is that it got there in a second, I was looking at it when it happens. - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. I really can't find a pattern here?and I'm becoming a little desperate. Thank you for your help again. Morgan. Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : > You system definitely leaking some resources :-/ > - Check number of used FD(s) may be you exceeded limit there > - What was overall system memory / cpu utilisation before crash? > - Check how many connections you got before crash, may be you can reproduce it at dev > > - Dmitry > > On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: > >> Ok, it finally got into the infinite loop? >> >> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >> >> So back to square one? no way to investigate correctly so far :-/ >> >> Morgan. >> >> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >> >>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>> >>> I get back to the Erlang list as soon as I have more informations about this. >>> >>> Thank you all ! >>> >>> Morgan. >>> >>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>> >>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>> So I should go back to R15B ? >>>> >>>> erlang:memory() gives me >>>> >>>> [{total,1525779584}, >>>> {processes,1272881427}, >>>> {processes_used,1272789743}, >>>> {system,252898157}, >>>> {atom,372217}, >>>> {atom_used,346096}, >>>> {binary,148093608}, >>>> {code,8274446}, >>>> {ets,1546832}] >>>> >>>> >>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>> >>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>> >>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>> >>>>> Unfortunately, I had no time play around with R16B at production? >>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>> >>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>> >>>>> BTW, What erlang:memory() says on you production node? >>>>> >>>>> - Dmitry >>>>> >>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>> >>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>> >>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>> [], >>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>> [], >>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>> [undefined|4]], >>>>>> [], >>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>> [],[],[], >>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>> [{{ssl_connection,init,1}, >>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>> 1366], >>>>>> [{unknown,unknown}|3]], >>>>>> [],[], >>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>> [], >>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>> ok >>>>>> >>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>> >>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>> >>>>>>> >>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>> >>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>> >>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>> >>>>>>> - Dmitry >>>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 24 10:13:46 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 24 May 2013 10:13:46 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> Message-ID: <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> The problem is that the VM freezes completely, it does not generate a crash dump Is there a way to limit the memory that a VM can allocate, so the server is not overwhelmed in order to create a crash dump ? Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : > Have you analyzed the crash dump file with the crash dump viewer? > On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: > Yeah you got that right ! leaking at a huge rate at some point ! > > - The number of Fd - I don't get close to the max > # cat /proc/sys/fs/file-nr > 3264 0 6455368 > > - On the production server there is only the erlang node, no other service? > The beam.smp was through the roof at 300% CPU and 97% RAM > The weird thing is that it got there in a second, I was looking at it when it happens. > > - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. > > I really can't find a pattern here?and I'm becoming a little desperate. > > Thank you for your help again. > > Morgan. > > Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : > >> You system definitely leaking some resources :-/ >> - Check number of used FD(s) may be you exceeded limit there >> - What was overall system memory / cpu utilisation before crash? >> - Check how many connections you got before crash, may be you can reproduce it at dev >> >> - Dmitry >> >> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >> >>> Ok, it finally got into the infinite loop? >>> >>> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >>> >>> So back to square one? no way to investigate correctly so far :-/ >>> >>> Morgan. >>> >>> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >>> >>>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>>> >>>> I get back to the Erlang list as soon as I have more informations about this. >>>> >>>> Thank you all ! >>>> >>>> Morgan. >>>> >>>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>>> >>>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>>> So I should go back to R15B ? >>>>> >>>>> erlang:memory() gives me >>>>> >>>>> [{total,1525779584}, >>>>> {processes,1272881427}, >>>>> {processes_used,1272789743}, >>>>> {system,252898157}, >>>>> {atom,372217}, >>>>> {atom_used,346096}, >>>>> {binary,148093608}, >>>>> {code,8274446}, >>>>> {ets,1546832}] >>>>> >>>>> >>>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>>> >>>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>>> >>>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>>> >>>>>> Unfortunately, I had no time play around with R16B at production? >>>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>>> >>>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>>> >>>>>> BTW, What erlang:memory() says on you production node? >>>>>> >>>>>> - Dmitry >>>>>> >>>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>>> >>>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>>> >>>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>>> [], >>>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>>> [], >>>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>>> [undefined|4]], >>>>>>> [], >>>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>>> [],[],[], >>>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>>> [{{ssl_connection,init,1}, >>>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>>> 1366], >>>>>>> [{unknown,unknown}|3]], >>>>>>> [],[], >>>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>>> [], >>>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>>> ok >>>>>>> >>>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>>> >>>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>>> >>>>>>>> >>>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>>> >>>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>>> >>>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>>> >>>>>>>> - Dmitry >>>>>>> >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Fri May 24 10:17:38 2013 From: dch@REDACTED (Dave Cottlehuber) Date: Fri, 24 May 2013 10:17:38 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> Message-ID: On 24 May 2013 10:13, Morgan Segalis wrote: > The problem is that the VM freezes completely, it does not generate a crash > dump > > Is there a way to limit the memory that a VM can allocate, so the server is > not overwhelmed in order to create a crash dump ? You could use ulimit -v perhaps, I never got ulimit to work right with resident memory. A+ Dave From bourinov@REDACTED Fri May 24 10:28:54 2013 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 24 May 2013 10:28:54 +0200 Subject: [erlang-questions] Rabbit under small load Message-ID: Hello Dear Erlangers! I have about 3000 erlang-processes which should send messages to AMQP-queue of destination and receive replies from it. The message rate at the moment is about 100 M/s Average message size: 4 Kb At the moment I have one locally registered gen_server that does all communication with RabbitMQ. It consumes messages and routes them to corresponding worker processes (via gproc). When worker process need to send something, it cast message to RabbitMQ process via local name. With this approach I see some problems with message consumption speed. What is the right way to interact with RabbitMQ server in Erlang code? Should it be more consumer processes? Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 24 10:31:17 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 24 May 2013 11:31:17 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> Message-ID: <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> Hello, I am not aware of a single flag to limit the memory like in Java. You can try to configure memory allocation http://www.erlang.org/doc/man/erts_alloc.html One of the freeze reason might be a huge crash_dump. See the flags at bottom of page how to tune its behaviour http://www.erlang.org/doc/man/erl.html If you switch off a swap it helps to observe OOM. Would you share to the list app you running applications? application:which_applications() - Dmitry On May 24, 2013, at 11:13 AM, Morgan Segalis wrote: > The problem is that the VM freezes completely, it does not generate a crash dump > > Is there a way to limit the memory that a VM can allocate, so the server is not overwhelmed in order to create a crash dump ? > > Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : > >> Have you analyzed the crash dump file with the crash dump viewer? >> On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: >> Yeah you got that right ! leaking at a huge rate at some point ! >> >> - The number of Fd - I don't get close to the max >> # cat /proc/sys/fs/file-nr >> 3264 0 6455368 >> >> - On the production server there is only the erlang node, no other service? >> The beam.smp was through the roof at 300% CPU and 97% RAM >> The weird thing is that it got there in a second, I was looking at it when it happens. >> >> - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. >> >> I really can't find a pattern here?and I'm becoming a little desperate. >> >> Thank you for your help again. >> >> Morgan. >> >> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : >> >>> You system definitely leaking some resources :-/ >>> - Check number of used FD(s) may be you exceeded limit there >>> - What was overall system memory / cpu utilisation before crash? >>> - Check how many connections you got before crash, may be you can reproduce it at dev >>> >>> - Dmitry >>> >>> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >>> >>>> Ok, it finally got into the infinite loop? >>>> >>>> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >>>> >>>> So back to square one? no way to investigate correctly so far :-/ >>>> >>>> Morgan. >>>> >>>> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >>>> >>>>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>>>> >>>>> I get back to the Erlang list as soon as I have more informations about this. >>>>> >>>>> Thank you all ! >>>>> >>>>> Morgan. >>>>> >>>>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>>>> >>>>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>>>> So I should go back to R15B ? >>>>>> >>>>>> erlang:memory() gives me >>>>>> >>>>>> [{total,1525779584}, >>>>>> {processes,1272881427}, >>>>>> {processes_used,1272789743}, >>>>>> {system,252898157}, >>>>>> {atom,372217}, >>>>>> {atom_used,346096}, >>>>>> {binary,148093608}, >>>>>> {code,8274446}, >>>>>> {ets,1546832}] >>>>>> >>>>>> >>>>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>>>> >>>>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>>>> >>>>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>>>> >>>>>>> Unfortunately, I had no time play around with R16B at production? >>>>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>>>> >>>>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>>>> >>>>>>> BTW, What erlang:memory() says on you production node? >>>>>>> >>>>>>> - Dmitry >>>>>>> >>>>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>>>> >>>>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>>>> >>>>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>>>> [], >>>>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>>>> [], >>>>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>>>> [undefined|4]], >>>>>>>> [], >>>>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>>>> [],[],[], >>>>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>>>> [{{ssl_connection,init,1}, >>>>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>>>> 1366], >>>>>>>> [{unknown,unknown}|3]], >>>>>>>> [],[], >>>>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>>>> [], >>>>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>>>> ok >>>>>>>> >>>>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>>>> >>>>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>>>> >>>>>>>>> >>>>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>>>> >>>>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>>>> >>>>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>>>> >>>>>>>>> - Dmitry >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 24 10:34:37 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 24 May 2013 10:34:37 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> Message-ID: Thank you, I'll look into it. Heres what application:which_application() gives me : [{emysql,"Emysql - Erlang MySQL driver","0.2"}, {ssl,"Erlang/OTP SSL application","5.2.1"}, {public_key,"Public key infrastructure","0.18"}, {crypto,"CRYPTO version 2","2.3"}, {stdlib,"ERTS CXC 138 10","1.19.1"}, {kernel,"ERTS CXC 138 10","2.16.1"}] nothing fancy has you can see... Le 24 mai 2013 ? 10:31, Dmitry Kolesnikov a ?crit : > Hello, > > I am not aware of a single flag to limit the memory like in Java. > You can try to configure memory allocation > http://www.erlang.org/doc/man/erts_alloc.html > > One of the freeze reason might be a huge crash_dump. > See the flags at bottom of page how to tune its behaviour > http://www.erlang.org/doc/man/erl.html > > If you switch off a swap it helps to observe OOM. > > Would you share to the list app you running applications? > application:which_applications() > > > - Dmitry > > > On May 24, 2013, at 11:13 AM, Morgan Segalis wrote: > >> The problem is that the VM freezes completely, it does not generate a crash dump >> >> Is there a way to limit the memory that a VM can allocate, so the server is not overwhelmed in order to create a crash dump ? >> >> Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : >> >>> Have you analyzed the crash dump file with the crash dump viewer? >>> On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: >>> Yeah you got that right ! leaking at a huge rate at some point ! >>> >>> - The number of Fd - I don't get close to the max >>> # cat /proc/sys/fs/file-nr >>> 3264 0 6455368 >>> >>> - On the production server there is only the erlang node, no other service? >>> The beam.smp was through the roof at 300% CPU and 97% RAM >>> The weird thing is that it got there in a second, I was looking at it when it happens. >>> >>> - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. >>> >>> I really can't find a pattern here?and I'm becoming a little desperate. >>> >>> Thank you for your help again. >>> >>> Morgan. >>> >>> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : >>> >>>> You system definitely leaking some resources :-/ >>>> - Check number of used FD(s) may be you exceeded limit there >>>> - What was overall system memory / cpu utilisation before crash? >>>> - Check how many connections you got before crash, may be you can reproduce it at dev >>>> >>>> - Dmitry >>>> >>>> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >>>> >>>>> Ok, it finally got into the infinite loop? >>>>> >>>>> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >>>>> >>>>> So back to square one? no way to investigate correctly so far :-/ >>>>> >>>>> Morgan. >>>>> >>>>> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >>>>> >>>>>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>>>>> >>>>>> I get back to the Erlang list as soon as I have more informations about this. >>>>>> >>>>>> Thank you all ! >>>>>> >>>>>> Morgan. >>>>>> >>>>>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>>>>> >>>>>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>>>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>>>>> So I should go back to R15B ? >>>>>>> >>>>>>> erlang:memory() gives me >>>>>>> >>>>>>> [{total,1525779584}, >>>>>>> {processes,1272881427}, >>>>>>> {processes_used,1272789743}, >>>>>>> {system,252898157}, >>>>>>> {atom,372217}, >>>>>>> {atom_used,346096}, >>>>>>> {binary,148093608}, >>>>>>> {code,8274446}, >>>>>>> {ets,1546832}] >>>>>>> >>>>>>> >>>>>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>>>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>>>>> >>>>>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>>>>> >>>>>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>>>>> >>>>>>>> Unfortunately, I had no time play around with R16B at production? >>>>>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>>>>> >>>>>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>>>>> >>>>>>>> BTW, What erlang:memory() says on you production node? >>>>>>>> >>>>>>>> - Dmitry >>>>>>>> >>>>>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>>>>> >>>>>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>>>>> >>>>>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>>>>> [], >>>>>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>>>>> [], >>>>>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>>>>> [undefined|4]], >>>>>>>>> [], >>>>>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>>>>> [],[],[], >>>>>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>>>>> [{{ssl_connection,init,1}, >>>>>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>>>>> 1366], >>>>>>>>> [{unknown,unknown}|3]], >>>>>>>>> [],[], >>>>>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>>>>> [], >>>>>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>>>>> ok >>>>>>>>> >>>>>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>>>>> >>>>>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>>>>> >>>>>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>>>>> >>>>>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>>>>> >>>>>>>>>> - Dmitry >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 24 10:35:48 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 24 May 2013 10:35:48 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> Message-ID: Thanks ! I'll try that? will keep you posted. Le 24 mai 2013 ? 10:30, masked.prize@REDACTED a ?crit : > You can run a little function that writes process info to files every N seconds. Like this: > F = fun(F2, T) -> Seconds=calendar:datetime_to_gregorian_seconds(calendar:now_to_local_time(now())), Fname=lists:flatten(io_lib:format("/tmp/f-~p", [Seconds])), [begin Info=process_info(Pid), Data=io_lib:format("~p:~n~p~n", [Pid, Info]), file:write_file(Fname, Data, [append]) end || Pid <- processes()], timer:sleep(T), F2(F2, T) end. > > run it from console with F(F, 5000) and get a bunch of files in /tmp that probably can provide you something useful > > On Friday, May 24, 2013 1:13:29 AM UTC+4, Morgan Segalis wrote: > Ok, it finally got into the infinite loop? > > And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. > > So back to square one? no way to investigate correctly so far :-/ > > Morgan. > > Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : > >> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >> >> I get back to the Erlang list as soon as I have more informations about this. >> >> Thank you all ! >> >> Morgan. >> >> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >> >>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>> So I should go back to R15B ? >>> >>> erlang:memory() gives me >>> >>> [{total,1525779584}, >>> {processes,1272881427}, >>> {processes_used,1272789743}, >>> {system,252898157}, >>> {atom,372217}, >>> {atom_used,346096}, >>> {binary,148093608}, >>> {code,8274446}, >>> {ets,1546832}] >>> >>> >>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>> >>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>> >>>> Right, you do not have many processes. Same time you goes out of memory? >>>> >>>> Unfortunately, I had no time play around with R16B at production? >>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>> >>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>> >>>> BTW, What erlang:memory() says on you production node? >>>> >>>> - Dmitry >>>> >>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>> >>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>> >>>>> Dict: {dict,16,16,16,8,80,48, >>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>> [], >>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>> [], >>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>> [undefined|4]], >>>>> [], >>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>> [],[],[], >>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>> [{{ssl_connection,init,1}, >>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>> 1366], >>>>> [{unknown,unknown}|3]], >>>>> [],[], >>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>> [], >>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>> ok >>>>> >>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>> >>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>> >>>>>> >>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>> >>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>> >>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>> >>>>>> - Dmitry >>>>> >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-q...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Fri May 24 10:48:22 2013 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 24 May 2013 11:48:22 +0300 Subject: [erlang-questions] R14B04 and OpenSSL 1.0.1c In-Reply-To: <88DD116E-E734-462C-87A6-42D8A69C9E4F@freebsd.org> References: <7843DA5D-0B16-46D8-B19A-4AFF899ADD09@freebsd.org> <61E5F453-3A4C-4B82-B274-D41C45AA9663@freebsd.org> <88DD116E-E734-462C-87A6-42D8A69C9E4F@freebsd.org> Message-ID: in erts/configure.in there is a section: openbsd*) DED_LD="$CC" DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" DED_LDFLAGS="-shared" ;; Spoke with guys on the misc@REDACTED maillist and they told me to run autoreconf, to regenerate configure from configure.in . And indeed that did the trick and now is working fine. There is no need to specify : DED_LDFLAGS="-shared /usr/lib/crtbeginS.o /usr/lib/crtendS.o" because this generates conflict on symbol '__guard_local' Based on discussions in this topic and in misc@REDACTED maillist here is a small recipe for those who need to compile erlang R14B04 from sources on OpenBSD 5.3 amd64: 1. cd otp_src_R14B04 2. apply the patches founded in directory /usr/ports/lang/erlang/patches/ 3. $ autoreconf 4. compile and install as usual Thank you for your help, Bogdan On Thu, May 23, 2013 at 11:50 PM, Stanislav Sedov wrote: > On May 23, 2013, at 2:46 AM, Bogdan Andu wrote: > > > I have applied the patches: > > cd $OTP_SRC_DIR_R14B04 > > patch -p0 < > > /usr/ports/lang/erlang/patches/patch-erts_configure_in > > patch -p0 < > > /usr/ports/lang/erlang/patches/patch-erts_emulator_Makefile_in > > patch -p0 < > > /usr/ports/lang/erlang/patches/patch-erts_lib_src_Makefile_in > > patch -p0 < > > /usr/ports/lang/erlang/patches/patch-lib_tools_c_src_Makefile_in > > patch -p0 < > > /usr/ports/lang/erlang/patches/patch-make_target_mk > > > > *all patches succedded, > > and then: > > > > ./configure > > sudo gmake install > > the same error > > > > I modified > > erts/configure.in: > > from: > > openbsd*) > > DED_LD="$CC" > > > DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" > > DED_LDFLAGS="-shared" > > ;; > > in: > > openbsd*) > > > > DED_LD="$CC" > > > > DED_LD_FLAG_RUNTIME_LIBRARY_PATH="$CFLAG_RUNTIME_LIBRARY_PATH" > > > > DED_LDFLAGS="-shared /usr/lib/crtbeginS.o /usr/lib/crtbegin.o" > > ;; > > because: > > $ nm /usr/lib/crtbegin.o > > U _GLOBAL_OFFSET_TABLE_ > > U > > _Jv_RegisterClasses > > 00000000 D __dso_handle > > 00000000 T __fini > > 00000000 D > > __guard_local > > 00000000 T __init > > 00000000 W __register_frame_info > > U > > atexit > > $ nm /usr/lib/crtbeginS.o > > U _GLOBAL_OFFSET_TABLE_ > > U > > _Jv_RegisterClasses > > U __cxa_finalize > > 00000000 D __dso_handle > > 00000000 > > D __guard_local > > 00000000 T _fini > > 00000000 T _init > > > > gmake clean > > ./configure > > sudo gmake install > > the same error > > > > Does anyone has an idea of how to solve this > > issue, because I really need R14B04. > > > > Please help. I am really out of ideas. > > Maybe I should link against other libraries but I do not know what they > are. > > How can I dicover these libraries that I should link erlang against? > > > > Did you try to link agains libcrtendS.o as well like openbsd guys do for > CDE? > Also, you need to make sure that crypto.o linking actually uses these > flags you > changes in erts/configure.in (also, btw, you need to rebuild configure if > you > changes configure.in). > > I don't have openbsd installed anywhere, so cannot provide more detailed > help, > unfortunately. I'd recommend you to play with the linking flags for > crypto.o: > log the entire build, find where it links crypto.o and use the same command > from the shell. It will be easier to try different flags combinations that > way. > > -- > ST4096-RIPE > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 24 11:06:03 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 24 May 2013 12:06:03 +0300 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> Message-ID: Indeed the list of apps is simple, basically this is either ssl, emysql or your integration layer. If you read a lot of data from mysql then that code might leak a memory by keeping ref to huge binaries. E.g. you you do select * from xxx then that data is returned as set of binaries. Whole binary hands in memory even if a first row is used. This is a one way to treat binaries received from somewhere: case binary:referenced_byte_size(X) of Large when Large > 2 * byte_size(X) -> binary:copy(X); _ -> X end as long as I see emysql does not use binary:copy anywhere but of course they might dereference them in other way around. If you could monitor a erlang:memory(binary) over a time then it might reveal the case. BTW, is this specific to R16 or other release if so then some glitches at ssl? - Dmitry On May 24, 2013, at 11:34 AM, Morgan Segalis wrote: > Thank you, I'll look into it. > > Heres what application:which_application() gives me : > > [{emysql,"Emysql - Erlang MySQL driver","0.2"}, > {ssl,"Erlang/OTP SSL application","5.2.1"}, > {public_key,"Public key infrastructure","0.18"}, > {crypto,"CRYPTO version 2","2.3"}, > {stdlib,"ERTS CXC 138 10","1.19.1"}, > {kernel,"ERTS CXC 138 10","2.16.1"}] > > nothing fancy has you can see... > > Le 24 mai 2013 ? 10:31, Dmitry Kolesnikov a ?crit : > >> Hello, >> >> I am not aware of a single flag to limit the memory like in Java. >> You can try to configure memory allocation >> http://www.erlang.org/doc/man/erts_alloc.html >> >> One of the freeze reason might be a huge crash_dump. >> See the flags at bottom of page how to tune its behaviour >> http://www.erlang.org/doc/man/erl.html >> >> If you switch off a swap it helps to observe OOM. >> >> Would you share to the list app you running applications? >> application:which_applications() >> >> >> - Dmitry >> >> >> On May 24, 2013, at 11:13 AM, Morgan Segalis wrote: >> >>> The problem is that the VM freezes completely, it does not generate a crash dump >>> >>> Is there a way to limit the memory that a VM can allocate, so the server is not overwhelmed in order to create a crash dump ? >>> >>> Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : >>> >>>> Have you analyzed the crash dump file with the crash dump viewer? >>>> On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: >>>> Yeah you got that right ! leaking at a huge rate at some point ! >>>> >>>> - The number of Fd - I don't get close to the max >>>> # cat /proc/sys/fs/file-nr >>>> 3264 0 6455368 >>>> >>>> - On the production server there is only the erlang node, no other service? >>>> The beam.smp was through the roof at 300% CPU and 97% RAM >>>> The weird thing is that it got there in a second, I was looking at it when it happens. >>>> >>>> - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. >>>> >>>> I really can't find a pattern here?and I'm becoming a little desperate. >>>> >>>> Thank you for your help again. >>>> >>>> Morgan. >>>> >>>> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : >>>> >>>>> You system definitely leaking some resources :-/ >>>>> - Check number of used FD(s) may be you exceeded limit there >>>>> - What was overall system memory / cpu utilisation before crash? >>>>> - Check how many connections you got before crash, may be you can reproduce it at dev >>>>> >>>>> - Dmitry >>>>> >>>>> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >>>>> >>>>>> Ok, it finally got into the infinite loop? >>>>>> >>>>>> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >>>>>> >>>>>> So back to square one? no way to investigate correctly so far :-/ >>>>>> >>>>>> Morgan. >>>>>> >>>>>> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >>>>>> >>>>>>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>>>>>> >>>>>>> I get back to the Erlang list as soon as I have more informations about this. >>>>>>> >>>>>>> Thank you all ! >>>>>>> >>>>>>> Morgan. >>>>>>> >>>>>>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>>>>>> >>>>>>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>>>>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>>>>>> So I should go back to R15B ? >>>>>>>> >>>>>>>> erlang:memory() gives me >>>>>>>> >>>>>>>> [{total,1525779584}, >>>>>>>> {processes,1272881427}, >>>>>>>> {processes_used,1272789743}, >>>>>>>> {system,252898157}, >>>>>>>> {atom,372217}, >>>>>>>> {atom_used,346096}, >>>>>>>> {binary,148093608}, >>>>>>>> {code,8274446}, >>>>>>>> {ets,1546832}] >>>>>>>> >>>>>>>> >>>>>>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>>>>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>>>>>> >>>>>>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>>>>>> >>>>>>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>>>>>> >>>>>>>>> Unfortunately, I had no time play around with R16B at production? >>>>>>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>>>>>> >>>>>>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>>>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>>>>>> >>>>>>>>> BTW, What erlang:memory() says on you production node? >>>>>>>>> >>>>>>>>> - Dmitry >>>>>>>>> >>>>>>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>>>>>> >>>>>>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>>>>>> >>>>>>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>>>>>> [], >>>>>>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>>>>>> [], >>>>>>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>>>>>> [undefined|4]], >>>>>>>>>> [], >>>>>>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>>>>>> [],[],[], >>>>>>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>>>>>> [{{ssl_connection,init,1}, >>>>>>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>>>>>> 1366], >>>>>>>>>> [{unknown,unknown}|3]], >>>>>>>>>> [],[], >>>>>>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>>>>>> [], >>>>>>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>>>>>> ok >>>>>>>>>> >>>>>>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>>>>>> >>>>>>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>>>>>> >>>>>>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>>>>>> >>>>>>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>>>>>> >>>>>>>>>>> - Dmitry >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri May 24 11:05:43 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 24 May 2013 14:35:43 +0530 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> Message-ID: Have you set the ERL_CRASH_DUMP_SECONDS environment variable?: It won't create one unless you set it to a positive value. Set it to 60 or more to be sure it completes. On May 24, 2013 1:43 PM, "Morgan Segalis" wrote: > The problem is that the VM freezes completely, it does not generate a > crash dump > > Is there a way to limit the memory that a VM can allocate, so the server > is not overwhelmed in order to create a crash dump ? > > Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : > > Have you analyzed the crash dump file with the crash dump viewer? > On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: > >> Yeah you got that right ! leaking at a huge rate at some point ! >> >> - The number of Fd - I don't get close to the max >> # cat /proc/sys/fs/file-nr >> 3264 0 6455368 >> >> - On the production server there is only the erlang node, no other >> service? >> The beam.smp was through the roof at 300% CPU and 97% RAM >> The weird thing is that it got there in a second, I was looking at it >> when it happens. >> >> - It has happened with 2000 connections, 4000 connections, and 10000 >> connections? 5 min after start, 5hours after start. >> >> I really can't find a pattern here?and I'm becoming a little desperate. >> >> Thank you for your help again. >> >> Morgan. >> >> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a >> ?crit : >> >> You system definitely leaking some resources :-/ >> - Check number of used FD(s) may be you exceeded limit there >> - What was overall system memory / cpu utilisation before crash? >> - Check how many connections you got before crash, may be you can >> reproduce it at dev >> >> - Dmitry >> >> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >> >> Ok, it finally got into the infinite loop? >> >> And of course, the node on which I was running etop could not give me >> anymore since it got disconnected from the production node. >> >> So back to square one? no way to investigate correctly so far :-/ >> >> Morgan. >> >> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >> >> Yeah that what I'm doing right now, but of course, when I'm monitoring >> it, it won't crash, only when I sleep !! >> >> I get back to the Erlang list as soon as I have more informations about >> this. >> >> Thank you all ! >> >> Morgan. >> >> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >> >> Keep etop running and capture the output to a file (e.g. etop ... | tee >> stop.log). After it gets into trouble look back and see what was happening >> beforehand. >> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >> >>> So I should go back to R15B ? >>> >>> erlang:memory() gives me >>> >>> [{total,1525779584}, >>> {processes,1272881427}, >>> {processes_used,1272789743}, >>> {system,252898157}, >>> {atom,372217}, >>> {atom_used,346096}, >>> {binary,148093608}, >>> {code,8274446}, >>> {ets,1546832}] >>> >>> >>> But keep in mind that right now, there is no infinite loop, or memory >>> issue at this exact time? >>> It will be more interesting to have that when the VM is asking for 14GB >>> of memory, but when it does, the console is unresponsive, so I can't get >>> anything then. >>> >>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a >>> ?crit : >>> >>> Right, you do not have many processes. Same time you goes out of memory? >>> >>> Unfortunately, I had no time play around with R16B at production? >>> Could it be some issue with SSL, I re-call there was some complains in >>> the list? >>> >>> I would use entop to spot the process that has either too much >>> reductions, queue len or heap. >>> Once you know they pid you can dig more info about them using >>> erlang:process_info(?) and/or sys:get:status(?) >>> >>> BTW, What erlang:memory() says on you production node? >>> >>> - Dmitry >>> >>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>> >>> No, I was talking about the function I made to investigate which >>> processes I have created, which gives me this output : >>> >>> Dict: {dict,16,16,16,8,80,48, >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>> >>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>> [], >>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>> >>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>> >>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>> [], >>> >>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>> [undefined|4]], >>> [], >>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>> >>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>> [],[],[], >>> >>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>> [{{ssl_connection,init,1}, >>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>> 1366], >>> [{unknown,unknown}|3]], >>> [],[], >>> >>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>> [], >>> >>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>> ok >>> >>> I'm very satisfied with supervisor, and I don't think to have the >>> expertise tweaking it... >>> >>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a >>> ?crit : >>> >>> >>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>> >>> I have made a little function a while back, getting all processes and >>> removing the processes inited at the beginning? >>> >>> >>> Could you please elaborate on that? Why you are not satisfied with >>> supervisor? >>> >>> - Dmitry >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri May 24 11:27:43 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 24 May 2013 14:57:43 +0530 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> Message-ID: It "freezes"? So it hasn't crashed at all. In that case you just need to be more patient and wait for it to either crash, and leave a crash dump, or output to etop. Possibly setting process priorities would help. Give the suspicious ones low priority. If it's CPU resources which are being depleted you would want to observe which have the most reductions. Use stop to order by reductions and see who's the busiest. Another way would be to run a debug emulator and interrupt it while it's frozen. Then inspect the backtrace to see what it has been doing. On May 24, 2013 1:43 PM, "Morgan Segalis" wrote: > The problem is that the VM freezes completely, it does not generate a > crash dump > > Is there a way to limit the memory that a VM can allocate, so the server > is not overwhelmed in order to create a crash dump ? > > Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : > > Have you analyzed the crash dump file with the crash dump viewer? > On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: > >> Yeah you got that right ! leaking at a huge rate at some point ! >> >> - The number of Fd - I don't get close to the max >> # cat /proc/sys/fs/file-nr >> 3264 0 6455368 >> >> - On the production server there is only the erlang node, no other >> service? >> The beam.smp was through the roof at 300% CPU and 97% RAM >> The weird thing is that it got there in a second, I was looking at it >> when it happens. >> >> - It has happened with 2000 connections, 4000 connections, and 10000 >> connections? 5 min after start, 5hours after start. >> >> I really can't find a pattern here?and I'm becoming a little desperate. >> >> Thank you for your help again. >> >> Morgan. >> >> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a >> ?crit : >> >> You system definitely leaking some resources :-/ >> - Check number of used FD(s) may be you exceeded limit there >> - What was overall system memory / cpu utilisation before crash? >> - Check how many connections you got before crash, may be you can >> reproduce it at dev >> >> - Dmitry >> >> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >> >> Ok, it finally got into the infinite loop? >> >> And of course, the node on which I was running etop could not give me >> anymore since it got disconnected from the production node. >> >> So back to square one? no way to investigate correctly so far :-/ >> >> Morgan. >> >> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >> >> Yeah that what I'm doing right now, but of course, when I'm monitoring >> it, it won't crash, only when I sleep !! >> >> I get back to the Erlang list as soon as I have more informations about >> this. >> >> Thank you all ! >> >> Morgan. >> >> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >> >> Keep etop running and capture the output to a file (e.g. etop ... | tee >> stop.log). After it gets into trouble look back and see what was happening >> beforehand. >> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >> >>> So I should go back to R15B ? >>> >>> erlang:memory() gives me >>> >>> [{total,1525779584}, >>> {processes,1272881427}, >>> {processes_used,1272789743}, >>> {system,252898157}, >>> {atom,372217}, >>> {atom_used,346096}, >>> {binary,148093608}, >>> {code,8274446}, >>> {ets,1546832}] >>> >>> >>> But keep in mind that right now, there is no infinite loop, or memory >>> issue at this exact time? >>> It will be more interesting to have that when the VM is asking for 14GB >>> of memory, but when it does, the console is unresponsive, so I can't get >>> anything then. >>> >>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a >>> ?crit : >>> >>> Right, you do not have many processes. Same time you goes out of memory? >>> >>> Unfortunately, I had no time play around with R16B at production? >>> Could it be some issue with SSL, I re-call there was some complains in >>> the list? >>> >>> I would use entop to spot the process that has either too much >>> reductions, queue len or heap. >>> Once you know they pid you can dig more info about them using >>> erlang:process_info(?) and/or sys:get:status(?) >>> >>> BTW, What erlang:memory() says on you production node? >>> >>> - Dmitry >>> >>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>> >>> No, I was talking about the function I made to investigate which >>> processes I have created, which gives me this output : >>> >>> Dict: {dict,16,16,16,8,80,48, >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>> >>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>> [], >>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>> >>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>> >>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>> [], >>> >>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>> [undefined|4]], >>> [], >>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>> >>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>> [],[],[], >>> >>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>> [{{ssl_connection,init,1}, >>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>> 1366], >>> [{unknown,unknown}|3]], >>> [],[], >>> >>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>> [], >>> >>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>> ok >>> >>> I'm very satisfied with supervisor, and I don't think to have the >>> expertise tweaking it... >>> >>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a >>> ?crit : >>> >>> >>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>> >>> I have made a little function a while back, getting all processes and >>> removing the processes inited at the beginning? >>> >>> >>> Could you please elaborate on that? Why you are not satisfied with >>> supervisor? >>> >>> - Dmitry >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri May 24 13:15:25 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 24 May 2013 13:15:25 +0200 Subject: [erlang-questions] Rabbit under small load In-Reply-To: References: Message-ID: On May 24, 2013, at 10:28 AM, Max Bourinov wrote: > Hello Dear Erlangers! > > I have about 3000 erlang-processes which should send messages to AMQP-queue of destination and receive replies from it. > > The message rate at the moment is about 100 M/s > Average message size: 4 Kb > Is that 100 * 1000 * 1000 messages per second or 100 messages per second? RabbitMQ should easily be able to handle the latter load without much trouble. I have built a gen_server as a middle-man for RabbitMQ which handles the request/reply toward RMQ more or less like your approach and I can easily do 6000 messages per second with this setup and very little to no tuning. I don't know where the limit to that solution is, but there is a one-core bottleneck in the gen_server. 100 million messages is a different question however. Given the average message size, you are at 400 megabytes per second and hence you need 4 Gigabit of bandwidth to even hope hitting target. Apart from that, I don't think RMQ can handle that amount of load. You may need something else entirely to handle that. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From jesper.louis.andersen@REDACTED Fri May 24 13:17:59 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 24 May 2013 13:17:59 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: References: <20130523053834.GC428@aluminium.local> <6FA1A68D-11E0-4D54-A582-93F53EF88106@gmail.com> <05FA7DB8-2437-45DF-BE28-536BACFB0235@gmail.com> <53FFEBFF-823C-422F-BC38-08EFB5EB2320@gmail.com> <0BB52D78-76AC-4EB0-9F5C-24753942C936@gmail.com> <66CAC435-8619-46ED-A079-DA2F1084DA24@gmail.com> <5BDCBECA-6D8D-4539-8483-D15A821DFE9B@gmail.com> <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> Message-ID: emysql? check your MySQL query log for queries with large return results. Emysql has no cursor-support, so you have to build the result in memory and this can *easily* blow up your heap space if you do a large select. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On May 24, 2013, at 10:34 AM, Morgan Segalis wrote: > Thank you, I'll look into it. > > Heres what application:which_application() gives me : > > [{emysql,"Emysql - Erlang MySQL driver","0.2"}, > {ssl,"Erlang/OTP SSL application","5.2.1"}, > {public_key,"Public key infrastructure","0.18"}, > {crypto,"CRYPTO version 2","2.3"}, > {stdlib,"ERTS CXC 138 10","1.19.1"}, > {kernel,"ERTS CXC 138 10","2.16.1"}] > > nothing fancy has you can see... > > Le 24 mai 2013 ? 10:31, Dmitry Kolesnikov a ?crit : > >> Hello, >> >> I am not aware of a single flag to limit the memory like in Java. >> You can try to configure memory allocation >> http://www.erlang.org/doc/man/erts_alloc.html >> >> One of the freeze reason might be a huge crash_dump. >> See the flags at bottom of page how to tune its behaviour >> http://www.erlang.org/doc/man/erl.html >> >> If you switch off a swap it helps to observe OOM. >> >> Would you share to the list app you running applications? >> application:which_applications() >> >> >> - Dmitry >> >> >> On May 24, 2013, at 11:13 AM, Morgan Segalis wrote: >> >>> The problem is that the VM freezes completely, it does not generate a crash dump >>> >>> Is there a way to limit the memory that a VM can allocate, so the server is not overwhelmed in order to create a crash dump ? >>> >>> Le 24 mai 2013 ? 02:00, Vance Shipley a ?crit : >>> >>>> Have you analyzed the crash dump file with the crash dump viewer? >>>> On May 24, 2013 3:00 AM, "Morgan Segalis" wrote: >>>> Yeah you got that right ! leaking at a huge rate at some point ! >>>> >>>> - The number of Fd - I don't get close to the max >>>> # cat /proc/sys/fs/file-nr >>>> 3264 0 6455368 >>>> >>>> - On the production server there is only the erlang node, no other service? >>>> The beam.smp was through the roof at 300% CPU and 97% RAM >>>> The weird thing is that it got there in a second, I was looking at it when it happens. >>>> >>>> - It has happened with 2000 connections, 4000 connections, and 10000 connections? 5 min after start, 5hours after start. >>>> >>>> I really can't find a pattern here?and I'm becoming a little desperate. >>>> >>>> Thank you for your help again. >>>> >>>> Morgan. >>>> >>>> Le 23 mai 2013 ? 23:20, Dmitry Kolesnikov a ?crit : >>>> >>>>> You system definitely leaking some resources :-/ >>>>> - Check number of used FD(s) may be you exceeded limit there >>>>> - What was overall system memory / cpu utilisation before crash? >>>>> - Check how many connections you got before crash, may be you can reproduce it at dev >>>>> >>>>> - Dmitry >>>>> >>>>> On May 24, 2013, at 12:13 AM, Morgan Segalis wrote: >>>>> >>>>>> Ok, it finally got into the infinite loop? >>>>>> >>>>>> And of course, the node on which I was running etop could not give me anymore since it got disconnected from the production node. >>>>>> >>>>>> So back to square one? no way to investigate correctly so far :-/ >>>>>> >>>>>> Morgan. >>>>>> >>>>>> Le 23 mai 2013 ? 16:34, Morgan Segalis a ?crit : >>>>>> >>>>>>> Yeah that what I'm doing right now, but of course, when I'm monitoring it, it won't crash, only when I sleep !! >>>>>>> >>>>>>> I get back to the Erlang list as soon as I have more informations about this. >>>>>>> >>>>>>> Thank you all ! >>>>>>> >>>>>>> Morgan. >>>>>>> >>>>>>> Le 23 mai 2013 ? 16:30, Vance Shipley a ?crit : >>>>>>> >>>>>>>> Keep etop running and capture the output to a file (e.g. etop ... | tee stop.log). After it gets into trouble look back and see what was happening beforehand. >>>>>>>> On May 23, 2013 6:16 PM, "Morgan Segalis" wrote: >>>>>>>> So I should go back to R15B ? >>>>>>>> >>>>>>>> erlang:memory() gives me >>>>>>>> >>>>>>>> [{total,1525779584}, >>>>>>>> {processes,1272881427}, >>>>>>>> {processes_used,1272789743}, >>>>>>>> {system,252898157}, >>>>>>>> {atom,372217}, >>>>>>>> {atom_used,346096}, >>>>>>>> {binary,148093608}, >>>>>>>> {code,8274446}, >>>>>>>> {ets,1546832}] >>>>>>>> >>>>>>>> >>>>>>>> But keep in mind that right now, there is no infinite loop, or memory issue at this exact time? >>>>>>>> It will be more interesting to have that when the VM is asking for 14GB of memory, but when it does, the console is unresponsive, so I can't get anything then. >>>>>>>> >>>>>>>> Le 23 mai 2013 ? 14:39, Dmitry Kolesnikov a ?crit : >>>>>>>> >>>>>>>>> Right, you do not have many processes. Same time you goes out of memory? >>>>>>>>> >>>>>>>>> Unfortunately, I had no time play around with R16B at production? >>>>>>>>> Could it be some issue with SSL, I re-call there was some complains in the list? >>>>>>>>> >>>>>>>>> I would use entop to spot the process that has either too much reductions, queue len or heap. >>>>>>>>> Once you know they pid you can dig more info about them using erlang:process_info(?) and/or sys:get:status(?) >>>>>>>>> >>>>>>>>> BTW, What erlang:memory() says on you production node? >>>>>>>>> >>>>>>>>> - Dmitry >>>>>>>>> >>>>>>>>> On May 23, 2013, at 3:25 PM, Morgan Segalis wrote: >>>>>>>>> >>>>>>>>>> No, I was talking about the function I made to investigate which processes I have created, which gives me this output : >>>>>>>>>> >>>>>>>>>> Dict: {dict,16,16,16,8,80,48, >>>>>>>>>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >>>>>>>>>> {{[[{{connector_serv,init,1},[connector_suprc42,connector,<0.42.0>]}|548]], >>>>>>>>>> [], >>>>>>>>>> [[{{supervisor,connector_sup,1},[connector,<0.42.0>]}|3], >>>>>>>>>> [{{connector_serv,init,1},[connector_supssl,connector,<0.42.0>]}|1460], >>>>>>>>>> [{{supervisor,casserl_sup,1},[connector,<0.42.0>]}|1], >>>>>>>>>> [{{supervisor,pushiphone_sup,1},[connector,<0.42.0>]}|2], >>>>>>>>>> [{{pushiphone,init,1},['pushiphone-lite',connector,<0.42.0>]}|3], >>>>>>>>>> [{{supervisor,clientpool_sup,1},[connector,<0.42.0>]}|1]], >>>>>>>>>> [], >>>>>>>>>> [[{{clientpool,init,1},[clientpool_sup,connector,<0.42.0>]}|1], >>>>>>>>>> [undefined|4]], >>>>>>>>>> [], >>>>>>>>>> [[{{supervisor,connector,1},[<0.42.0>]}|1], >>>>>>>>>> [{{casserl_serv,init,1},[casserl_sup,connector,<0.42.0>]}|50]], >>>>>>>>>> [],[],[], >>>>>>>>>> [[{{connector_serv,init,1},[connector_suprc4,connector,<0.42.0>]}|472], >>>>>>>>>> [{{ssl_connection,init,1}, >>>>>>>>>> [ssl_connection_sup,ssl_sup,<0.51.0>]}| >>>>>>>>>> 1366], >>>>>>>>>> [{unknown,unknown}|3]], >>>>>>>>>> [],[], >>>>>>>>>> [[{{pushiphone,init,1},['pushiphone-full',connector,<0.42.0>]}|3]], >>>>>>>>>> [], >>>>>>>>>> [[{{pg2,init,1},[kernel_safe_sup,kernel_sup,<0.10.0>]}|1]]}}} >>>>>>>>>> ok >>>>>>>>>> >>>>>>>>>> I'm very satisfied with supervisor, and I don't think to have the expertise tweaking it... >>>>>>>>>> >>>>>>>>>> Le 23 mai 2013 ? 14:19, Dmitry Kolesnikov a ?crit : >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On May 23, 2013, at 1:04 PM, Morgan Segalis wrote: >>>>>>>>>>> >>>>>>>>>>>> I have made a little function a while back, getting all processes and removing the processes inited at the beginning? >>>>>>>>>>> >>>>>>>>>>> Could you please elaborate on that? Why you are not satisfied with supervisor? >>>>>>>>>>> >>>>>>>>>>> - Dmitry >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From yogishb@REDACTED Fri May 24 19:04:59 2013 From: yogishb@REDACTED (Yogish Baliga) Date: Fri, 24 May 2013 10:04:59 -0700 Subject: [erlang-questions] Rabbit under small load In-Reply-To: References: Message-ID: Single RabbitMQ server cannot handle your load requirement. You need to setup multiple RabbitMQ servers and handle shard the messages across multiple servers. -- baliga On Fri, May 24, 2013 at 1:28 AM, Max Bourinov wrote: > Hello Dear Erlangers! > > I have about 3000 erlang-processes which should send messages to > AMQP-queue of destination and receive replies from it. > > The message rate at the moment is about 100 M/s > Average message size: 4 Kb > > At the moment I have one locally registered gen_server that does all > communication with RabbitMQ. It consumes messages and routes them > to corresponding worker processes (via gproc). When worker process need to > send something, it cast message to RabbitMQ process via local name. With > this approach I see some problems with message consumption speed. > > What is the right way to interact with RabbitMQ server in Erlang > code? Should it be more consumer processes? > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri May 24 19:18:30 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 24 May 2013 18:18:30 +0100 Subject: [erlang-questions] Rabbit under small load In-Reply-To: References: Message-ID: <72AF0945-1130-45F1-B624-966BA19164E0@gmail.com> Nonsense. He hasn't stated what his target throughput actually is, so how can you say that? On 24 May 2013, at 18:04, Yogish Baliga wrote: > Single RabbitMQ server cannot handle your load requirement. You need to setup multiple RabbitMQ servers and handle shard the messages across multiple servers. > > -- baliga > > > On Fri, May 24, 2013 at 1:28 AM, Max Bourinov wrote: > Hello Dear Erlangers! > > I have about 3000 erlang-processes which should send messages to AMQP-queue of destination and receive replies from it. > > The message rate at the moment is about 100 M/s > Average message size: 4 Kb > > At the moment I have one locally registered gen_server that does all communication with RabbitMQ. It consumes messages and routes them to corresponding worker processes (via gproc). When worker process need to send something, it cast message to RabbitMQ process via local name. With this approach I see some problems with message consumption speed. > > What is the right way to interact with RabbitMQ server in Erlang code? Should it be more consumer processes? > > Best regards, > Max > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri May 24 19:49:20 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 24 May 2013 18:49:20 +0100 Subject: [erlang-questions] Rabbit under small load In-Reply-To: References: Message-ID: <296B1E70-57CC-4871-863C-7405E5FA5669@rabbitmq.com> On 24 May 2013, at 12:15, Jesper Louis Andersen wrote: > > 100 million messages is a different question however. Given the average message size, you are at 400 megabytes per second and hence you need 4 Gigabit of bandwidth to even hope hitting target. Apart from that, I don't think RMQ can handle that amount of load. You may need something else entirely to handle that. > I'm pretty sure what he said was that he's currently handling 100 M/s, which probably fits the "messages per second" guess, since as you point out, rabbit has not been demonstrated to go anywhere near 1millions messages per second. Is your gen_server wrapper open source btw? From bourinov@REDACTED Fri May 24 21:59:28 2013 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 24 May 2013 21:59:28 +0200 Subject: [erlang-questions] Rabbit under small load In-Reply-To: <296B1E70-57CC-4871-863C-7405E5FA5669@rabbitmq.com> References: <296B1E70-57CC-4871-863C-7405E5FA5669@rabbitmq.com> Message-ID: >> At the moment I have 100 messages per second. It is only 100-500 messages per second. Best regards, Max On Fri, May 24, 2013 at 7:49 PM, Tim Watson wrote: > On 24 May 2013, at 12:15, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > > > 100 million messages is a different question however. Given the average > message size, you are at 400 megabytes per second and hence you need 4 > Gigabit of bandwidth to even hope hitting target. Apart from that, I don't > think RMQ can handle that amount of load. You may need something else > entirely to handle that. > > > > I'm pretty sure what he said was that he's currently handling 100 M/s, > which probably fits the "messages per second" guess, since as you point > out, rabbit has not been demonstrated to go anywhere near 1millions > messages per second. > > Is your gen_server wrapper open source btw? -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Sat May 25 12:14:42 2013 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Sat, 25 May 2013 11:14:42 +0100 Subject: [erlang-questions] nif driver c code must be reentrant? In-Reply-To: <67065.1369160037@snookles.snookles.com> References: <67065.1369160037@snookles.snookles.com> Message-ID: I'm not entirely sure how this function works but the implementation of bump_reductions seems to cap the reduction bump to CONTEXT_REDS here: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/bif.c#L4628 One place I've seen reductions calculated is in sending distributed messages which suggests that the CONTEXT_REDS threshold is only specific to erlang:bump_reductions/1. Though I'm also not sure if there's a reason to bump more than 2000. https://github.com/erlang/otp/blob/maint/erts/emulator/beam/dist.c#L1847 Also, I've been wondering lately if anyone had tried erlang:yield/0 to address the NIF problem on newer Erlang versions. Most of the stuff I work on is still on R14 so I haven't had to invest much time in addressing the reductions issue but forcing a yield seems to be mostly what people are trying to accomplish with maybe a bit less than a yield for smaller operations. On Tue, May 21, 2013 at 7:13 PM, Scott Lystig Fritchie wrote: > Dan Gudmundsson wrote: > > dg> [...] > dg> erlang:bump_reductions(1000000) > dg> [...] > > Dan, that prompts a good question that has been bugging several people > (including me). Does this 1 million reduction count bump have an effect > more than a 2000 reduction bump? IIRC, a process is scheduled out after > 2000 reductions. And my (poor) memory says that the reduction count > starts at 2000, that count is decremented, and a scheduling decision is > made when the reduction count goes negative. > > Does it really make a difference if the count goes negative by a little > (e.g., only one) or by a lot (e.g., 999,990)? > > -Scott > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rajmanoj.bottle@REDACTED Sat May 25 13:50:26 2013 From: rajmanoj.bottle@REDACTED (Manoj Raj) Date: Sat, 25 May 2013 17:20:26 +0530 Subject: [erlang-questions] Mnesia Dirty select -> use of "or" in guards Message-ID: Hi, I have the following record schema saved in the table(eg) -record(details,{user,age,hobby,status}). I want to select the hobby from the mnesia table when the status is "public" or "universal" I have written the following query for testing mnesia:dirty_select(details, [ { #details{ user= "manoj", _ = '_', hobby = '$1', status='$2' }, [{'==', '$2', "public"}], ['$1'] } ]). It works fine...How should i add orelse inside the guard sequence...I want to check whether status is equal to "public" or "universal"... Is there a way or should i go the qlc way? Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From anotherworldofworld@REDACTED Sat May 25 16:39:10 2013 From: anotherworldofworld@REDACTED (Alex toyer) Date: Sat, 25 May 2013 20:39:10 +0600 Subject: [erlang-questions] [ANN] Ybot-0.3.4 released Message-ID: Hello, just released Ybot chat bot version 0.3.4. Last changes: * today plugin compatible with Mac OS X date. * fixed http application/json data handling/receiving. * #62 fixed. Security issue. Command args escaping. * Added support for notifications * New concept - Ybot channels - write-only transport. * Twitter channel (Build twitter bot with Ybot now easy - http://0xax.github.io/Build-Twitter-Bot-With-Ybot.html) * New dependence - gen_smtpc * Mail channel added. full changelog you can find - https://github.com/0xAX/Ybot/blob/master/Changelog.md Links: Ybot source code - https://github.com/0xAX/Ybot Ybot plugins - https://github.com/0xAX/ybot-contrib Ybot backend for lager - https://github.com/0xAX/lager_ybot_backend -- best regards, twitter: @0xAX github: 0xAX -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sat May 25 19:40:34 2013 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 25 May 2013 19:40:34 +0200 Subject: [erlang-questions] Mnesia Dirty select -> use of "or" in guards In-Reply-To: References: Message-ID: <5A044644-2283-47E1-80F3-31BF7C23FDBE@feuerlabs.com> Try [ {'or', {'==', '$2', "public"}, {'==', '$2', "universal"}} ] BR, Ulf W On 25 May 2013, at 13:50, Manoj Raj wrote: > Hi, > > I have the following record schema saved in the table(eg) > > -record(details,{user,age,hobby,status}). > > I want to select the hobby from the mnesia table when the status is "public" or "universal" > > I have written the following query for testing > > mnesia:dirty_select(details, > [ > { > #details{ user= "manoj", _ = '_', hobby = '$1', status='$2' }, > [{'==', '$2', "public"}], > ['$1'] > } > ]). > > It works fine...How should i add orelse inside the guard sequence...I want to check whether status is equal to "public" or "universal"... > > Is there a way or should i go the qlc way? > > Thank you > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajmanoj.bottle@REDACTED Sun May 26 15:06:18 2013 From: rajmanoj.bottle@REDACTED (Manoj Raj) Date: Sun, 26 May 2013 18:36:18 +0530 Subject: [erlang-questions] Mnesia Dirty select -> use of "or" in guards In-Reply-To: <5A044644-2283-47E1-80F3-31BF7C23FDBE@feuerlabs.com> References: <5A044644-2283-47E1-80F3-31BF7C23FDBE@feuerlabs.com> Message-ID: Thanks for the code sir...It works great... On Sat, May 25, 2013 at 11:10 PM, Ulf Wiger wrote: > > Try [ {'or', {'==', '$2', "public"}, {'==', '$2', "universal"}} ] > > BR, > Ulf W > > On 25 May 2013, at 13:50, Manoj Raj wrote: > > Hi, > > I have the following record schema saved in the table(eg) > > -record(details,{user,age,hobby,status}). > > I want to select the hobby from the mnesia table when the status is > "public" or "universal" > > I have written the following query for testing > > mnesia:dirty_select(details, > [ > { > #details{ user= "manoj", _ = '_', hobby = '$1', status='$2' }, > [{'==', '$2', "public"}], > ['$1'] > } > ]). > > It works fine...How should i add orelse inside the guard sequence...I want > to check whether status is equal to "public" or "universal"... > > Is there a way or should i go the qlc way? > > Thank you > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Sun May 26 22:46:20 2013 From: donpedrothird@REDACTED (John Doe) Date: Mon, 27 May 2013 00:46:20 +0400 Subject: [erlang-questions] Mnesia - select with placeholder in table key Message-ID: If I use select when the key is tuple with placeholders, something like this (taken from erlang docs and changed a bit): MatchHead = #person{name = {'$1', "K", "Jameson"}, sex = '$2', _ = '_'}, Result = '$2', mnesia:select(Tab,[{MatchHead, [], [Result]}]) where name is the table key will mnesia do full table scan in this case? Or it will be smarter and select all records with key matching {_, "K", "Jameson"} using index somehow? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun May 26 23:17:38 2013 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 26 May 2013 23:17:38 +0200 Subject: [erlang-questions] Mnesia - select with placeholder in table key In-Reply-To: References: Message-ID: <95637636-3AB7-451A-B5EF-453E6C1433E5@feuerlabs.com> If the key position is unbound, mnesia will check if any indexed position is bound, and if so, use the first such value for an index lookup. It then uses match_spec_run() on the result. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com 26 maj 2013 kl. 22:46 skrev John Doe : > If I use select when the key is tuple with placeholders, something like this (taken from erlang docs and changed a bit): > MatchHead = #person{name = {'$1', "K", "Jameson"}, sex = '$2', _ = '_'}, > Result = '$2', > mnesia:select(Tab,[{MatchHead, [], [Result]}]) > where name is the table key > will mnesia do full table scan in this case? Or it will be smarter and select all records with key matching {_, "K", "Jameson"} using index somehow? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf@REDACTED Mon May 27 06:44:05 2013 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 27 May 2013 06:44:05 +0200 Subject: [erlang-questions] Mnesia - select with placeholder in table key In-Reply-To: <3290be71-8ea4-49ab-b8b0-63d4b10973ad@googlegroups.com> References: <95637636-3AB7-451A-B5EF-453E6C1433E5@feuerlabs.com> <3290be71-8ea4-49ab-b8b0-63d4b10973ad@googlegroups.com> Message-ID: <4C8A11F7-AF21-4C2B-ADED-F841EFABA914@feuerlabs.com> In that case, mnesia will search the entire table. If the table is an ordered_set and the first parts of the key are bound - e.g. {a, x,'$1'} - then ets can (and does) limit the search. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com 27 maj 2013 kl. 00:33 skrev Evgeny M : > Sorry, I don't quite understand you. I'm not a native speaker, maybe I miss something. > > We select by primary key only. There are no secondary indexes in the table, and there are no filters by other fields in the query. But the primary key is a tuple and one of the elements of this tuple is wildcard, ie we have records with keys {a, x, y}, {b, x, y}, {c, x, y}, {a, z, z} and want to select all records that match {'$1', x, y}. This tuple is the key, there aresome other fields in the record. > Will the mnesia use index in this case? > > ???????????, 27 ??? 2013 ?., 1:17:38 UTC+4 ???????????? Ulf Wiger ???????: >> >> If the key position is unbound, mnesia will check if any indexed position is bound, and if so, use the first such value for an index lookup. It then uses match_spec_run() on the result. >> >> BR, >> Ulf W >> >> Ulf Wiger, Feuerlabs, Inc. >> http://www.feuerlabs.com >> >> 26 maj 2013 kl. 22:46 skrev John Doe : >> >> > If I use select when the key is tuple with placeholders, something like this (taken from erlang docs and changed a bit): >> > MatchHead = #person{name = {'$1', "K", "Jameson"}, sex = '$2', _ = '_'}, >> > Result = '$2', >> > mnesia:select(Tab,[{MatchHead, [], [Result]}]) >> > where name is the table key >> > will mnesia do full table scan in this case? Or it will be smarter and select all records with key matching {_, "K", "Jameson"} using index somehow? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-q...@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Mon May 27 00:33:02 2013 From: donpedrothird@REDACTED (Evgeny M) Date: Sun, 26 May 2013 15:33:02 -0700 (PDT) Subject: [erlang-questions] Mnesia - select with placeholder in table key In-Reply-To: <95637636-3AB7-451A-B5EF-453E6C1433E5@feuerlabs.com> References: <95637636-3AB7-451A-B5EF-453E6C1433E5@feuerlabs.com> Message-ID: <3290be71-8ea4-49ab-b8b0-63d4b10973ad@googlegroups.com> Sorry, I don't quite understand you. I'm not a native speaker, maybe I miss something. We select by primary key only. There are no secondary indexes in the table, and there are no filters by other fields in the query. But the primary key is a tuple and one of the elements of this tuple is wildcard, ie we have records with keys {a, x, y}, {b, x, y}, {c, x, y}, {a, z, z} and want to select all records that match {'$1', x, y}. This tuple is the key, there aresome other fields in the record. Will the mnesia use index in this case? ???????????, 27 ??? 2013 ?., 1:17:38 UTC+4 ???????????? Ulf Wiger ???????: > > If the key position is unbound, mnesia will check if any indexed position > is bound, and if so, use the first such value for an index lookup. It then > uses match_spec_run() on the result. > > BR, > Ulf W > > Ulf Wiger, Feuerlabs, Inc. > http://www.feuerlabs.com > > 26 maj 2013 kl. 22:46 skrev John Doe >: > > > If I use select when the key is tuple with placeholders, something like > this (taken from erlang docs and changed a bit): > > MatchHead = #person{name = {'$1', "K", "Jameson"}, sex = '$2', _ = '_'}, > > Result = '$2', > > mnesia:select(Tab,[{MatchHead, [], [Result]}]) > > where name is the table key > > will mnesia do full table scan in this case? Or it will be smarter and > select all records with key matching {_, "K", "Jameson"} using index > somehow? > > _______________________________________________ > > erlang-questions mailing list > > erlang-q...@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon May 27 10:34:07 2013 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 27 May 2013 10:34:07 +0200 Subject: [erlang-questions] Why does an idle erlang process (couchdb, to be precise) call epoll_wait so often? In-Reply-To: <20130524144153.GX15261@debjann.fritz.box> References: <20130522110928.GP15261@debjann.fritz.box> <20130522135638.GQ15261@debjann.fritz.box> <20130523085656.GT15261@debjann.fritz.box> <20130524144153.GX15261@debjann.fritz.box> Message-ID: On Fri, May 24, 2013 at 4:41 PM, Jann Horn wrote: > On Thu, May 23, 2013 at 11:13:15AM +0200, Lukas Larsson wrote: > > On Thu, May 23, 2013 at 10:56 AM, Jann Horn wrote: > > > > > On Wed, May 22, 2013 at 06:42:59PM +0200, Lukas Larsson wrote: > > > > You have to poll like that because fd's are not the only thing which > can > > > > trigger load on the system. Timeouts for instance are triggered by > > > calling > > > > gettimeofday which means you have to break out of epoll_wait before > the > > > > next timeout happens. > > > > > > Why not by specifying a timeout in the epoll_wait call? > > > > > > > If you look closely to the strace you can see that for every n:th call > > there is a small timeout given to epoll_wait. This timeout is calculated > by > > looking at the next timeout and a number of other factors. > > Does that mean that the couchdb code has set some timer that fires with a > high frequency? > Because even when epoll_wait is called with a timeout, the timeout is > below one second. > > It could mean that there is a timer, there is also a bunch of other internal emulator events which have to be taken care of which could cause the low timeout. > > > > > > > > Also the spinning is done to make the system respond > > > > faster to events by delaying sleeping in the kernel. > > > > > > Ah, ok... and that brings a performance gain? > > > > > > > It brings a latency gain, which in turn can bring a performance gain. > > Sleeping in the kernel is a (relatively) expensive thing to do and by > > spinning before sleeping the schedulers stay more responsive. You can > > configure this behaviour through the runtime flags +sbwt, +sws, +swt. See > > http://www.erlang.org/doc/man/erl.html for some more details. > > But e.g. on a single-core machine, spinning before sleeping makes no > sense, right? > So for single-core machines (VMs/mobile devices), this should be turned > off? > I'd assume so, but I haven't made any measurements for it. > Btw, what's so expensive about sleeping in the kernel? The context > switches? > Trashing the cache? > It varies depending on OS and HW combination, but those are the main things. -------------- next part -------------- An HTML attachment was scrubbed... URL: From publicityifl@REDACTED Mon May 27 11:03:51 2013 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Mon, 27 May 2013 02:03:51 -0700 (PDT) Subject: [erlang-questions] Call for Papers IFL 2013 Message-ID: <51a32177.48770e0a.266f.4910@mx.google.com> Hello, Please, find below the first call for papers for IFL 2013. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Publicity Chair of IFL CALL FOR PAPERS 25th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2013 RADBOUD UNIVERSITY NIJMEGEN, THE NETHERLANDS ACM In-Cooperation / ACM SIGPLAN AUGUST 28 - 30 2013 "Landgoed Holthurnsche Hof" http://ifl2013.cs.ru.nl We are proud to announce that the 25th edition of the IFL series returns to its roots at the Radboud University Nijmegen in the Netherlands. The symposium is held from 28th to 30th of August 2013. Scope ----- The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2013 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2013 will use a post-symposium review process to produce the formal proceedings which will be published in the ACM Digital Library. All participants of IFL 2013 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings do not count as publication for the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. Invited Speaker --------------- Lennart Augustsson, currently employed by the Standard Chartered Bank, well-known for his work on Haskell, parallel Haskell, Cayenne, and Bluespec, is the invited speaker of IFL 2013. He will be talking about practical applications of functional programming. Submission Details ------------------ Submission deadline draft papers: July 31 Notification of acceptance for presentation: August 2 Early registration deadline: August 7 Late registration deadline: August 14 Submission deadline for pre-symposium proceedings: August 21 25th IFL Symposium: August 28-30 Submission deadline for post-symposium proceedings: November 11 Notification of acceptance for post-symposium proceedings: December 18 Camera-ready version for post-symposium proceedings: February 3 2014 Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must adhere to the standard ACM two columns conference format. For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. A suitable document template for LaTeX can be found at: http://www.acm.org/sigs/sigplan/authorInformation.htm Papers are to be submitted via the conference's EasyChair submission page: https://www.easychair.org/conferences/?conf=ifl2013 Topics ------ IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2013, please contact the PC chair at rinus@REDACTED Topics of interest include, but are not limited to: ? language concepts ? type systems, type checking, type inferencing ? compilation techniques ? staged compilation ? run-time function specialization ? run-time code generation ? partial evaluation ? (abstract) interpretation ? metaprogramming ? generic programming ? automatic program generation ? array processing ? concurrent/parallel programming ? concurrent/parallel program execution ? embedded systems ? web applications ? (embedded) domain specific languages ? security ? novel memory management techniques ? run-time profiling performance measurements ? debugging and tracing ? virtual/abstract machine architectures ? validation, verification of functional programs ? tools and programming techniques ? (industrial) applications Peter Landin Prize ------------------ The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honoured article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee ------------------- ? Thomas Arts, Quviq, Gothenburg, Sweden ? Andrew Butterfield, Trinity College, Dublin, Ireland ? Edwin Brady, University of St. Andrews, UK ? Clemens Grelck, University of Amsterdam, Netherlands ? Adam Granicz, IntelliFactory, Budapest, Hungary ? Jeremy Gibbons, University of Oxford, UK ? Fritz Henglein, University of Copenhagen, Denmark ? Stephan Herhut, Intel Labs, Santa Clara, US ? Ralf Hinze (co-chair), University of Oxford, UK ? Zolt?n Horv?th, E?tv?s Lor?nd University, Budapest, Hungary ? Zhenjiang Hu, University of Tokyo, Japan ? Mauro Jaskelioff, Universidad Nacional de Rosario, Argentina ? Johan Jeuring, University of Utrecht, Netherlands ? Rita Loogen, University of Marburg, Germany ? Marco T. Moraz?n, Seton Hall University, New Jersey, US ? Dominic Orchard, University of Cambridge, UK ? Rinus Plasmeijer (chair), Radboud University Nijmegen, Netherlands ? Tim Sheard, Portland State University, US ? Sam Tobin-Hochstadt, Northeastern University / Indiana University, US ? Peter Thiemann, University of Freiburg, Germany ? Simon Thompson, University of Kent, UK Venue ----- The 25th IFL is organized by the Radboud University Nijmegen, Model Based Software Development Department at the Nijmegen Institute for Computing and Information Sciences. The event is held in the Landgoed ?Holthurnsche Hof?, a rural estate in the woodlands surrounding Nijmegen. It can be reached quickly and easily by public transport. From anotherworldofworld@REDACTED Mon May 27 12:02:03 2013 From: anotherworldofworld@REDACTED (Alex toyer) Date: Mon, 27 May 2013 16:02:03 +0600 Subject: [erlang-questions] [ANN] Ybot-0.3.4 released In-Reply-To: <8EE9B416-7ABF-44F7-A3D7-36C8E1CB24C8@erlang-solutions.com> References: <8EE9B416-7ABF-44F7-A3D7-36C8E1CB24C8@erlang-solutions.com> Message-ID: Hello Jesper, Thank you for advice, usually i wrote Ybot's description, forgot in this time. 2013/5/27 Jesper Louis Andersen > Congratulations! > > If it were me, I would have included a tagline which tells what the > project is all about in a single line. I couldn't remember and it would > have been helpful. I know I can click on a link to get it, but ? > > Happy hacking :) > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > On May 25, 2013, at 4:39 PM, Alex toyer > wrote: > > > > > Hello, > > > > just released Ybot chat bot version 0.3.4. > > > > Last changes: > > > > * today plugin compatible with Mac OS X date. > > > > * fixed http application/json data handling/receiving. > > > > * #62 fixed. Security issue. Command args escaping. > > > > * Added support for notifications > > > > * New concept - Ybot channels - write-only transport. > > > > * Twitter channel (Build twitter bot with Ybot now easy - > http://0xax.github.io/Build-Twitter-Bot-With-Ybot.html) > > > > * New dependence - gen_smtpc > > > > * Mail channel added. > > > > full changelog you can find - > https://github.com/0xAX/Ybot/blob/master/Changelog.md > > > > Links: > > > > Ybot source code - https://github.com/0xAX/Ybot > > > > Ybot plugins - https://github.com/0xAX/ybot-contrib > > > > Ybot backend for lager - https://github.com/0xAX/lager_ybot_backend > > > > > > -- > > best regards, > > > > twitter: @0xAX > > github: 0xAX > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- best regards, twitter: @0xAX github: 0xAX -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Tue May 28 00:50:26 2013 From: matthias@REDACTED (Matthias Lang) Date: Tue, 28 May 2013 00:50:26 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> References: <25431A1E-F5CF-430C-AD08-231CDF9FEEBF@gmail.com> <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> Message-ID: <20130527225026.GA20880@corelatus.se> > On May 24, 2013, at 11:13 AM, Morgan Segalis wrote: > > Is there a way to limit the memory that a VM can allocate, so the > > server is not overwhelmed in order to create a crash dump ? Sorry for the late input, I haven't been following this thread. Here's how we limit Erlang's memory use (on linux): static void set_limit_for_memory_consumption() { struct rlimit rlim; if(getrlimit(RLIMIT_AS, &rlim)) { die("getrlimit failed"); } else { rlim.rlim_cur = MAX_MEM; if(setrlimit(RLIMIT_AS, &rlim)) { die("setrlimit failed"); } syslog(LOG_INFO, "Limiting Erlang memory use to %u\n", MAX_MEM); } } works well for us. Matt From matthias@REDACTED Tue May 28 01:01:29 2013 From: matthias@REDACTED (Matthias Lang) Date: Tue, 28 May 2013 01:01:29 +0200 Subject: [erlang-questions] Investigate an infinite loop on production servers In-Reply-To: <20130527225026.GA20880@corelatus.se> References: <4DFE4400-3CF5-4D08-8271-35BB2E848E9C@gmail.com> <6EB19456-3035-4028-A685-697B358E0D53@gmail.com> <9FD04072-C0A7-4D44-A733-6E939428E131@gmail.com> <0167DA5F-7371-4998-97FC-C5FC8680B753@gmail.com> <7354A21F-7054-4EC7-8BDA-5FE9DF727F6D@gmail.com> <4D842DBF-9553-452F-A963-3A7C662E9B79@gmail.com> <6B74D39A-B15A-42F1-B5A0-9DDEF7923D3A@gmail.com> <20130527225026.GA20880@corelatus.se> Message-ID: <20130527230129.GA24050@corelatus.se> Quoting myself: > Here's how we limit Erlang's memory use (on linux): [C code cut, only really makes sense if you're starting Erlang from some sort of heartbeat system] If you just want to see this work from bash (again, on linux, but probably works on many *nixes): > erl Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] 1> lists:seq(1, 123456789). %% at this point, pressing ^C or ^G doesn't work for me, but 'kill' does. >ulimit -v 1000000 ~ >erl Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] 1> lists:seq(1, 123456789). %% a few seconds pass Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 306104640 bytes of memory (of type "heap"). Aborted Matt From 249505968@REDACTED Tue May 28 04:27:03 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Tue, 28 May 2013 10:27:03 +0800 Subject: [erlang-questions] How to use ets with a foreach way? Message-ID: Like [A+1|A<-List] How to use ets like this in efficient way? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue May 28 06:25:47 2013 From: vances@REDACTED (Vance Shipley) Date: Tue, 28 May 2013 09:55:47 +0530 Subject: [erlang-questions] How to use ets with a foreach way? In-Reply-To: References: Message-ID: <20130528042547.GB2107@aluminium.motivity.ca> On Tue, May 28, 2013 at 10:27:03AM +0800, ?? wrote: } Like [A+1|A<-List] } How to use ets like this in efficient way? ets:foldl/3 http://www.erlang.org/doc/man/ets.html#foldl-3 -- -Vance From 249505968@REDACTED Tue May 28 08:18:34 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Tue, 28 May 2013 14:18:34 +0800 Subject: [erlang-questions] =?gb18030?b?u9i4tKO6ICBIb3cgdG8gdXNlIGV0cyB3?= =?gb18030?q?ith_a_foreach_way=3F?= Message-ID: foldl with update_element? Is there any more efficient way? ------------------ ???? ------------------ ???: "Vance Shipley"; ????: 2013?5?28?(???) ??12:55 ???: "??"<249505968@REDACTED>; ??: "erlang-questions"; ??: Re: [erlang-questions] How to use ets with a foreach way? On Tue, May 28, 2013 at 10:27:03AM +0800, ?? wrote: } Like [A+1|A<-List] } How to use ets like this in efficient way? ets:foldl/3 http://www.erlang.org/doc/man/ets.html#foldl-3 -- -Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue May 28 08:35:48 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 28 May 2013 10:35:48 +0400 Subject: [erlang-questions] =?utf-8?q?+sbt_s_flag_=E2=80=94_it_is_complete?= =?utf-8?q?ly_unmanageable_now?= Message-ID: Hi. I don't know what to do with +sbt s flag now. Since R16 erlang:system_flag doesn't work. Without +sbt s Erlyvideo cannot work on more than 1 gigabit/s On some computers +sbt s with R16 simply doesn't work and _fail_ to start erlang. So, I don't understand how can I make a package that tries to set +sbt s flag on start and don't set if it is incompatible. From stas@REDACTED Tue May 28 09:49:22 2013 From: stas@REDACTED (Stanislav Sedov) Date: Tue, 28 May 2013 00:49:22 -0700 Subject: [erlang-questions] =?windows-1252?q?+sbt_s_flag_=97_it_is_complet?= =?windows-1252?q?ely_unmanageable_now?= In-Reply-To: References: Message-ID: On May 27, 2013, at 11:35 PM, Max Lapshin wrote: > Hi. > > I don't know what to do with +sbt s flag now. > > Since R16 erlang:system_flag doesn't work. Without +sbt s Erlyvideo > cannot work on more than 1 gigabit/s > > > On some computers +sbt s with R16 simply doesn't work and _fail_ to > start erlang. > > So, I don't understand how can I make a package that tries to set +sbt > s flag on start and don't set if it is incompatible What about the +stbt flag? Does that one work? -- ST4096-RIPE From vances@REDACTED Tue May 28 10:50:44 2013 From: vances@REDACTED (Vance Shipley) Date: Tue, 28 May 2013 14:20:44 +0530 Subject: [erlang-questions] How to use ets with a foreach way? In-Reply-To: References: Message-ID: <20130528085044.GA4542@aluminium.local> On Tue, May 28, 2013 at 02:18:34PM +0800, ?? wrote: } foldl with update_element? } Is there any more efficient way? Sorry, I missed the point that you want to increment each element. You can use ets:update_counter/3 to efficiently increment values: -module(ets_increment_all). -export([new/1, increment_all/1]). new(N) -> Tab = ets:new(count, []), ets:insert(Tab, [{K, 0} || K <- lists:seq(1, N)]), Tab. increment_all(Tab) -> Key = ets:first(Tab), increment_all(Tab, Key). increment_all(_Tab, '$end_of_table') -> ok; increment_all(Tab, Key) -> ets:update_counter(Tab, Key, {2, 1}), NextKey = ets:next(Tab, Key), increment_all(Tab, NextKey). 1> Tab = ets_increment_all:new(10). 16400 2> ets:i(Tab). <1 > {10,0} <2 > {9,0} <3 > {6,0} <4 > {7,0} <5 > {5,0} <6 > {3,0} <7 > {1,0} <8 > {4,0} <9 > {2,0} <10 > {8,0} EOT (q)uit (p)Digits (k)ill /Regexp -->q ok 3> ets_increment_all:increment_all(Tab). ok 4> ets:i(Tab). <1 > {10,1} <2 > {9,1} <3 > {6,1} <4 > {7,1} <5 > {5,1} <6 > {3,1} <7 > {1,1} <8 > {4,1} <9 > {2,1} <10 > {8,1} EOT (q)uit (p)Digits (k)ill /Regexp -->q ok On Tue, May 28, 2013 at 10:27:03AM +0800, ?? wrote: } Like [A+1|A<-List] } How to use ets like this in efficient way? -- -Vance From max.lapshin@REDACTED Tue May 28 12:33:45 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 28 May 2013 14:33:45 +0400 Subject: [erlang-questions] =?utf-8?q?+sbt_s_flag_=E2=80=94_it_is_complete?= =?utf-8?q?ly_unmanageable_now?= In-Reply-To: References: Message-ID: Yes, it looks like it is required to stick to R16 and use +stbt. In R15 this flag doesn't allow to start. It is a pity that this feature (erlang:system_flag) was removed without any smooth migration flag. From bengt.kleberg@REDACTED Tue May 28 13:41:41 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 28 May 2013 13:41:41 +0200 Subject: [erlang-questions] Eplot, how to produce the example? Message-ID: <1369741301.4777.10.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, Eplot (https://github.com/psyeugenic/eplot) has an example (https://github.com/psyeugenic/eplot/blob/master/example/test1.png). How was it produced? bengt From bengt.kleberg@REDACTED Tue May 28 14:26:14 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 28 May 2013 14:26:14 +0200 Subject: [erlang-questions] Eplot, how to produce the example? In-Reply-To: <1369741301.4777.10.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1369741301.4777.10.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <1369743974.4777.11.camel@sekic1152.rnd.ki.sw.ericsson.se> Found it on the Eplot page. It uses escript, not Erlang shell. bengt On Tue, 2013-05-28 at 13:41 +0200, Bengt Kleberg wrote: > Greetings, > > Eplot (https://github.com/psyeugenic/eplot) has an example > (https://github.com/psyeugenic/eplot/blob/master/example/test1.png). How > was it produced? > > > bengt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From egil@REDACTED Tue May 28 14:32:10 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Tue, 28 May 2013 14:32:10 +0200 Subject: [erlang-questions] Eplot, how to produce the example? In-Reply-To: <1369741301.4777.10.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1369741301.4777.10.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <51A4A3CA.7070304@erlang.org> On 2013-05-28 13:41, Bengt Kleberg wrote: > Greetings, > > Eplot (https://github.com/psyeugenic/eplot) has an example > (https://github.com/psyeugenic/eplot/blob/master/example/test1.png). How > was it produced? > git clone git://github.com/psyeugenic/eplot.git > cd eplot > make > bin/eplot -o example/test1.png example/data*.dat If you want to produce graphs from within erlang-code you should take a look at egd_chart.erl or eplot_main.erl. The docs are not really useful. =) // Bj?rn-Egil > > > bengt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From jon@REDACTED Tue May 28 14:45:06 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 28 May 2013 13:45:06 +0100 Subject: [erlang-questions] Tripped at first hurdle with XMPP echo_client Message-ID: <4218fb3a4ebcaa874d901a2a5be6329d.squirrel@mail.jschneider.net> I've grabbed https://github.com/processone/exmpp , modified echo_client.erl's start() with my credentials but got no_supported_auth_method. See below for complete output. Should this work ? It's a regular jabber.org account. Jon =ERROR REPORT==== 28-May-2013::13:07:05 === ** State machine <0.74.0> terminating ** Last event in was {xmlstreamelement, {xmlel,'jabber:client',[],iq, [{xmlattr,undefined,<<"type">>,<<"result">>}, {xmlattr,undefined,<<"id">>, <<"auth-xxxxxxxx">>}], [{xmlel,'jabber:iq:auth', [{'jabber:iq:auth',none}], query,[], [{xmlel,'jabber:iq:auth',[],username,[], [{xmlcdata,<<"my username here">>}]}, {xmlel,'jabber:iq:auth',[],resource,[],[]}, {xmlel,'jabber:iq:auth',[],password,[], []}]}]}} ** When State == wait_for_legacy_auth_method ** Data == {state,digest, {{jid,<<"my user name@REDACTED/exmpp#some digits">>, <<"my user name">>,<<"jabber.org">>, <<"exmpp#some digits">>}, "my password"}, {0,0}, false,false,false, [{compression,enabled},{starttls,enabled}], "jabber.org","jabber.org",<0.58.0>,exmpp_socket, {gen_tcp,#Port<0.3545>}, {xml_stream, {gen_fsm,<0.74.0>}, {xml_parser, [{max_size,infinity}, {root_depth,1}, {names_as_atom,true}, {emit_endtag,true}, {check_nss,xmpp}, {check_elems,xmpp}], #Port<0.3467>}, new,false,undefined}, "69946f127a07be75",undefined,<0.88.0>, {<0.58.0>,#Ref<0.0.0.1044>}, undefined,infinity} ** Reason for termination = ** {error,no_supported_auth_method} =ERROR REPORT==== 28-May-2013::13:07:05 === Error in process <0.58.0> on node 'reading@REDACTED' with exit value: {{nocatch,{error,no_supported_auth_method}},[{exmpp_session,login,2,[{file,"./network/exmpp_session.erl"},{line,360}]},{echo_client,session,3,[{file,"echo_client.erl"},{line,63}]}]} From magnus@REDACTED Tue May 28 14:55:14 2013 From: magnus@REDACTED (Magnus Henoch) Date: Tue, 28 May 2013 13:55:14 +0100 Subject: [erlang-questions] Tripped at first hurdle with XMPP echo_client In-Reply-To: <4218fb3a4ebcaa874d901a2a5be6329d.squirrel@mail.jschneider.net> (Jon Schneider's message of "Tue, 28 May 2013 13:45:06 +0100") References: <4218fb3a4ebcaa874d901a2a5be6329d.squirrel@mail.jschneider.net> Message-ID: "Jon Schneider" writes: > I've grabbed https://github.com/processone/exmpp , modified > echo_client.erl's start() with my credentials but got > no_supported_auth_method. See below for complete output. > > Should this work ? It's a regular jabber.org account. > > Jon > > =ERROR REPORT==== 28-May-2013::13:07:05 === > ** State machine <0.74.0> terminating > ** Last event in was {xmlstreamelement, > {xmlel,'jabber:client',[],iq, > [{xmlattr,undefined,<<"type">>,<<"result">>}, > {xmlattr,undefined,<<"id">>, > <<"auth-xxxxxxxx">>}], > [{xmlel,'jabber:iq:auth', > [{'jabber:iq:auth',none}], > query,[], > [{xmlel,'jabber:iq:auth',[],username,[], > [{xmlcdata,<<"my username here">>}]}, > {xmlel,'jabber:iq:auth',[],resource,[],[]}, > {xmlel,'jabber:iq:auth',[],password,[], > []}]}]}} The response from the server contains but not , i.e. the server only supports plain-text authentication... > ** When State == wait_for_legacy_auth_method ** Data == {state,digest, ...but the client is configured to used digest authentication. Try changing the call to exmpp_session:auth_basic_digest to exmpp_session:auth_basic. Regards, Magnus From donpedrothird@REDACTED Tue May 28 14:26:01 2013 From: donpedrothird@REDACTED (John Doe) Date: Tue, 28 May 2013 16:26:01 +0400 Subject: [erlang-questions] mnesia - secondary indexes vs separate tables Message-ID: Hello, I'm trying to do some kind of poor man's composite keys with mnesia for some running statistical data. What should be faster to read and write - one table with a few indexes or separate tables with primary key only? There will be about 1-5 millions of records, approx the equal number of reads, inserts and deletes. The queries will be mostly "check if at least one record exists where given indexed field value = ...", "insert ...", and "delete all records where indexed field value=xxx". Total of 4 fields, so either one table with 4 indexed fields, or 4 tables with key_field + counter. The table(s) will be disc_only_copy, as RAM matters. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Tue May 28 15:14:57 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 28 May 2013 16:14:57 +0300 Subject: [erlang-questions] mnesia - secondary indexes vs separate tables In-Reply-To: References: Message-ID: On Tue, May 28, 2013 at 3:26 PM, John Doe wrote: > Hello, > > I'm trying to do some kind of poor man's composite keys with mnesia for some > running statistical data. What should be faster to read and write - one > table with a few indexes or separate tables with primary key only? Some folks did some benchmarks with mnesia with a secondary index, and write performance was terrible (~40 times slowdown for inserts?). But I could not find it in the archives. Who was it? > There will be about 1-5 millions of records, approx the equal number of > reads, inserts and deletes. > The queries will be mostly "check if at least one record exists where given > indexed field value = ...", "insert ...", and "delete all records where > indexed field value=xxx". Total of 4 fields, so either one table with 4 > indexed fields, or 4 tables with key_field + counter. > The table(s) will be disc_only_copy, as RAM matters. On the other hand, it does not seem like too much work to measure both. And you will have a definite answer. -- Motiejus Jak?tys From aleksandr.vin@REDACTED Tue May 28 16:55:25 2013 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Tue, 28 May 2013 18:55:25 +0400 Subject: [erlang-questions] "'check-rpaths' detected a broken RPATH" in crypto.so Message-ID: Hello guys, I'm building an RPM for RHEL 5 x64_86 and got this error by check-rpaths: ERROR 0001: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains a standard rpath '/usr/lib64' in [/usr/lib64:/usr/loc\ al/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/ssl/l\ ib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib/ope\ nssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/lib64' in [/usr/lib64:/u\ sr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local\ /ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/l\ ib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/sfw/lib64' in [/usr/lib64:/usr\ /local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/s\ sl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib\ /openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/opt/local/lib64' in [/usr/lib64:/u\ sr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local\ /ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/l\ ib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/pkg/lib64' in [/usr/lib64:/usr\ /local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/s\ sl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib\ /openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/openssl/lib64' in [/usr/\ lib64:/usr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/u\ sr/local/ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/li\ b:/usr/lib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/ssl/lib64' in [/usr/lib6\ 4:/usr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/l\ ocal/ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/u\ sr/lib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/ssl/lib64' in [/usr/lib64:/usr\ /local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/s\ sl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib\ /openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/lib' in [/usr/lib64:/usr\ /local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/s\ sl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib\ /openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/sfw/lib' in [/usr/lib64:/usr/l\ ocal/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/ssl\ /lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib/o\ penssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0001: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains a standard rpath '/usr/lib' in [/usr/lib64:/usr/local\ /lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/ssl/lib\ 64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib/opens\ sl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/opt/local/lib' in [/usr/lib64:/usr\ /local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/s\ sl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib\ /openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/pkg/lib' in [/usr/lib64:/usr/l\ ocal/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/ssl\ /lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib/o\ penssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/openssl/lib' in [/usr/li\ b64:/usr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr\ /local/ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:\ /usr/lib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/openssl/lib' in [/usr/lib64:/u\ sr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local\ /ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/l\ ib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/local/ssl/lib' in [/usr/lib64:\ /usr/local/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/loc\ al/ssl/lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr\ /lib/openssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] ERROR 0002: file '/usr/local/lib/erlang/lib/crypto-2.3/priv/lib/crypto.so' contains an invalid rpath '/usr/ssl/lib' in [/usr/lib64:/usr/l\ ocal/lib64:/usr/sfw/lib64:/opt/local/lib64:/usr/pkg/lib64:/usr/local/openssl/lib64:/usr/lib/openssl/lib64:/usr/openssl/lib64:/usr/local/ssl\ /lib64:/usr/lib/ssl/lib64:/usr/ssl/lib64:/usr/local/lib:/usr/sfw/lib:/usr/lib:/opt/local/lib:/usr/pkg/lib:/usr/local/openssl/lib:/usr/lib/o\ penssl/lib:/usr/openssl/lib:/usr/local/ssl/lib:/usr/lib/ssl/lib:/usr/ssl/lib] I've found the specific config in otp_src_R16B/lib/crypto/c_src/x86_64-unknown-linux-gnu/Makefile: ifeq ($(DYNAMIC_CRYPTO_LIB),yes) SSL_DED_LD_RUNTIME_LIBRARY_PATH = -Wl,-R/usr/lib64 -Wl,-R/usr/local/lib64 -Wl,-R/usr/sfw/lib64 -Wl,-R/opt/local/lib64 -Wl,-R/usr/pkg/lib64 \ -Wl,-R/usr/local/openssl/lib64 -Wl,-R/usr/lib/openssl/lib64 -Wl,-R/usr/openssl/lib64 -Wl,-R/usr/local/ssl/lib64 -Wl,-R/usr/lib/ssl/lib64 -W\ l,-R/usr/ssl/lib64 -Wl,-R/usr/local/lib -Wl,-R/usr/sfw/lib -Wl,-R/usr/lib -Wl,-R/opt/local/lib -Wl,-R/usr/pkg/lib -Wl,-R/usr/local/openssl/\ lib -Wl,-R/usr/lib/openssl/lib -Wl,-R/usr/openssl/lib -Wl,-R/usr/local/ssl/lib -Wl,-R/usr/lib/ssl/lib -Wl,-R/usr/ssl/lib Is it really needed to have hardcoded RPATHS there? Will appreciate any clearance on this issue. -- Aleksandr Vinokurov +7 (921) 982-21-43 @aleksandrvin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue May 28 17:19:02 2013 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 28 May 2013 17:19:02 +0200 Subject: [erlang-questions] mnesia - secondary indexes vs separate tables In-Reply-To: References: Message-ID: <3D070AC3-01F8-4C59-A680-0494BF09FBBE@feuerlabs.com> On 28 May 2013, at 15:14, Motiejus Jak?tys wrote: > Some folks did some benchmarks with mnesia with a secondary index, and > write performance was terrible (~40 times slowdown for inserts?). But > I could not find it in the archives. Who was it? I believe it was Scott Fritchie, but it was for the special case where an inordinate number of secondary keys were of the same value (e.g. indexing a boolean attribute). Since index tables are 'bag' tables, this can indeed result in terrible performance. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From andrew.pennebaker@REDACTED Tue May 28 23:07:48 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 28 May 2013 17:07:48 -0400 Subject: [erlang-questions] Erlang package managers? Message-ID: Which package manager are we supposed to use to install Erlang packages? CEAN looks good, as well as EXPM, but they don't come with Erlang/OTP by default, and I can't seem to find a package manager repo with a working version of QuickCheck. -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Tue May 28 23:17:16 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 28 May 2013 17:17:16 -0400 Subject: [erlang-questions] Installing EUnit Message-ID: Does EUnit come with Erlang/OTP by default, or should I try to install it through CEAN, etc.? -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayson.barley@REDACTED Tue May 28 23:22:41 2013 From: jayson.barley@REDACTED (Jayson Barley) Date: Tue, 28 May 2013 14:22:41 -0700 Subject: [erlang-questions] Installing EUnit In-Reply-To: References: Message-ID: It comes with Erlang by default. You should be able to run the following tests. -module(io_lib_tests). -include_lib("eunit/include/eunit.hrl"). fread_stringint_int_test() -> String = "5", {ok, [Res], _Rest} = io_lib:fread("~1d", String), ?assertEqual(5, Res). fread_stringint_rest_test() -> String = "54321", {ok, _Ret, Res} = io_lib:fread("~1d", String), ?assertEqual("4321", Res). On Tue, May 28, 2013 at 2:17 PM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > Does EUnit come with Erlang/OTP by default, or should I try to install it > through CEAN, etc.? > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue May 28 23:42:45 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 28 May 2013 14:42:45 -0700 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: Message-ID: <51A524D5.7090204@gmail.com> QuickCheck is commercial, so you won't find a package for free, unless you try to get the Haskell version (Haskell is the free version, but Erlang is the commercial version). There currently isn't an accepted package manager, so the options may be older (without much activity) or lack features. None of the package managers currently available seem to provide compilation from source code from validated sources. However, you could always use rebar with github to get a similar effect. (A package manager that relies on only binaries means any malicious code can hide within, so using a binary package manager means trusting the binary distributor... a binary package manager is contrary to open source development because of its lack of transparency) On 05/28/2013 02:07 PM, Andrew Pennebaker wrote: > Which package manager are we supposed to use to install Erlang packages? CEAN looks good, as well as EXPM , but they don't come with Erlang/OTP by default, and I can't seem to find a package manager repo with a working version of QuickCheck. > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Wed May 29 00:05:47 2013 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Wed, 29 May 2013 00:05:47 +0200 Subject: [erlang-questions] mnesia - secondary indexes vs separate tables In-Reply-To: <3D070AC3-01F8-4C59-A680-0494BF09FBBE@feuerlabs.com> References: <3D070AC3-01F8-4C59-A680-0494BF09FBBE@feuerlabs.com> Message-ID: <51A52A3B.8040801@gmail.com> 05/28/2013 05:19 PM, Ulf Wiger wrote: > On 28 May 2013, at 15:14, Motiejus Jak?tys wrote: > >> Some folks did some benchmarks with mnesia with a secondary index, >> and write performance was terrible (~40 times slowdown for >> inserts?). But I could not find it in the archives. Who was it? > > I believe it was Scott Fritchie, but it was for the special case > where an inordinate number of secondary keys were of the same value > (e.g. indexing a boolean attribute). Since index tables are 'bag' > tables, this can indeed result in terrible performance. Indeed it was Scott, here: http://erlang.2086793.n4.nabble.com/Mnesia-and-additional-indexes-a-cautionary-tale-td2088434.html And I have done some benchmarks which confirmed crappy insert performance with secondary indices, but it was not very thorough, because I bumped into a more severe problem, see below. On 05/28/2013 02:26 PM, John Doe wrote: > The table(s) will be disc_only_copy, as RAM matters. Last time I checked (R14 I think) mnesia never stored the secondary indices, so it would recreate them every time you restarted the node, and given that you have an order of 10^6 record this will have either a significant or a very significant impact on your start time, so you might want check if that's still the case and decide whether it's a problem for you. -- Ignas From andrew.pennebaker@REDACTED Wed May 29 00:34:09 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 28 May 2013 18:34:09 -0400 Subject: [erlang-questions] How to include lib in erl interpreter? Message-ID: In a compiled Erlang program, we use -include_lib("../.hrl"). How can we include libraries in an active erl interpreter session? -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Wed May 29 00:44:52 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Wed, 29 May 2013 00:44:52 +0200 Subject: [erlang-questions] How to include lib in erl interpreter? In-Reply-To: References: Message-ID: Include for what purpose, exactly? Record definitions can be read into the shell using (I think) the "rr" command - cf. the command list provided by "help().". If it is macros you wish to use, however, I don't think the shell supports that. (Include files may contain other definitions than these two kinds, of course, but it would surprise me a bit if it's something else you need...) Den 29/05/2013 00.34 skrev "Andrew Pennebaker" : > In a compiled Erlang program, we use -include_lib("../.hrl"). How can we > include libraries in an active erl interpreter session? > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Tue May 28 23:10:18 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Tue, 28 May 2013 23:10:18 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: Message-ID: I just use rebar and ./rebar get-deps. It uses VCS commands to pull in dependencies. HTH, Aaron On Tue, May 28, 2013 at 11:07 PM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > Which package manager are we supposed to use to install Erlang packages? > CEAN looks good, as well as EXPM, > but they don't come with Erlang/OTP by default, and I can't seem to find a > package manager repo with a working version of QuickCheck. > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Wed May 29 07:17:03 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Wed, 29 May 2013 07:17:03 +0200 Subject: [erlang-questions] How to include lib in erl interpreter? In-Reply-To: References: Message-ID: You can do that - and that would give access to any exported functions defined by the header file. But not to macros or record definitions, as they are not present in the compiled module. Den 29/05/2013 04.06 skrev "???" : > According to http://www.erlang.org/doc/man/shell.html: > > *If a command (local function call) is not recognized by the shell, an > attempt is first made to find the function in the module user_default, > where customized local commands can be placed. If found, then the function > is evaluated. Otherwise, an attempt is made to evaluate the function in the > module shell_default. The module user_default must be explicitly loaded.* > * > * > So you can place a "user_default.erl" file in your project, and place " > -include_lib("../.hrl")." in the file. > > > 2013/5/29 Erik S?e S?rensen > >> Include for what purpose, exactly? >> Record definitions can be read into the shell using (I think) the "rr" >> command - cf. the command list provided by "help().". >> If it is macros you wish to use, however, I don't think the shell >> supports that. >> (Include files may contain other definitions than these two kinds, of >> course, but it would surprise me a bit if it's something else you need...) >> Den 29/05/2013 00.34 skrev "Andrew Pennebaker" < >> andrew.pennebaker@REDACTED>: >> >>> In a compiled Erlang program, we use -include_lib("../.hrl"). How can >>> we include libraries in an active erl interpreter session? >>> >>> -- >>> Cheers, >>> >>> Andrew Pennebaker >>> www.yellosoft.us >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed May 29 08:41:44 2013 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 29 May 2013 08:41:44 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: Message-ID: +1 ./rebar get-deps is enoough for me at the moment. Best regards, Max On Tue, May 28, 2013 at 11:10 PM, Aaron France wrote: > I just use rebar and ./rebar get-deps. It uses VCS commands to pull in > dependencies. > > HTH, > Aaron > > > On Tue, May 28, 2013 at 11:07 PM, Andrew Pennebaker < > andrew.pennebaker@REDACTED> wrote: > >> Which package manager are we supposed to use to install Erlang packages? >> CEAN looks good, as well as EXPM, >> but they don't come with Erlang/OTP by default, and I can't seem to find a >> package manager repo with a working version of QuickCheck. >> >> -- >> Cheers, >> >> Andrew Pennebaker >> www.yellosoft.us >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail@REDACTED Wed May 29 10:26:24 2013 From: mail@REDACTED (Dominic Williams) Date: Wed, 29 May 2013 10:26:24 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: <51A524D5.7090204@gmail.com> References: <51A524D5.7090204@gmail.com> Message-ID: Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : > QuickCheck is commercial, so you won't find a package for free, unless you try to get the Haskell version (Haskell is the free version, but Erlang is the commercial version). QuickCheck mini, which contains all the basic features, is free and can be downloaded from QuviQ's web site. Regards, Dominic Williams http://dominicwilliams.net From bog495@REDACTED Wed May 29 10:06:49 2013 From: bog495@REDACTED (bog495@REDACTED) Date: Wed, 29 May 2013 01:06:49 -0700 (PDT) Subject: [erlang-questions] R16B and ssl sockets Message-ID: Hello I have the following functions that work perfectly in R14B04: recv_frame_ssl(S, N, SslRecvTimeout) -> %% io:format("SslRecvTimeout = ~p~n", [SslRecvTimeout]), {ok, Header} = ssl:recv(S, 4, SslRecvTimeout), <> = Header, {ok, Data} = ssl:recv(S, Length - 4, SslRecvTimeout), <> = Data, {ok, binary_to_list(Data1)}. send_frame_ssl(Sock, Data) -> DataBin = unicode:characters_to_binary(Data), HeaderLen = erlang:size(DataBin) + 4, %% error_logger:info_msg("SEND-FRAME:[~p]:~n:~p:~n:~p:~n:~p~n:~p~n", [self(), Data, DataBin, HeaderLen, <>]), ssl:send(Sock, <>), ssl:send(Sock, <>). However in R16 the code fails with:{error, socket_closed_remotely} . This is because the server correctly receives and decodes the packet from the client, in recv_ssl_frame/3, but when the server must send response to the client, in function send_frame_ssl/2 , the packet is no more correctly assembled and the client closes the connection because does not receives the header which is the total length of the packet and hence does not know the length of data to be read from the socket and hence the error on the server side {error, socket_closed_remotely} . In R14B* this is working as expected, however in R15 and R16 does not. What am I missing, what should I do in order to send to the client a correctly formatted packet ? Thank you in advanced, Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Wed May 29 11:11:29 2013 From: olav@REDACTED (Olav Frengstad) Date: Wed, 29 May 2013 11:11:29 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: <51A524D5.7090204@gmail.com> Message-ID: I'v previously been using Rebar to handle source dependencies, but recently started working on an alternative solution called EPM [1] which fixes some of the issues I had with Rebar's dependency management. [1] https://github.com/lafka/epm 2013/5/29 Dominic Williams > Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : > > > QuickCheck is commercial, so you won't find a package for free, unless > you try to get the Haskell version (Haskell is the free version, but Erlang > is the commercial version). > > QuickCheck mini, which contains all the basic features, is free and can be > downloaded from QuviQ's web site. > > Regards, > Dominic Williams > http://dominicwilliams.net > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed May 29 11:29:15 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 29 May 2013 12:29:15 +0300 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: <51A524D5.7090204@gmail.com> Message-ID: <31BBC259-B4B1-447B-AA32-6B8C4D8FB001@gmail.com> Hello, Beauty of git + rebar is peer-to-peer solution. You are not bound to single repository (the concept of trust and security is other topic). Additionally rebar + reltool is very flexible to produce a "deployable tar ball". The vars overlay concept allows you to generate dev / stage and prod deployable packages following same workflow. No-doubt that concept of dependencies is not very flexible like with yum. On-another hand it is good enough to assemble relatively large VM nodes. It would be very good to get idea what issue you had with rebar on dependencies management. - Dmitry On May 29, 2013, at 12:11 PM, Olav Frengstad wrote: > I'v previously been using Rebar to handle source dependencies, but recently started working on an alternative solution called EPM [1] which fixes some of the issues I had with Rebar's dependency management. > > [1] https://github.com/lafka/epm > > > 2013/5/29 Dominic Williams > Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : > > > QuickCheck is commercial, so you won't find a package for free, unless you try to get the Haskell version (Haskell is the free version, but Erlang is the commercial version). > > QuickCheck mini, which contains all the basic features, is free and can be downloaded from QuviQ's web site. > > Regards, > Dominic Williams > http://dominicwilliams.net > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Med Vennlig Hilsen > Olav Frengstad > > Systemutvikler // FWT > +47 920 42 090 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Wed May 29 12:45:21 2013 From: olav@REDACTED (Olav Frengstad) Date: Wed, 29 May 2013 12:45:21 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: <51A524D5.7090204@gmail.com> Message-ID: @Andre: Didn't find that when searching, a rename is required from my part @Dmitry: I would call that the beauty of git, having a ./deps.sh doing the same thing would be same amount of lines. My issues with Rebar are minor things related to git, but this forces me every now and then to do manual work which interrupts my workflow. 1) updating git remotes does not work 2) tags that has been forcefully updated does not work 3) If Rebar was interrupted while fetching deps it will refuse restart the fetch because of a missing .app file 4) the little trick of calling get-deps before update-deps Ideally I want a build env where i can pick different refs for a dependency based on the environment, i.e. dev can use a different branch than dev/stage/prod/test. Rebar does not give me an easy way of doing this. All of the problems above could off course all be scripted around, but I found using other tools removed the complexity of my previous development cycle (at the cost of a Makefile and epm + relx) All in all I think Rebar is a good tool for simplifying the Erlang development cycle, but I have found it difficult to find good workflow. Olav 2013/5/29 Andr? Graf > Hello Olav > > I am confused now, are you talking of 'epm' by Jacob Vorreuter > (https://github.com/JacobVorreuter/epm) or is your 'EPM' something > different? > > Cheers, > Andre > > On 29 May 2013 11:11, Olav Frengstad wrote: > > I'v previously been using Rebar to handle source dependencies, but > recently > > started working on an alternative solution called EPM [1] which fixes > some > > of the issues I had with Rebar's dependency management. > > > > [1] https://github.com/lafka/epm > > > > > > 2013/5/29 Dominic Williams > >> > >> Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : > >> > >> > QuickCheck is commercial, so you won't find a package for free, unless > >> > you try to get the Haskell version (Haskell is the free version, but > Erlang > >> > is the commercial version). > >> > >> QuickCheck mini, which contains all the basic features, is free and can > be > >> downloaded from QuviQ's web site. > >> > >> Regards, > >> Dominic Williams > >> http://dominicwilliams.net > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > -- > > Med Vennlig Hilsen > > Olav Frengstad > > > > Systemutvikler // FWT > > +47 920 42 090 > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Wed May 29 15:58:50 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Wed, 29 May 2013 13:58:50 +0000 Subject: [erlang-questions] Configuring rebar across directories Message-ID: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> An HTML attachment was scrubbed... URL: From sven.heyll@REDACTED Wed May 29 16:09:06 2013 From: sven.heyll@REDACTED (Sven Heyll) Date: Wed, 29 May 2013 16:09:06 +0200 Subject: [erlang-questions] et_collector:load_event_file/2 documented but not exported Message-ID: Hello List, The documentation of the 'et' module advertises the existance of et_collector:load_event_file/2. Although the function exists, the export is commented out without any comment. Is that function usable, and if why is it not exported? With best regards, Sven Heyll -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed May 29 16:26:24 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 29 May 2013 10:26:24 -0400 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: <20130529142623.GA37277@ferdair.local> Hi, Usually the structure would be to have your module that stands on its own as a library application -- an OTP application that contains all the usual metadata in its .app file, except for the {mod, _} tuple, which is absent. This will let Erlang/OTP see it as an OTP application. All these applications can be put in your path if they're in a directory (say deps/ and apps/) by adding them to the ERL_LIBS env variable in any of the following ways: $ export ERL_LIBS=`pwd`/deps/:`pwd`/apps/:$ERL_LIBS $ erl $ ERL_LIBS=`pwd`/deps/:`pwd`/apps/:$ERL_LIBS erl ... $ erl -env ERL_LIBS `pwd`/deps/:`pwd`/apps/ and so on. Then the file can be accessed by any library by using -include_lib("my_library_app/include/file.hrl"). ERL_LIBS and how it works with other OTP applications is something more people should be aware of, and that thing is much simpler than hardcoding paths or whatever. Regards, Fred. On 05/29, yashgt@REDACTED wrote: >

I have
myapp
/src
myapp.erl
myapp_sup.erl
/include
and
mymodule
/src
mymodule.erl
/include
mymodule.hrl

>

As you can see, I have developed a module which is outside the structure of the myapp. If a module in myapp has to use the header files of mymodule, what do I need to do in the rebar.config of myapp?

>

Currently, when I compile myapp, it gives an error that the mymoduel.hrl cannot be found.

>

My intention is to develop mymodule as a shared module across several apps.

>

Thanks,
Yash

> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andra.dinu@REDACTED Wed May 29 13:32:25 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 29 May 2013 12:32:25 +0100 (BST) Subject: [erlang-questions] 8 free student passes left for Erlang User Conference 13-14 June Stockholm Message-ID: <394025425.131933464.1369827145749.JavaMail.root@erlang-solutions.com> Hi all, We've got 8 free student tickets left for EUC 2013: https://www.erlang-factory.com/conference/ErlangUserConference2013 As a student, you?ll get a 2-day pass for the conference, excluding dinner and drinks. All you need to do is email conferences@REDACTED from your student account and we'll e-mail you the rest of the details. Cheers, Andra -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From aaron.l.france@REDACTED Wed May 29 14:41:39 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Wed, 29 May 2013 05:41:39 -0700 (PDT) Subject: [erlang-questions] SSL certificate unknown error In-Reply-To: References: Message-ID: Hi, I'm having this issue as well, I'm not super familiar with that dbg module and have been unsuccessful in using it. Any ideas which you can give me? On Tuesday, March 12, 2013 9:48:06 AM UTC+1, Ingela Andin wrote: > > Hi! > > This error typically happens when there is a problem in the ASN-1 > decoding of a certificate. > You could try tracing on public_key:pkix_path_validation. If we can > get the input values > to that function we could create a way to reproduce the error and make > it much easier for us > to fix the problem. > > dbg:tracer(). > dbg:p(all, [call]). > dbg:tpl(public_key, pkix_path_validation, x). > > Regards Ingela Erlang/OTP- team Ericsson AB > > > 2013/3/12, Andrei Soroker >: > > On Mon, Mar 11, 2013 at 8:14 PM, Tristan Sloughter > > > wrote: > >> I ran into the same problem on R16A. I sent this to erlang-bugs > >> > >> http://erlang.org/pipermail/erlang-bugs/2013-February/003369.html > >> > >> I sadly didn't take care of continued investigation.... But if it still > >> exists in R16B I guess I'll have to now, unless you figure it out :) > > > > I reverted to R15B03-1 for the time being. I'll poke around some more > > later in the week. > > > >> > >> > >> On Mon, Mar 11, 2013 at 10:09 PM, Andrei Soroker > > > >> wrote: > >>> > >>> Hi again, > >>> > >>> I'm using lhttpc to make some calls over https. Everything worked fine > >>> with R15B03-1, but with R16B I'm seeing this: > >>> > >>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown > >>> > >>> Thanks, > >>> Andrei > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-q...@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-q...@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed May 29 17:12:04 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 29 May 2013 18:12:04 +0300 Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: <51A524D5.7090204@gmail.com> Message-ID: <00094C1C-EB90-45E4-9CE4-4B2D44538E32@gmail.com> Hello Olav, Well this is hard to find the solution that fits all possible work flows. The one thing which is solvable in nice manner is the ability to maintain different set of dependencies for different environments. I do have a dedicated file to maintain environment specific dependencies (e.g. dev.config, prod.config, etc). The file contains deps declaration same as "deps" section from rebar.config. The I am using rebar.config.script to inject those dependencies depends on env variable. The file is simple see below. The same techniques can be used to swap branches, tags etc. -- CLIP -- DEPS = fun(File, Config) -> case file:consult(File) of {ok, DEPS} -> {deps, List} = lists:keyfind(deps, 1, Config), lists:keystore(deps, 1, Config, {deps, List ++ DEPS}); _ -> Config end end, Fdeps = fun(Config) -> DEPS("deps.config", Config) end, lists:foldl( fun(X, Acc) -> X(Acc) end, CONFIG, [Fdeps] ). -- CLIP -- - Dmitry On May 29, 2013, at 1:45 PM, Olav Frengstad wrote: > @Andre: Didn't find that when searching, a rename is required from my part > > @Dmitry: > I would call that the beauty of git, having a ./deps.sh doing the same thing would be same amount of lines. > > My issues with Rebar are minor things related to git, but this forces me every now and then to do manual work which interrupts my workflow. > > 1) updating git remotes does not work > 2) tags that has been forcefully updated does not work > 3) If Rebar was interrupted while fetching deps it will refuse restart the fetch because of a missing .app file > 4) the little trick of calling get-deps before update-deps > > Ideally I want a build env where i can pick different refs for a dependency based on the environment, i.e. dev can use a different branch than dev/stage/prod/test. Rebar does not give me an easy way of doing this. > > All of the problems above could off course all be scripted around, but I found using other tools removed the complexity of my previous development cycle (at the cost of a Makefile and epm + relx) > > All in all I think Rebar is a good tool for simplifying the Erlang development cycle, but I have found it difficult to find good workflow. > > Olav > > > > 2013/5/29 Andr? Graf > Hello Olav > > I am confused now, are you talking of 'epm' by Jacob Vorreuter > (https://github.com/JacobVorreuter/epm) or is your 'EPM' something > different? > > Cheers, > Andre > > On 29 May 2013 11:11, Olav Frengstad wrote: > > I'v previously been using Rebar to handle source dependencies, but recently > > started working on an alternative solution called EPM [1] which fixes some > > of the issues I had with Rebar's dependency management. > > > > [1] https://github.com/lafka/epm > > > > > > 2013/5/29 Dominic Williams > >> > >> Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : > >> > >> > QuickCheck is commercial, so you won't find a package for free, unless > >> > you try to get the Haskell version (Haskell is the free version, but Erlang > >> > is the commercial version). > >> > >> QuickCheck mini, which contains all the basic features, is free and can be > >> downloaded from QuviQ's web site. > >> > >> Regards, > >> Dominic Williams > >> http://dominicwilliams.net > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > -- > > Med Vennlig Hilsen > > Olav Frengstad > > > > Systemutvikler // FWT > > +47 920 42 090 > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Med Vennlig Hilsen > Olav Frengstad > > Systemutvikler // FWT > +47 920 42 090 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrice.bruno@REDACTED Wed May 29 17:37:54 2013 From: patrice.bruno@REDACTED (Patrice Bruno) Date: Wed, 29 May 2013 17:37:54 +0200 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: <51A620D2.60101@kuantic.com> Hello, I use the following to do what you want to do in rebar.config % Erlang compiler options % {i,Dir} : Add Dir to the list of directories to be searched when including a file. % Allows to share .hrl files across modules without duplicate them. {erl_opts, [{i, "../dir1"}, {i, "../dir2"}, {i, "../dir3"}]}. with a structure like rebar rebar.config dir1 dir2 dir3 with dirX/src and dir/include. Hope it's help. Le 29/05/2013 15:58, yashgt@REDACTED a ?crit : > I have > myapp > /src > myapp.erl > myapp_sup.erl > /include > and > mymodule > /src > mymodule.erl > /include > mymodule.hrl > > As you can see, I have developed a module which is outside the structure > of the myapp. If a module in myapp has to use the header files of > mymodule, what do I need to do in the rebar.config of myapp? > > Currently, when I compile myapp, it gives an error that the mymoduel.hrl > cannot be found. > > My intention is to develop mymodule as a shared module across several apps. > > Thanks, > Yash > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Cordialement, Patrice Bruno Kuantic --- Le contenu de cet email est confidentiel et r?serv? aux seuls destinataires. Reproduction et diffusion interdites, sauf accord pr?alable. From andrew.pennebaker@REDACTED Thu May 30 00:34:53 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Wed, 29 May 2013 18:34:53 -0400 Subject: [erlang-questions] More file types recognized by erlang-mode Message-ID: In the next version of erlang-mode, could we automatically load erlang-mode for more types, like rebar.config files? -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Thu May 30 00:35:23 2013 From: olav@REDACTED (Olav Frengstad) Date: Thu, 30 May 2013 00:35:23 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: <00094C1C-EB90-45E4-9CE4-4B2D44538E32@gmail.com> References: <51A524D5.7090204@gmail.com> <00094C1C-EB90-45E4-9CE4-4B2D44538E32@gmail.com> Message-ID: > Well this is hard to find the solution that fits all possible work flows. I agree, there is no such thing as one-size-fits-all and speaking for myself I have, at least for my work, a very tailored development environment . > The one thing which is solvable in nice manner is the ability to maintain different set of dependencies for different environments. As i said, most things can be scripted, but i prefer to let the tool to it for me. I'v used that approach earlier to maintain separate configs, and your approach is a great example of handling dynamic dependencies. Olav 2013/5/29 Dmitry Kolesnikov > > Hello Olav, > > Well this is hard to find the solution that fits all possible work flows. > The one thing which is solvable in nice manner is the ability to maintain different set of dependencies for different environments. > > I do have a dedicated file to maintain environment specific dependencies (e.g. dev.config, prod.config, etc). The file contains deps declaration same as "deps" section from rebar.config. The I am using rebar.config.script to inject those dependencies depends on env variable. The file is simple see below. The same techniques can be used to swap branches, tags etc. > > -- CLIP -- > DEPS = fun(File, Config) -> > case file:consult(File) of > {ok, DEPS} -> > {deps, List} = lists:keyfind(deps, 1, Config), > lists:keystore(deps, 1, Config, {deps, List ++ DEPS}); > _ -> > Config > end > end, > Fdeps = fun(Config) -> DEPS("deps.config", Config) end, > > lists:foldl( > fun(X, Acc) -> X(Acc) end, > CONFIG, > [Fdeps] > ). > -- CLIP -- > > - Dmitry > > On May 29, 2013, at 1:45 PM, Olav Frengstad wrote: > > @Andre: Didn't find that when searching, a rename is required from my part > > @Dmitry: > I would call that the beauty of git, having a ./deps.sh doing the same thing would be same amount of lines. > > My issues with Rebar are minor things related to git, but this forces me every now and then to do manual work which interrupts my workflow. > > 1) updating git remotes does not work > 2) tags that has been forcefully updated does not work > 3) If Rebar was interrupted while fetching deps it will refuse restart the fetch because of a missing .app file > 4) the little trick of calling get-deps before update-deps > > Ideally I want a build env where i can pick different refs for a dependency based on the environment, i.e. dev can use a different branch than dev/stage/prod/test. Rebar does not give me an easy way of doing this. > > All of the problems above could off course all be scripted around, but I found using other tools removed the complexity of my previous development cycle (at the cost of a Makefile and epm + relx) > > All in all I think Rebar is a good tool for simplifying the Erlang development cycle, but I have found it difficult to find good workflow. > > Olav > > > > 2013/5/29 Andr? Graf >> >> Hello Olav >> >> I am confused now, are you talking of 'epm' by Jacob Vorreuter >> (https://github.com/JacobVorreuter/epm) or is your 'EPM' something >> different? >> >> Cheers, >> Andre >> >> On 29 May 2013 11:11, Olav Frengstad wrote: >> > I'v previously been using Rebar to handle source dependencies, but recently >> > started working on an alternative solution called EPM [1] which fixes some >> > of the issues I had with Rebar's dependency management. >> > >> > [1] https://github.com/lafka/epm >> > >> > >> > 2013/5/29 Dominic Williams >> >> >> >> Le 28 mai 2013 ? 23:42, Michael Truog a ?crit : >> >> >> >> > QuickCheck is commercial, so you won't find a package for free, unless >> >> > you try to get the Haskell version (Haskell is the free version, but Erlang >> >> > is the commercial version). >> >> >> >> QuickCheck mini, which contains all the basic features, is free and can be >> >> downloaded from QuviQ's web site. >> >> >> >> Regards, >> >> Dominic Williams >> >> http://dominicwilliams.net >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > >> > -- >> > Med Vennlig Hilsen >> > Olav Frengstad >> > >> > Systemutvikler // FWT >> > +47 920 42 090 >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > > -- > Med Vennlig Hilsen > Olav Frengstad > > Systemutvikler // FWT > +47 920 42 090 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 From steven.charles.davis@REDACTED Thu May 30 01:29:02 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 29 May 2013 16:29:02 -0700 (PDT) Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: Message-ID: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> I'm a bit bemused about the whole "deps" thing and am likely being naive about this issue. However, it seems to me that ERL_LIBS works pretty good for me... except where we have "deps" on particular versions. I'm throwing it out that if our apps were reliably semantically versioned (www.semver.org) and that the .app file were updated to recognize versions in {application, _, {applications, [...]}} ...where atom was "latest" and {atom, version} were some .ez or other ERL_LIBS folder... ...would that not solve a whole raft of the issues here? Interested in the responses :) /s -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Thu May 30 03:40:13 2013 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 30 May 2013 02:40:13 +0100 (BST) Subject: [erlang-questions] Mnesia - select with placeholder in table key In-Reply-To: <3290be71-8ea4-49ab-b8b0-63d4b10973ad@googlegroups.com> Message-ID: <413689887.132689084.1369878013810.JavaMail.root@erlang-solutions.com> As I read it in this case Mnesia cannot use the index as the key contains a variable and Mnesia hashes the key for indexed lookup. So in this case Mnesia must do a scan. This is the same as for ETS. Robert ----- Original Message ----- > From: "Evgeny M" > To: erlang-programming@REDACTED > Cc: erlang-questions@REDACTED > Sent: Monday, 27 May, 2013 12:33:02 AM > Subject: Re: [erlang-questions] Mnesia - select with placeholder in > table key > Sorry, I don't quite understand you. I'm not a native speaker, maybe > I miss something. > We select by primary key only. There are no secondary indexes in the > table, and there are no filters by other fields in the query. But > the primary key is a tuple and one of the elements of this tuple is > wildcard, ie we have records with keys {a, x, y}, {b, x, y}, {c, x, > y}, {a, z, z} and want to select all records that match {'$1', x, > y}. This tuple is the key, there aresome other fields in the record. > Will the mnesia use index in this case? > ???????????, 27 ??? 2013 ?., 1:17:38 UTC+4 ???????????? Ulf Wiger > ???????: > > If the key position is unbound, mnesia will check if any indexed > > position is bound, and if so, use the first such value for an index > > lookup. It then uses match_spec_run() on the result. > > > BR, > > > Ulf W > > > Ulf Wiger, Feuerlabs, Inc. > > > http://www.feuerlabs.com > > > 26 maj 2013 kl. 22:46 skrev John Doe < donped...@REDACTED >: > > > > If I use select when the key is tuple with placeholders, > > > something > > > like this (taken from erlang docs and changed a bit): > > > > MatchHead = #person{name = {'$1', "K", "Jameson"}, sex = '$2', _ > > > = > > > '_'}, > > > > Result = '$2', > > > > mnesia:select(Tab,[{MatchHead, [], [Result]}]) > > > > where name is the table key > > > > will mnesia do full table scan in this case? Or it will be > > > smarter > > > and select all records with key matching {_, "K", "Jameson"} > > > using > > > index somehow? > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-q...@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-q...@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Thu May 30 10:42:06 2013 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 30 May 2013 10:42:06 +0200 Subject: [erlang-questions] More file types recognized by erlang-mode In-Reply-To: References: Message-ID: Hello Andrew, I have this in my ~/.emacs (require 'erlang-start) (setq auto-mode-alist (append auto-mode-alist '(("\\.rel$" . erlang-mode) ("\\.app$" . erlang-mode) ("\\.appSrc$" . erlang-mode) ("\\.app.src$" . erlang-mode) ("\\.hrl$" . erlang-mode) ("\\.erl$" . erlang-mode) ("\\.yrl$" . erlang-mode)))) Just add whatever extensions you want to have as erlang specific. Lukas On Thu, May 30, 2013 at 12:34 AM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > In the next version of erlang-mode, could we automatically load > erlang-mode for more types, like rebar.config files? > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Thu May 30 13:59:01 2013 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 30 May 2013 12:59:01 +0100 (BST) Subject: [erlang-questions] More file types recognized by erlang-mode In-Reply-To: Message-ID: <250563679.133183263.1369915141351.JavaMail.root@erlang-solutions.com> An alternative is to add a comment line like %% -*- mode: erlang; -*- near the beginning of the file. With the correct comment structure of course. Robert ----- Original Message ----- > From: "Lukas Larsson" > To: "Andrew Pennebaker" > Cc: "Erlang" > Sent: Thursday, 30 May, 2013 10:42:06 AM > Subject: Re: [erlang-questions] More file types recognized by > erlang-mode > Hello Andrew, > I have this in my ~/.emacs > (require 'erlang-start) > (setq auto-mode-alist (append auto-mode-alist > '(("\\.rel$" . erlang-mode) > ("\\.app$" . erlang-mode) > ("\\.appSrc$" . erlang-mode) > ("\\.app.src$" . erlang-mode) > ("\\.hrl$" . erlang-mode) > ("\\.erl$" . erlang-mode) > ("\\.yrl$" . erlang-mode)))) > Just add whatever extensions you want to have as erlang specific. > Lukas > On Thu, May 30, 2013 at 12:34 AM, Andrew Pennebaker < > andrew.pennebaker@REDACTED > wrote: > > In the next version of erlang-mode, could we automatically load > > erlang-mode for more types, like rebar.config files? > > > -- > > > Cheers, > > > Andrew Pennebaker > > > www.yellosoft.us > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Thu May 30 15:07:12 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Thu, 30 May 2013 09:07:12 -0400 Subject: [erlang-questions] More file types recognized by erlang-mode In-Reply-To: References: Message-ID: > I have this in my ~/.emacs > > (require 'erlang-start) > (setq auto-mode-alist (append auto-mode-alist > '(("\\.rel$" . erlang-mode) > ("\\.app$" . erlang-mode) > ("\\.appSrc$" . erlang-mode) > ("\\.app.src$" . erlang-mode) > ("\\.hrl$" . erlang-mode) > ("\\.erl$" . erlang-mode) > ("\\.yrl$" . erlang-mode)))) > > Just add whatever extensions you want to have as erlang specific. > Right. What I'm saying is, we could have erlang-mode register these file types automatically the first time it's loaded, so that we no longer need to add these to .emacs. -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Thu May 30 15:54:15 2013 From: yashgt@REDACTED (Yash Ganthe) Date: Thu, 30 May 2013 19:24:15 +0530 Subject: [erlang-questions] Invoking a stored procedure fails Message-ID: Using ODBC, I am trying to invoke a stored procedure: mydb:myfunc(db:get_connection("MYDB"),1). ** exception error: no function clause matching odbc:param_query({ok,<0.66.0>}, "call sp_mysp(?) ", [{sql_integer,[1]}], infinity) (odbc.erl, line 363) >From myfunc I am making a call to odbc:param_query, passing it the connection reference, the Query string and the params. Is there something else I need to do? Thanks, Yash -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Thu May 30 16:24:04 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Thu, 30 May 2013 10:24:04 -0400 Subject: [erlang-questions] Safer, functional version of erl_tidy Message-ID: I like what erl_tidy recommends about changing my code, but I don't like how it enacts the changes automatically. Could the next version default to outputting warnings, a la hlint, splint, jshint, etc., leaving the decision of what to tidy up to the programmer? We could keep automatic tidying as a flag in erl_tidy:dir/2. Also, it would be convenient to have a shell command, maybe elint or such, so that you don't have to erl -noshell -s erl_tidy dir -s init stop. -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Thu May 30 16:41:26 2013 From: olav@REDACTED (Olav Frengstad) Date: Thu, 30 May 2013 16:41:26 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> References: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> Message-ID: Hey Steve, You are touching on something different than "how to maintain my dependencies". As i see it there are 2 cases where you need to select correct dependencies: when you run the app and when you run tests. For the first one can easily be solved if you generate a release for every run, you will then specify the version to include in reltool (i use relx from erlware which has a nice way to specify dependency versions). Otherwise your are stuck with same approach as when running tests. For the second one it depends on how you invoke your tests, I add everything in lib/* to the codepath and assume there are no conflicts. Is there a need for running applications with specific dependency versions? So far i have managed fine by adding everything. 2013/5/30 Steve Davis : > I'm a bit bemused about the whole "deps" thing and am likely being naive > about this issue. > > However, it seems to me that ERL_LIBS works pretty good for me... except > where we have "deps" on particular versions. > > I'm throwing it out that if our apps were reliably semantically versioned > (www.semver.org) and that the .app file were updated to recognize versions > in > {application, _, {applications, [...]}} > > ...where atom was "latest" and {atom, version} were some .ez or other > ERL_LIBS folder... > > ...would that not solve a whole raft of the issues here? > > Interested in the responses :) > > /s > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 From k.petrauskas@REDACTED Thu May 30 17:12:44 2013 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Thu, 30 May 2013 18:12:44 +0300 Subject: [erlang-questions] Invoking a stored procedure fails In-Reply-To: References: Message-ID: I beleave you should pass <0.66.0> instead of {ok,<0.66.0>} for the first argument. Karolis On Thu, May 30, 2013 at 4:54 PM, Yash Ganthe wrote: > Using ODBC, I am trying to invoke a stored procedure: > > mydb:myfunc(db:get_connection("MYDB"),1). > ** exception error: no function clause matching > odbc:param_query({ok,<0.66.0>}, > "call sp_mysp(?) ", > [{sql_integer,[1]}], > infinity) (odbc.erl, line 363) > > From myfunc I am making a call to odbc:param_query, passing it the > connection reference, the Query string and the params. Is there something > else I need to do? > > Thanks, > Yash > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andrew.pennebaker@REDACTED Thu May 30 17:28:32 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Thu, 30 May 2013 11:28:32 -0400 Subject: [erlang-questions] Better indentation rules for erlang-mode Message-ID: List comprehentions could be indented better in erlang-mode: Current: [ foo(Bar) || Bar <- baz() ] Desired: [ foo(Bar) || Bar <- baz() ] -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Thu May 30 17:38:04 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 30 May 2013 17:38:04 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? Message-ID: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> Hi everyone, I'm currently looking for better ways to do a clustered connected client list. On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. Each row in the ets table is for now a simple {"id", } I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. What would be in your opinion the best way to do that ? It is a messaging system, so it has to handle massive message passing through processes. Thank you for your help. Morgan. From aaron.l.france@REDACTED Thu May 30 16:43:04 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Thu, 30 May 2013 16:43:04 +0200 Subject: [erlang-questions] SSL Errors with R16B Message-ID: Hi, I'm trying to make HTTPS calls to an API. My stack trace is: SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown ** exception error: no match of right hand side value {error, {failed_connect, [{to_address, {"region-b.geo-1.identity.hpcloudsvc.com ",35357}}, {inet,[inet],{tls_alert,"certificate unknown"}}]}} in function herp_identity:login/3 (src/herp_identity.erl, line 9) I've looked at some other issues which point it to being a regression in R16B and I've reverted to R15 and the issue is non-existent. Any tips on getting around this or will it require a patch in R16B? The bug is still evident in HEAD so possibly not enough data has been gathered about it. Regards, Aaron -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Thu May 30 21:12:54 2013 From: bog495@REDACTED (bog495@REDACTED) Date: Thu, 30 May 2013 12:12:54 -0700 (PDT) Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: Message-ID: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Hello, verify the output of the command: $ openssl s_client -connect region-b.geo-1.identity.hpcloudsvc.com:35357 ogdan it seems that there is a certificate self signed. May be a solution is to disable certificate verification on peer and see what happens. Bogdan On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: > > Hi, > > I'm trying to make HTTPS calls to an API. > > My stack trace is: > > SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown > ** exception error: no match of right hand side value > {error, > {failed_connect, > [{to_address, > {"region-b.geo-1.identity.hpcloudsvc.com > ",35357}}, > {inet,[inet],{tls_alert,"certificate > unknown"}}]}} > in function herp_identity:login/3 (src/herp_identity.erl, line 9) > > I've looked at some other issues which point it to being a regression in > R16B and I've reverted to R15 and the issue is non-existent. > > Any tips on getting around this or will it require a patch in R16B? The > bug is still evident in HEAD so possibly not enough data has been gathered > about it. > > Regards, > Aaron > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Thu May 30 21:17:11 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Thu, 30 May 2013 21:17:11 +0200 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Hi, But why is it working on R15B? Is the ssl code basically broken on R15B? On Thu, May 30, 2013 at 9:12 PM, wrote: > Hello, > > verify the output of the command: > > $ openssl s_client -connect region-b.geo-1.identity.hpcloudsvc.com:35357 > ogdan > it seems that there is a certificate self signed. > > May be a solution is to disable certificate verification on peer and see > what happens. > > Bogdan > > > On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >> >> Hi, >> >> I'm trying to make HTTPS calls to an API. >> >> My stack trace is: >> >> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >> ** exception error: no match of right hand side value >> {error, >> {failed_connect, >> [{to_address, >> {"region-b.geo-1.identity.** >> hpcloudsvc.com ",35357}}, >> {inet,[inet],{tls_alert,"**certificate >> unknown"}}]}} >> in function herp_identity:login/3 (src/herp_identity.erl, line 9) >> >> I've looked at some other issues which point it to being a regression in >> R16B and I've reverted to R15 and the issue is non-existent. >> >> Any tips on getting around this or will it require a patch in R16B? The >> bug is still evident in HEAD so possibly not enough data has been gathered >> about it. >> >> Regards, >> Aaron >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Thu May 30 21:36:35 2013 From: bog495@REDACTED (bog495@REDACTED) Date: Thu, 30 May 2013 12:36:35 -0700 (PDT) Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: Message-ID: <63d22df7-a721-439c-b8fc-e7590db75f64@googlegroups.com> may be this can help : http://www.erlang.org/doc/man/ssl.html#connect-2 you may pass this ssl option to ssl:connect {verify, none} may be because the R16 ssl impl. has ither defaults, and playing with ssl options may help you. Bogdan On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: > > Hi, > > I'm trying to make HTTPS calls to an API. > > My stack trace is: > > SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown > ** exception error: no match of right hand side value > {error, > {failed_connect, > [{to_address, > {"region-b.geo-1.identity.hpcloudsvc.com > ",35357}}, > {inet,[inet],{tls_alert,"certificate > unknown"}}]}} > in function herp_identity:login/3 (src/herp_identity.erl, line 9) > > I've looked at some other issues which point it to being a regression in > R16B and I've reverted to R15 and the issue is non-existent. > > Any tips on getting around this or will it require a patch in R16B? The > bug is still evident in HEAD so possibly not enough data has been gathered > about it. > > Regards, > Aaron > -------------- next part -------------- An HTML attachment was scrubbed... URL: From schrappe.t@REDACTED Thu May 30 21:52:18 2013 From: schrappe.t@REDACTED (Timo Schrappe) Date: Thu, 30 May 2013 21:52:18 +0200 Subject: [erlang-questions] managing subprocesses Message-ID: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> Hello, I am wondering that there isn't a great way for managing and parsing subprocesses (through os:cmd/1) and their output. I spent some time with google and found erlexec which basically will do my job here. But it feels a bit cheesy to use an external module (or is it called application in erlang?) to use such a fundamental functionality. I want to manage some shell scripts which will start some e.g. daemons. What is the erly way to do that kind of things? Im specially interested in the exit code from the bash scripts. - Timo From mjtruog@REDACTED Thu May 30 22:10:01 2013 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 30 May 2013 13:10:01 -0700 Subject: [erlang-questions] managing subprocesses In-Reply-To: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> References: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> Message-ID: <51A7B219.1020501@gmail.com> If you want to do it manually, you would be using: http://www.erlang.org/doc/man/erlang.html#open_port-2 Note that you can also do something like (if you only care about the exit code): shell(Command, Arguments) -> CommandSuffix = " && echo 0 || echo 1", [Result | _] = os:cmd(lists:flatten( io_lib:format(Command ++ CommandSuffix, Arguments))), if Result =:= $0 -> ok; Result =:= $1 -> io:format("\"~s\" failed!~n", [io_lib:format(Command, Arguments)]), error end. On 05/30/2013 12:52 PM, Timo Schrappe wrote: > Hello, > > I am wondering that there isn't a great way for managing and parsing subprocesses (through os:cmd/1) and their output. > I spent some time with google and found erlexec which basically will do my job here. But it feels a bit cheesy to use an external module (or is it called application in erlang?) to use such a fundamental functionality. > > I want to manage some shell scripts which will start some e.g. daemons. > > > What is the erly way to do that kind of things? Im specially interested in the exit code from the bash scripts. > > > - Timo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From msegalis@REDACTED Thu May 30 23:57:44 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Thu, 30 May 2013 23:57:44 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> Message-ID: <9AD5AC8E-C44F-4D53-BCAD-C0E103D1DDE9@gmail.com> Hi, I have tried, I cannot seems to find how they do it. Le 30 mai 2013 ? 23:47, Antoine Koener a ?crit : > Have you looked at ejabberd ? > > > > On Thu, May 30, 2013 at 5:38 PM, Morgan Segalis wrote: > Hi everyone, > > I'm currently looking for better ways to do a clustered connected client list. > > On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. > > Each row in the ets table is for now a simple {"id", } > > I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). > > If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). > If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. > > What would be in your opinion the best way to do that ? > > It is a messaging system, so it has to handle massive message passing through processes. > > Thank you for your help. > > Morgan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri May 31 00:26:16 2013 From: bob@REDACTED (Bob Ippolito) Date: Thu, 30 May 2013 15:26:16 -0700 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> Message-ID: The best implementation depends on what properties the system needs to have. What are the guarantees that you need to make? On Thursday, May 30, 2013, Morgan Segalis wrote: > Hi everyone, > > I'm currently looking for better ways to do a clustered connected client > list. > > On every server I have an ets table, in which I insert / delete / search > the process of each client connecting to the server. > > Each row in the ets table is for now a simple {"id", } > > I have tried the gproc module from Ulf Wiger, but when a cluster goes > down, everything goes wrong? (especially if it is the elected leader). > > If a cluster goes down, other clusters should consider that every client > connected on the said cluster are actually not connected (even if is just a > simple connection error between clusters). > If it goes back online, back on the nodes() list, other clusters should > consider clients on this cluster back online. > > What would be in your opinion the best way to do that ? > > It is a messaging system, so it has to handle massive message passing > through processes. > > Thank you for your help. > > Morgan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Fri May 31 01:37:38 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 30 May 2013 16:37:38 -0700 (PDT) Subject: [erlang-questions] Erlang package managers? In-Reply-To: References: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> Message-ID: Hi Olav, I guess I'm being dense about this (if so, accept my apology and educate my ignorance), but if your (versioned) app file was allowed to specify (where needed) a version of a dependent app - then what is the distinction that remains? best, /s On Thursday, May 30, 2013 9:41:26 AM UTC-5, Olav Frengstad wrote: > > Hey Steve, > > You are touching on something different than "how to maintain my > dependencies". As i see it there are 2 cases where you need to select > correct dependencies: when you run the app and when you run tests. > > For the first one can easily be solved if you generate a release for > every run, you will then specify the version to include in reltool (i > use relx from erlware which has a nice way to specify dependency > versions). Otherwise your are stuck with same approach as when running > tests. > > For the second one it depends on how you invoke your tests, I add > everything in lib/* to the codepath and assume there are no conflicts. > > Is there a need for running applications with specific dependency > versions? So far i have managed fine by adding everything. > > > 2013/5/30 Steve Davis >: > > I'm a bit bemused about the whole "deps" thing and am likely being naive > > about this issue. > > > > However, it seems to me that ERL_LIBS works pretty good for me... except > > where we have "deps" on particular versions. > > > > I'm throwing it out that if our apps were reliably semantically > versioned > > (www.semver.org) and that the .app file were updated to recognize > versions > > in > > {application, _, {applications, [...]}} > > > > ...where atom was "latest" and {atom, version} were some .ez or other > > ERL_LIBS folder... > > > > ...would that not solve a whole raft of the issues here? > > > > Interested in the responses :) > > > > /s > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-q...@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Med Vennlig Hilsen > Olav Frengstad > > Systemutvikler // FWT > +47 920 42 090 > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri May 31 06:01:00 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 31 May 2013 05:01:00 +0100 Subject: [erlang-questions] Erlang package managers? In-Reply-To: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> References: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> Message-ID: <37FE68BE-1FE3-4A98-95E6-14C42A15C230@gmail.com> Hi Steve, On 30 May 2013, at 00:29, Steve Davis wrote: > However, it seems to me that ERL_LIBS works pretty good for me... except where we have "deps" on particular versions. > Yeah, that's the problem with ERL_LIBS - otherwise, it works great. > I'm throwing it out that if our apps were reliably semantically versioned (www.semver.org) and that the .app file were updated to recognize versions in > {application, _, {applications, [...]}} That's an OTP level change though right? I think the Olav was trying to address some of the issues around dependency handling without assuming changes to core Erlang/OTP or rebar. > > ...where atom was "latest" and {atom, version} were some .ez or other ERL_LIBS folder... > > ...would that not solve a whole raft of the issues here? > > Interested in the responses :) > Possibly it might help for some tools, but rebar doesn't use the application file currently. Even if you did get this feature into OTP, such that the code loader recognised the versions of things you're attempting to work with specifically, you wouldn't get around all the issues. What about applications where dependencies share dependencies on different major versions, i.e., you need rabbitmq-client + webmachine + mochiweb2 but also raik + webmachine + mochiweb 3 or whatever? How do you pick the right one to download, to compiler, to use? And what about dependencies that aren't packaged so that rebar can build them? Do you fork those repositories and maintain rebar support all yourself? What about needing to build multiple times against, say, multiple versions of OTP for testing. What about when a dependency has native code that needs to be built and you need to ensure that it is built for the same arch/platform as the emulator it's getting loaded into, assuming you've got multiple OTP installs? And so on... Some of the things I wrote in that blog post (that Olav references in his project's README) were intended to address, or at least think about, those kinds of issues. I've since moved back to plain old rebar, with a few plugins to customise the dependency handling, since I had neither the time nor the appetite to try and solve all the various problems. Now I'm left with annoyances, but nothing that prevents me from getting stuff done. I wish Olav the best of luck with his tool though, and will be keeping a close eye on it. Cheers, Tim From jleivent@REDACTED Fri May 31 00:17:37 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 30 May 2013 18:17:37 -0400 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? Message-ID: <51A7D001.6050707@comcast.net> I noticed that a lot of the opaque data structures don't have an is_empty predicate. What is the best way to test if one of these is empty? Preferably an O(1) test, not size(X)==0 (unless that is O(1)), because the structure might be quite large. One way I thought of is to use fold on a function that just throws: is_empty(Dict) -> catch dict:fold(fun (_,_) -> throw(false) end, true, Dict). But that is disturbingly ugly. Is there a preferred way to do this? Thanks. Jonathan Leivent From khandrish@REDACTED Fri May 31 00:36:38 2013 From: khandrish@REDACTED (Chris Hicks) Date: Thu, 30 May 2013 15:36:38 -0700 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> Message-ID: Please keep in mind that this is the result of a total of about 5 seconds of thinking. You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: > Hi everyone, > > I'm currently looking for better ways to do a clustered connected client > list. > > On every server I have an ets table, in which I insert / delete / search > the process of each client connecting to the server. > > Each row in the ets table is for now a simple {"id", } > > I have tried the gproc module from Ulf Wiger, but when a cluster goes > down, everything goes wrong? (especially if it is the elected leader). > > If a cluster goes down, other clusters should consider that every client > connected on the said cluster are actually not connected (even if is just a > simple connection error between clusters). > If it goes back online, back on the nodes() list, other clusters should > consider clients on this cluster back online. > > What would be in your opinion the best way to do that ? > > It is a messaging system, so it has to handle massive message passing > through processes. > > Thank you for your help. > > Morgan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Fri May 31 08:31:38 2013 From: rapsey@REDACTED (Rapsey) Date: Fri, 31 May 2013 08:31:38 +0200 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <51A7D001.6050707@comcast.net> References: <51A7D001.6050707@comcast.net> Message-ID: Check the implementation of those data structures. There is always a function that could tell you if it is empty or not. If size would not be a O(1) call, it would probably say calc_size or something. dict:size is O(1). On Fri, May 31, 2013 at 12:17 AM, Jonathan Leivent wrote: > I noticed that a lot of the opaque data structures don't have an is_empty > predicate. What is the best way to test if one of these is empty? > Preferably an O(1) test, not size(X)==0 (unless that is O(1)), because the > structure might be quite large. > > One way I thought of is to use fold on a function that just throws: > > is_empty(Dict) -> > catch dict:fold(fun (_,_) -> throw(false) end, true, Dict). > > But that is disturbingly ugly. Is there a preferred way to do this? > > Thanks. > Jonathan Leivent > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vances@REDACTED Fri May 31 09:03:48 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 31 May 2013 12:33:48 +0530 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <51A7D001.6050707@comcast.net> References: <51A7D001.6050707@comcast.net> Message-ID: <20130531070348.GC451@aluminium.local> On Thu, May 30, 2013 at 06:17:37PM -0400, Jonathan Leivent wrote: } I noticed that a lot of the opaque data structures don't have an } is_empty predicate. What is the best way to test if one of these } is empty? Preferably an O(1) test, not size(X)==0 (unless that is } O(1)), because the structure might be quite large. With gb_trees and gb_sets size/1 is O(1). These opaque data structures happen to be tuples where the first element is the size: 1> G1 = gb_trees:empty(). {0,nil} 2> G2 = gb_trees:insert(foo, 1, G1). {1,{foo,1,nil,nil}} 3> G3 = gb_trees:insert(bar, 2, G2). {2,{foo,1,{bar,2,nil,nil},nil}} 4> gb_sets:from_list([1,2,3,4,5]). {5,{3,{2,{1,nil,nil},nil},{5,{4,nil,nil},nil}}} I have in the past cheated by using clause heads which pattern match based on the size of the data structures (e.g.): handle_call(Request, From, State) when element(1, State) > ?MAX -> -- -Vance From bog495@REDACTED Fri May 31 09:19:48 2013 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 31 May 2013 10:19:48 +0300 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: may be the ssl stack changed a bit and some defaults parameters. you may try to pass the ssl option {verify, verify_none} to ssl:connect function and see what happens. Bogdan On Thu, May 30, 2013 at 10:17 PM, Aaron France wrote: > Hi, > > But why is it working on R15B? Is the ssl code basically broken on R15B? > > > On Thu, May 30, 2013 at 9:12 PM, wrote: > >> Hello, >> >> verify the output of the command: >> >> $ openssl s_client -connect region-b.geo-1.identity.hpcloudsvc.com:35357 >> ogdan >> it seems that there is a certificate self signed. >> >> May be a solution is to disable certificate verification on peer and see >> what happens. >> >> Bogdan >> >> >> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>> >>> Hi, >>> >>> I'm trying to make HTTPS calls to an API. >>> >>> My stack trace is: >>> >>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>> ** exception error: no match of right hand side value >>> {error, >>> {failed_connect, >>> [{to_address, >>> {"region-b.geo-1.identity.** >>> hpcloudsvc.com ",35357}}, >>> {inet,[inet],{tls_alert,"**certificate >>> unknown"}}]}} >>> in function herp_identity:login/3 (src/herp_identity.erl, line 9) >>> >>> I've looked at some other issues which point it to being a regression in >>> R16B and I've reverted to R15 and the issue is non-existent. >>> >>> Any tips on getting around this or will it require a patch in R16B? The >>> bug is still evident in HEAD so possibly not enough data has been gathered >>> about it. >>> >>> Regards, >>> Aaron >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Fri May 31 09:25:48 2013 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 31 May 2013 09:25:48 +0200 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <20130531070348.GC451@aluminium.local> References: <51A7D001.6050707@comcast.net> <20130531070348.GC451@aluminium.local> Message-ID: <51A8507C.5070303@cs.ntua.gr> On 05/31/2013 09:03 AM, Vance Shipley wrote: > On Thu, May 30, 2013 at 06:17:37PM -0400, Jonathan Leivent wrote: > } I noticed that a lot of the opaque data structures don't have an > } is_empty predicate. What is the best way to test if one of these > } is empty? Preferably an O(1) test, not size(X)==0 (unless that is > } O(1)), because the structure might be quite large. > > With gb_trees and gb_sets size/1 is O(1). These opaque data structures > happen to be tuples where the first element is the size: I am afraid this statement (relying on the structure being a tuple) breaks the opacity of the data structure. So does your call to element/2 below. > I have in the past cheated by using clause heads which pattern match > based on the size of the data structures (e.g.): > > handle_call(Request, From, State) when element(1, State)> ?MAX -> So, this cannot possibly be the proper answer to Jonathan's question. Since I've also stumbled across something analogous in the past, IMO the proper solution is to extend the API of these opaque data structures (these modules) by providing is_empty/1 functions for them. I am sure the OTP folks would welcome a patch. Kostis From aaron.l.france@REDACTED Fri May 31 09:28:30 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Fri, 31 May 2013 09:28:30 +0200 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Hi, I'm using httpc, so I tried: httpc:set_options([{socket, {verify, verify_none}}]). and I still receive the same error. That's correct? If not; let me know. Thanks! Regards, Aaron On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: > may be the ssl stack changed a bit and some defaults parameters. > > you may try to pass the ssl option {verify, verify_none} to ssl:connect > function and see what happens. > > Bogdan > > > On Thu, May 30, 2013 at 10:17 PM, Aaron France wrote: > >> Hi, >> >> But why is it working on R15B? Is the ssl code basically broken on R15B? >> >> >> On Thu, May 30, 2013 at 9:12 PM, wrote: >> >>> Hello, >>> >>> verify the output of the command: >>> >>> $ openssl s_client -connect region-b.geo-1.identity.hpcloudsvc.com:35357 >>> ogdan >>> it seems that there is a certificate self signed. >>> >>> May be a solution is to disable certificate verification on peer and see >>> what happens. >>> >>> Bogdan >>> >>> >>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>> >>>> Hi, >>>> >>>> I'm trying to make HTTPS calls to an API. >>>> >>>> My stack trace is: >>>> >>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>>> ** exception error: no match of right hand side value >>>> {error, >>>> {failed_connect, >>>> [{to_address, >>>> {"region-b.geo-1.identity.** >>>> hpcloudsvc.com >>>> ",35357}}, >>>> {inet,[inet],{tls_alert,"**certificate >>>> unknown"}}]}} >>>> in function herp_identity:login/3 (src/herp_identity.erl, line 9) >>>> >>>> I've looked at some other issues which point it to being a regression >>>> in R16B and I've reverted to R15 and the issue is non-existent. >>>> >>>> Any tips on getting around this or will it require a patch in R16B? The >>>> bug is still evident in HEAD so possibly not enough data has been gathered >>>> about it. >>>> >>>> Regards, >>>> Aaron >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri May 31 09:39:15 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 31 May 2013 09:39:15 +0200 Subject: [erlang-questions] Safer, functional version of erl_tidy In-Reply-To: References: Message-ID: <51A853A3.6030701@gmail.com> On 2013-05-30 16:24 , Andrew Pennebaker wrote: > I like what erl_tidy recommends about changing my code, but I don't like > how it enacts the changes automatically. Could the next version default > to outputting warnings, a la hlint, splint, jshint, etc., leaving the > decision of what to tidy up to the programmer? We could keep automatic > tidying as a flag in erl_tidy:dir/2. That flag already exists. Pass {test, true} (or just the atom 'test') in the options, and erl_tidy will do a dry run. > Also, it would be convenient to have a shell command, maybe elint or > such, so that you don't have to erl -noshell -s erl_tidy dir -s > init stop. I'm not sure it's used often enough to warrant making it a separate command, like e.g. dialyzer. You could easily make a shell script wrapper for it though. I haven't done any work on erl_tidy for a decade; it was mainly intended as a way to easily find (or even automatically get rid of) old deprecated idioms, such as the old-style guards. Kostis Sagonas and his students took the concept further and made "tidier", which can do a lot more than erl_tidy, and even has a graphical interface to step you through the suggested changes: http://tidier.softlab.ntua.gr:20000/tidier/getstarted It hasn't been released as open source though. /Richard From vances@REDACTED Fri May 31 10:15:20 2013 From: vances@REDACTED (Vance Shipley) Date: Fri, 31 May 2013 13:45:20 +0530 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <51A8507C.5070303@cs.ntua.gr> References: <51A7D001.6050707@comcast.net> <20130531070348.GC451@aluminium.local> <51A8507C.5070303@cs.ntua.gr> Message-ID: <20130531081520.GD451@aluminium.local> On 05/31/2013 09:03 AM, Vance Shipley wrote: } With gb_trees and gb_sets size/1 is O(1). These opaque data structures } happen to be tuples where the first element is the size: ^^^^^^^^^^^^ On Fri, May 31, 2013 at 09:25:48AM +0200, Kostis Sagonas wrote: } I am afraid this statement (relying on the structure being a tuple) } breaks the opacity of the data structure. So does your call to } element/2 below. Indeed, I made that point. My answer was in the first sentence. The rest of my post was unecessary editorializing. } Since I've also stumbled across something analogous in the past, } IMO the proper solution is to extend the API of these opaque data } structures (these modules) by providing is_empty/1 functions for } them. Only if they are bifs will it help with my (unsolicited) example. Sometimes it's just too compelling to resist taking advantage of knowing how the opaque data structures work. -- -Vance From bog495@REDACTED Fri May 31 11:02:52 2013 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 31 May 2013 12:02:52 +0300 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Tried this: $ erl Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Eshell V5.10.1 (abort with ^G) 1> inest:start(). ** exception error: undefined function inest:start/0 2> inets:start(). ok 3> ssl:start(). ok 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/"). {error,socket_closed_remotely} 5> httpc:request("https://region-b.geo-1.identity.hpcloudsvc.com:35357/"). {ok,{{"HTTP/1.1",200,"OK"}, [{"cache-control","no-cache"}, {"date","Fri, 31 May 2013 08:59:44 GMT"}, {"pragma","no-cache"}, {"server","Apache-Coyote/1.1"}, {"content-length","1080"}, {"content-type","application/json"}, {"expires","-1"}], "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n \"link\" : [ {\n \"href\" : \"https://docs.hpcloud.com/api/identity\",\n \"rel\" : \"describedby\",\n \"type\" : \"text/html\"\n } ],\n \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v2.0\",\n \"link\" : [ {\n \"href\" : \" https://docs.hpcloud.com/api/identity\",\n \"rel\" : \"describedby\",\n \"type\" : \"text/html\"\n } ],\n \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v3.0\",\n \"link\" : [ {\n \"href\" : \" https://docs.hpcloud.com/api/identity\",\n \"rel\" : \"describedby\",\n \"type\" : \"text/html\"\n } ],\n \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} 6> On Fri, May 31, 2013 at 10:28 AM, Aaron France wrote: > Hi, > > I'm using httpc, so I tried: httpc:set_options([{socket, {verify, > verify_none}}]). and I still receive the same error. > > That's correct? If not; let me know. Thanks! > > Regards, > Aaron > > > On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: > >> may be the ssl stack changed a bit and some defaults parameters. >> >> you may try to pass the ssl option {verify, verify_none} to ssl:connect >> function and see what happens. >> >> Bogdan >> >> >> On Thu, May 30, 2013 at 10:17 PM, Aaron France wrote: >> >>> Hi, >>> >>> But why is it working on R15B? Is the ssl code basically broken on R15B? >>> >>> >>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>> >>>> Hello, >>>> >>>> verify the output of the command: >>>> >>>> $ openssl s_client -connect >>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>> ogdan >>>> it seems that there is a certificate self signed. >>>> >>>> May be a solution is to disable certificate verification on peer and >>>> see what happens. >>>> >>>> Bogdan >>>> >>>> >>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>> >>>>> Hi, >>>>> >>>>> I'm trying to make HTTPS calls to an API. >>>>> >>>>> My stack trace is: >>>>> >>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>>>> ** exception error: no match of right hand side value >>>>> {error, >>>>> {failed_connect, >>>>> [{to_address, >>>>> {"region-b.geo-1.identity.** >>>>> hpcloudsvc.com >>>>> ",35357}}, >>>>> {inet,[inet],{tls_alert,"**certificate >>>>> unknown"}}]}} >>>>> in function herp_identity:login/3 (src/herp_identity.erl, line 9) >>>>> >>>>> I've looked at some other issues which point it to being a regression >>>>> in R16B and I've reverted to R15 and the issue is non-existent. >>>>> >>>>> Any tips on getting around this or will it require a patch in R16B? >>>>> The bug is still evident in HEAD so possibly not enough data has been >>>>> gathered about it. >>>>> >>>>> Regards, >>>>> Aaron >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Fri May 31 11:17:46 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Fri, 31 May 2013 11:17:46 +0200 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: What's this telling you? On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: > > Tried this: > > $ erl > Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [async-threads:10] > [kernel-poll:false] > > > Eshell V5.10.1 (abort with ^G) > 1> inest:start(). > ** exception error: undefined function inest:start/0 > 2> inets:start(). > ok > 3> ssl:start(). > ok > 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/"). > {error,socket_closed_remotely} > 5> httpc:request("https://region-b.geo-1.identity.hpcloudsvc.com:35357/"). > > {ok,{{"HTTP/1.1",200,"OK"}, > [{"cache-control","no-cache"}, > {"date","Fri, 31 May 2013 08:59:44 GMT"}, > > {"pragma","no-cache"}, > {"server","Apache-Coyote/1.1"}, > {"content-length","1080"}, > {"content-type","application/json"}, > {"expires","-1"}], > "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n > \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : > {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : > \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n > \"link\" : [ {\n \"href\" : \" > https://docs.hpcloud.com/api/identity\",\n \"rel\" : > \"describedby\",\n \"type\" : \"text/html\"\n } ],\n > \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n > \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : > \"v2.0\",\n \"link\" : [ {\n \"href\" : \" > https://docs.hpcloud.com/api/identity\",\n \"rel\" : > \"describedby\",\n \"type\" : \"text/html\"\n } ],\n > \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n > \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : > \"v3.0\",\n \"link\" : [ {\n \"href\" : \" > https://docs.hpcloud.com/api/identity\",\n \"rel\" : > \"describedby\",\n \"type\" : \"text/html\"\n } ],\n > \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n > \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} > 6> > > > > On Fri, May 31, 2013 at 10:28 AM, Aaron France wrote: > >> Hi, >> >> I'm using httpc, so I tried: httpc:set_options([{socket, {verify, >> verify_none}}]). and I still receive the same error. >> >> That's correct? If not; let me know. Thanks! >> >> Regards, >> Aaron >> >> >> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >> >>> may be the ssl stack changed a bit and some defaults parameters. >>> >>> you may try to pass the ssl option {verify, verify_none} to ssl:connect >>> function and see what happens. >>> >>> Bogdan >>> >>> >>> On Thu, May 30, 2013 at 10:17 PM, Aaron France >> > wrote: >>> >>>> Hi, >>>> >>>> But why is it working on R15B? Is the ssl code basically broken on R15B? >>>> >>>> >>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>> >>>>> Hello, >>>>> >>>>> verify the output of the command: >>>>> >>>>> $ openssl s_client -connect >>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>> ogdan >>>>> it seems that there is a certificate self signed. >>>>> >>>>> May be a solution is to disable certificate verification on peer and >>>>> see what happens. >>>>> >>>>> Bogdan >>>>> >>>>> >>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm trying to make HTTPS calls to an API. >>>>>> >>>>>> My stack trace is: >>>>>> >>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>>>>> ** exception error: no match of right hand side value >>>>>> {error, >>>>>> {failed_connect, >>>>>> [{to_address, >>>>>> {"region-b.geo-1.identity.** >>>>>> hpcloudsvc.com >>>>>> ",35357}}, >>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>> unknown"}}]}} >>>>>> in function herp_identity:login/3 (src/herp_identity.erl, line >>>>>> 9) >>>>>> >>>>>> I've looked at some other issues which point it to being a regression >>>>>> in R16B and I've reverted to R15 and the issue is non-existent. >>>>>> >>>>>> Any tips on getting around this or will it require a patch in R16B? >>>>>> The bug is still evident in HEAD so possibly not enough data has been >>>>>> gathered about it. >>>>>> >>>>>> Regards, >>>>>> Aaron >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Fri May 31 11:25:03 2013 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 31 May 2013 12:25:03 +0300 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: it works! On Fri, May 31, 2013 at 12:17 PM, Aaron France wrote: > What's this telling you? > > > On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: > >> >> Tried this: >> >> $ erl >> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [async-threads:10] >> [kernel-poll:false] >> >> >> Eshell V5.10.1 (abort with ^G) >> 1> inest:start(). >> ** exception error: undefined function inest:start/0 >> 2> inets:start(). >> ok >> 3> ssl:start(). >> ok >> 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >> {error,socket_closed_remotely} >> 5> httpc:request("https://region-b.geo-1.identity.hpcloudsvc.com:35357/ >> "). >> >> {ok,{{"HTTP/1.1",200,"OK"}, >> [{"cache-control","no-cache"}, >> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >> >> {"pragma","no-cache"}, >> {"server","Apache-Coyote/1.1"}, >> {"content-length","1080"}, >> {"content-type","application/json"}, >> {"expires","-1"}], >> "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n >> \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : >> {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : >> \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n >> \"link\" : [ {\n \"href\" : \" >> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >> 6> >> >> >> >> On Fri, May 31, 2013 at 10:28 AM, Aaron France wrote: >> >>> Hi, >>> >>> I'm using httpc, so I tried: httpc:set_options([{socket, {verify, >>> verify_none}}]). and I still receive the same error. >>> >>> That's correct? If not; let me know. Thanks! >>> >>> Regards, >>> Aaron >>> >>> >>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>> >>>> may be the ssl stack changed a bit and some defaults parameters. >>>> >>>> you may try to pass the ssl option {verify, verify_none} to >>>> ssl:connect function and see what happens. >>>> >>>> Bogdan >>>> >>>> >>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>> aaron.l.france@REDACTED> wrote: >>>> >>>>> Hi, >>>>> >>>>> But why is it working on R15B? Is the ssl code basically broken on >>>>> R15B? >>>>> >>>>> >>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> verify the output of the command: >>>>>> >>>>>> $ openssl s_client -connect >>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>> ogdan >>>>>> it seems that there is a certificate self signed. >>>>>> >>>>>> May be a solution is to disable certificate verification on peer and >>>>>> see what happens. >>>>>> >>>>>> Bogdan >>>>>> >>>>>> >>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>> >>>>>>> My stack trace is: >>>>>>> >>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>>>>>> ** exception error: no match of right hand side value >>>>>>> {error, >>>>>>> {failed_connect, >>>>>>> [{to_address, >>>>>>> {"region-b.geo-1.identity.** >>>>>>> hpcloudsvc.com >>>>>>> ",35357}}, >>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>> unknown"}}]}} >>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, line >>>>>>> 9) >>>>>>> >>>>>>> I've looked at some other issues which point it to being a >>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>> >>>>>>> Any tips on getting around this or will it require a patch in R16B? >>>>>>> The bug is still evident in HEAD so possibly not enough data has been >>>>>>> gathered about it. >>>>>>> >>>>>>> Regards, >>>>>>> Aaron >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Fri May 31 11:30:23 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Fri, 31 May 2013 11:30:23 +0200 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Right, but it doesn't work for me. It's also an endpoint where you need to post/get w/ specific bodies, perhaps that's getting in the way? On Fri, May 31, 2013 at 11:25 AM, Bogdan Andu wrote: > it works! > > > On Fri, May 31, 2013 at 12:17 PM, Aaron France wrote: > >> What's this telling you? >> >> >> On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: >> >>> >>> Tried this: >>> >>> $ erl >>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [async-threads:10] >>> [kernel-poll:false] >>> >>> >>> Eshell V5.10.1 (abort with ^G) >>> 1> inest:start(). >>> ** exception error: undefined function inest:start/0 >>> 2> inets:start(). >>> ok >>> 3> ssl:start(). >>> ok >>> 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/ >>> "). >>> {error,socket_closed_remotely} >>> 5> httpc:request("https://region-b.geo-1.identity.hpcloudsvc.com:35357/ >>> "). >>> >>> {ok,{{"HTTP/1.1",200,"OK"}, >>> [{"cache-control","no-cache"}, >>> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >>> >>> {"pragma","no-cache"}, >>> {"server","Apache-Coyote/1.1"}, >>> {"content-length","1080"}, >>> {"content-type","application/json"}, >>> {"expires","-1"}], >>> "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n >>> \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : >>> {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : >>> \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n >>> \"link\" : [ {\n \"href\" : \" >>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >>> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >>> 6> >>> >>> >>> >>> On Fri, May 31, 2013 at 10:28 AM, Aaron France >> > wrote: >>> >>>> Hi, >>>> >>>> I'm using httpc, so I tried: httpc:set_options([{socket, {verify, >>>> verify_none}}]). and I still receive the same error. >>>> >>>> That's correct? If not; let me know. Thanks! >>>> >>>> Regards, >>>> Aaron >>>> >>>> >>>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>>> >>>>> may be the ssl stack changed a bit and some defaults parameters. >>>>> >>>>> you may try to pass the ssl option {verify, verify_none} to >>>>> ssl:connect function and see what happens. >>>>> >>>>> Bogdan >>>>> >>>>> >>>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>>> aaron.l.france@REDACTED> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> But why is it working on R15B? Is the ssl code basically broken on >>>>>> R15B? >>>>>> >>>>>> >>>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> verify the output of the command: >>>>>>> >>>>>>> $ openssl s_client -connect >>>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>>> ogdan >>>>>>> it seems that there is a certificate self signed. >>>>>>> >>>>>>> May be a solution is to disable certificate verification on peer and >>>>>>> see what happens. >>>>>>> >>>>>>> Bogdan >>>>>>> >>>>>>> >>>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>>> >>>>>>>> My stack trace is: >>>>>>>> >>>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>>>>>>> ** exception error: no match of right hand side value >>>>>>>> {error, >>>>>>>> {failed_connect, >>>>>>>> [{to_address, >>>>>>>> {"region-b.geo-1.identity.** >>>>>>>> hpcloudsvc.com >>>>>>>> ",35357}}, >>>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>>> unknown"}}]}} >>>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, >>>>>>>> line 9) >>>>>>>> >>>>>>>> I've looked at some other issues which point it to being a >>>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>>> >>>>>>>> Any tips on getting around this or will it require a patch in R16B? >>>>>>>> The bug is still evident in HEAD so possibly not enough data has been >>>>>>>> gathered about it. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Aaron >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Fri May 31 11:47:14 2013 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 31 May 2013 12:47:14 +0300 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: what does tit mean "it doesn't work for me"? you can POST a Body of data to an Url like this: httpc:request(post, {"https://region-b.geo-1.identity.hpcloudsvc.com:35357/ ", [{"Content-Length", ContentLength}], ContentType, Body}, [], []). you have to make the Body and compute the length of the Body yourself. Best Regards and good luck Bogdan On Fri, May 31, 2013 at 12:30 PM, Aaron France wrote: > Right, but it doesn't work for me. It's also an endpoint where you need to > post/get w/ specific bodies, perhaps that's getting in the way? > > > On Fri, May 31, 2013 at 11:25 AM, Bogdan Andu wrote: > >> it works! >> >> >> On Fri, May 31, 2013 at 12:17 PM, Aaron France wrote: >> >>> What's this telling you? >>> >>> >>> On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: >>> >>>> >>>> Tried this: >>>> >>>> $ erl >>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] >>>> [async-threads:10] [kernel-poll:false] >>>> >>>> >>>> Eshell V5.10.1 (abort with ^G) >>>> 1> inest:start(). >>>> ** exception error: undefined function inest:start/0 >>>> 2> inets:start(). >>>> ok >>>> 3> ssl:start(). >>>> ok >>>> 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/ >>>> "). >>>> {error,socket_closed_remotely} >>>> 5> httpc:request("https://region-b.geo-1.identity.hpcloudsvc.com:35357/ >>>> "). >>>> >>>> {ok,{{"HTTP/1.1",200,"OK"}, >>>> [{"cache-control","no-cache"}, >>>> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >>>> >>>> {"pragma","no-cache"}, >>>> {"server","Apache-Coyote/1.1"}, >>>> {"content-length","1080"}, >>>> {"content-type","application/json"}, >>>> {"expires","-1"}], >>>> "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n >>>> \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : >>>> {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : >>>> \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n >>>> \"link\" : [ {\n \"href\" : \" >>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >>>> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >>>> 6> >>>> >>>> >>>> >>>> On Fri, May 31, 2013 at 10:28 AM, Aaron France < >>>> aaron.l.france@REDACTED> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm using httpc, so I tried: httpc:set_options([{socket, {verify, >>>>> verify_none}}]). and I still receive the same error. >>>>> >>>>> That's correct? If not; let me know. Thanks! >>>>> >>>>> Regards, >>>>> Aaron >>>>> >>>>> >>>>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>>>> >>>>>> may be the ssl stack changed a bit and some defaults parameters. >>>>>> >>>>>> you may try to pass the ssl option {verify, verify_none} to >>>>>> ssl:connect function and see what happens. >>>>>> >>>>>> Bogdan >>>>>> >>>>>> >>>>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>>>> aaron.l.france@REDACTED> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> But why is it working on R15B? Is the ssl code basically broken on >>>>>>> R15B? >>>>>>> >>>>>>> >>>>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> verify the output of the command: >>>>>>>> >>>>>>>> $ openssl s_client -connect >>>>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>>>> ogdan >>>>>>>> it seems that there is a certificate self signed. >>>>>>>> >>>>>>>> May be a solution is to disable certificate verification on peer >>>>>>>> and see what happens. >>>>>>>> >>>>>>>> Bogdan >>>>>>>> >>>>>>>> >>>>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>>>> >>>>>>>>> My stack trace is: >>>>>>>>> >>>>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate >>>>>>>>> unknown >>>>>>>>> ** exception error: no match of right hand side value >>>>>>>>> {error, >>>>>>>>> {failed_connect, >>>>>>>>> [{to_address, >>>>>>>>> {"region-b.geo-1.identity.** >>>>>>>>> hpcloudsvc.com >>>>>>>>> ",35357}}, >>>>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>>>> unknown"}}]}} >>>>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, >>>>>>>>> line 9) >>>>>>>>> >>>>>>>>> I've looked at some other issues which point it to being a >>>>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>>>> >>>>>>>>> Any tips on getting around this or will it require a patch in >>>>>>>>> R16B? The bug is still evident in HEAD so possibly not enough data has been >>>>>>>>> gathered about it. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Aaron >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Fri May 31 11:48:38 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Fri, 31 May 2013 11:48:38 +0200 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Hi, The code works fine on 15B, I know how to use the post. Please read what I am typing :) When I say "it doesn't work", I of course mean the error in the OP. Regards, Aaron On Fri, May 31, 2013 at 11:47 AM, Bogdan Andu wrote: > what does tit mean "it doesn't work for me"? > > you can POST a Body of data to an Url like this: > > httpc:request(post, {" > https://region-b.geo-1.identity.hpcloudsvc.com:35357/", > [{"Content-Length", ContentLength}], > ContentType, > Body}, > [], []). > > you have to make the Body and compute the length of the Body yourself. > > Best Regards and good luck > > Bogdan > > > On Fri, May 31, 2013 at 12:30 PM, Aaron France wrote: > >> Right, but it doesn't work for me. It's also an endpoint where you need >> to post/get w/ specific bodies, perhaps that's getting in the way? >> >> >> On Fri, May 31, 2013 at 11:25 AM, Bogdan Andu wrote: >> >>> it works! >>> >>> >>> On Fri, May 31, 2013 at 12:17 PM, Aaron France >> > wrote: >>> >>>> What's this telling you? >>>> >>>> >>>> On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: >>>> >>>>> >>>>> Tried this: >>>>> >>>>> $ erl >>>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] >>>>> [async-threads:10] [kernel-poll:false] >>>>> >>>>> >>>>> Eshell V5.10.1 (abort with ^G) >>>>> 1> inest:start(). >>>>> ** exception error: undefined function inest:start/0 >>>>> 2> inets:start(). >>>>> ok >>>>> 3> ssl:start(). >>>>> ok >>>>> 4> httpc:request("http://region-b.geo-1.identity.hpcloudsvc.com:35357/ >>>>> "). >>>>> {error,socket_closed_remotely} >>>>> 5> httpc:request(" >>>>> https://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >>>>> >>>>> {ok,{{"HTTP/1.1",200,"OK"}, >>>>> [{"cache-control","no-cache"}, >>>>> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >>>>> >>>>> {"pragma","no-cache"}, >>>>> {"server","Apache-Coyote/1.1"}, >>>>> {"content-length","1080"}, >>>>> {"content-type","application/json"}, >>>>> {"expires","-1"}], >>>>> "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n >>>>> \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : >>>>> {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : >>>>> \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n >>>>> \"link\" : [ {\n \"href\" : \" >>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >>>>> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >>>>> 6> >>>>> >>>>> >>>>> >>>>> On Fri, May 31, 2013 at 10:28 AM, Aaron France < >>>>> aaron.l.france@REDACTED> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm using httpc, so I tried: httpc:set_options([{socket, {verify, >>>>>> verify_none}}]). and I still receive the same error. >>>>>> >>>>>> That's correct? If not; let me know. Thanks! >>>>>> >>>>>> Regards, >>>>>> Aaron >>>>>> >>>>>> >>>>>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>>>>> >>>>>>> may be the ssl stack changed a bit and some defaults parameters. >>>>>>> >>>>>>> you may try to pass the ssl option {verify, verify_none} to >>>>>>> ssl:connect function and see what happens. >>>>>>> >>>>>>> Bogdan >>>>>>> >>>>>>> >>>>>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>>>>> aaron.l.france@REDACTED> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> But why is it working on R15B? Is the ssl code basically broken on >>>>>>>> R15B? >>>>>>>> >>>>>>>> >>>>>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> verify the output of the command: >>>>>>>>> >>>>>>>>> $ openssl s_client -connect >>>>>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>>>>> ogdan >>>>>>>>> it seems that there is a certificate self signed. >>>>>>>>> >>>>>>>>> May be a solution is to disable certificate verification on peer >>>>>>>>> and see what happens. >>>>>>>>> >>>>>>>>> Bogdan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>>>>> >>>>>>>>>> My stack trace is: >>>>>>>>>> >>>>>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate >>>>>>>>>> unknown >>>>>>>>>> ** exception error: no match of right hand side value >>>>>>>>>> {error, >>>>>>>>>> {failed_connect, >>>>>>>>>> [{to_address, >>>>>>>>>> {"region-b.geo-1.identity.** >>>>>>>>>> hpcloudsvc.com >>>>>>>>>> ",35357}}, >>>>>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>>>>> unknown"}}]}} >>>>>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, >>>>>>>>>> line 9) >>>>>>>>>> >>>>>>>>>> I've looked at some other issues which point it to being a >>>>>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>>>>> >>>>>>>>>> Any tips on getting around this or will it require a patch in >>>>>>>>>> R16B? The bug is still evident in HEAD so possibly not enough data has been >>>>>>>>>> gathered about it. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Aaron >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> erlang-questions mailing list >>>>>>>>> erlang-questions@REDACTED >>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri May 31 12:04:06 2013 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Fri, 31 May 2013 12:04:06 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? Message-ID: <779C4DE0-1A6C-4A5D-8999-A4141C2B340E@gmail.com> I've been meaning to open source this library. Now is a good of a time as any I guess. Maybe it fits your use case. https://github.com/biokoda/distreg Sergej Jurecko -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 31 12:12:55 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 31 May 2013 12:12:55 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> Message-ID: <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> Hi, Actually right now, I have this implementation : Each node has a clientpool Each client pool manage an ets table, where all client are inserted / removed / searched If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. Advantage of this implementation : no synchronization required if one node goes down and back up again? Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : > Please keep in mind that this is the result of a total of about 5 seconds of thinking. > > You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. > > Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. > > > On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: > Hi everyone, > > I'm currently looking for better ways to do a clustered connected client list. > > On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. > > Each row in the ets table is for now a simple {"id", } > > I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). > > If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). > If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. > > What would be in your opinion the best way to do that ? > > It is a messaging system, so it has to handle massive message passing through processes. > > Thank you for your help. > > Morgan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 31 12:16:13 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 31 May 2013 12:16:13 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <779C4DE0-1A6C-4A5D-8999-A4141C2B340E@gmail.com> References: <779C4DE0-1A6C-4A5D-8999-A4141C2B340E@gmail.com> Message-ID: <5A32460F-C2CF-4873-B588-63CE1E5A1083@gmail.com> Hi Sergej, Thank you for open source your work !! In your read me, when you said : "When you need to call a worker on another node, distreg will determine which node it should be running on and call it on that node." How does it will determine in which node the process is, exactly ? Le 31 mai 2013 ? 12:04, Sergej Jurecko a ?crit : > I've been meaning to open source this library. Now is a good of a time as any I guess. Maybe it fits your use case. > > https://github.com/biokoda/distreg > > > > Sergej Jurecko -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri May 31 12:20:29 2013 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Fri, 31 May 2013 12:20:29 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <5A32460F-C2CF-4873-B588-63CE1E5A1083@gmail.com> References: <779C4DE0-1A6C-4A5D-8999-A4141C2B340E@gmail.com> <5A32460F-C2CF-4873-B588-63CE1E5A1083@gmail.com> Message-ID: <6D1BE30F-59DC-4E85-BC49-7492DB2A7487@gmail.com> On May 31, 2013, at 12:16 PM, Morgan Segalis wrote: > Hi Sergej, > > Thank you for open source your work !! > > In your read me, when you said : "When you need to call a worker on another node, distreg will determine which node it should be running on and call it on that node." > > How does it will determine in which node the process is, exactly ? > Every worker has a name. Workers are spread throughout the cluster using consistent hashing. So it hashes the worker name, which results in the right node that the worker should be running on. It can happen that the worker is not actually running on that node (because nodes can go online/offline), but in that case the node will know the PID of the worker at least. > Le 31 mai 2013 ? 12:04, Sergej Jurecko a ?crit : > >> I've been meaning to open source this library. Now is a good of a time as any I guess. Maybe it fits your use case. >> >> https://github.com/biokoda/distreg >> >> >> >> Sergej Jurecko > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 31 12:30:40 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 31 May 2013 13:30:40 +0300 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> Message-ID: Hello, You current implementation starts to suffer from performance due to large number of messages to discover process location. You have to define a formal rule about "id" and its relation to node where processes exists. Essentially I am talking about consistent hashing. To be honest, I am not getting what is wrong with ETS and gproc. I am using a similar approach for my cluster management. I am using P2P methodology where local tables get sync periodically + updates on local table is replicated to cluster members. Each node is capable to observe the status of cluster members. Once node is disconnected the table is clean-up. However, I am using that approach for "internal" processes. "Client connection" are not distributed globally. Best Regards, Dmitry On May 31, 2013, at 1:12 PM, Morgan Segalis wrote: > Hi, > > Actually right now, I have this implementation : > > Each node has a clientpool > Each client pool manage an ets table, where all client are inserted / removed / searched > > If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. > > Advantage of this implementation : no synchronization required if one node goes down and back up again? > Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec > > > Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : > >> Please keep in mind that this is the result of a total of about 5 seconds of thinking. >> >> You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. >> >> Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. >> >> >> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: >> Hi everyone, >> >> I'm currently looking for better ways to do a clustered connected client list. >> >> On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. >> >> Each row in the ets table is for now a simple {"id", } >> >> I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). >> >> If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). >> If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. >> >> What would be in your opinion the best way to do that ? >> >> It is a messaging system, so it has to handle massive message passing through processes. >> >> Thank you for your help. >> >> Morgan. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 31 12:52:25 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 31 May 2013 12:52:25 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> Message-ID: <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> Hi Dmitry, I have though about consistent hashing, The only issue is that consistent hashing will work if we have a fixed number of cluster, If we add dynamically another cluster, the hash won't gives me the same cluster? I might be wrong? Actually right now I have a gateway, which will choose the cluster on which there is the less number of connected clients, and redirect the client to this one. Working like a load balancer among other things the gateway does. Best regards, Morgan. Le 31 mai 2013 ? 12:30, Dmitry Kolesnikov a ?crit : > Hello, > > You current implementation starts to suffer from performance due to large number of messages to discover process location. > You have to define a formal rule about "id" and its relation to node where processes exists. Essentially I am talking about consistent hashing. > > To be honest, I am not getting what is wrong with ETS and gproc. I am using a similar approach for my cluster management. I am using P2P methodology > where local tables get sync periodically + updates on local table is replicated to cluster members. Each node is capable to observe the status of cluster members. Once node is disconnected the table is clean-up. However, I am using that approach for "internal" processes. "Client connection" are not distributed globally. > > Best Regards, > Dmitry > > On May 31, 2013, at 1:12 PM, Morgan Segalis wrote: > >> Hi, >> >> Actually right now, I have this implementation : >> >> Each node has a clientpool >> Each client pool manage an ets table, where all client are inserted / removed / searched >> >> If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. >> >> Advantage of this implementation : no synchronization required if one node goes down and back up again? >> Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec >> >> >> Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : >> >>> Please keep in mind that this is the result of a total of about 5 seconds of thinking. >>> >>> You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. >>> >>> Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. >>> >>> >>> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: >>> Hi everyone, >>> >>> I'm currently looking for better ways to do a clustered connected client list. >>> >>> On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. >>> >>> Each row in the ets table is for now a simple {"id", } >>> >>> I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). >>> >>> If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). >>> If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. >>> >>> What would be in your opinion the best way to do that ? >>> >>> It is a messaging system, so it has to handle massive message passing through processes. >>> >>> Thank you for your help. >>> >>> Morgan. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 31 13:01:51 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 31 May 2013 14:01:51 +0300 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> Message-ID: <347E399F-F486-4E4D-A2E8-45E248B7C3C4@gmail.com> Hello, Nop, this is a sole purposes of consistent hashing to allow dynamically add/remove nodes. - Dmitry On May 31, 2013, at 1:52 PM, Morgan Segalis wrote: > Hi Dmitry, > > I have though about consistent hashing, > > The only issue is that consistent hashing will work if we have a fixed number of cluster, If we add dynamically another cluster, the hash won't gives me the same cluster? > I might be wrong? > > Actually right now I have a gateway, which will choose the cluster on which there is the less number of connected clients, and redirect the client to this one. Working like a load balancer among other things the gateway does. > > Best regards, > Morgan. > > > Le 31 mai 2013 ? 12:30, Dmitry Kolesnikov a ?crit : > >> Hello, >> >> You current implementation starts to suffer from performance due to large number of messages to discover process location. >> You have to define a formal rule about "id" and its relation to node where processes exists. Essentially I am talking about consistent hashing. >> >> To be honest, I am not getting what is wrong with ETS and gproc. I am using a similar approach for my cluster management. I am using P2P methodology >> where local tables get sync periodically + updates on local table is replicated to cluster members. Each node is capable to observe the status of cluster members. Once node is disconnected the table is clean-up. However, I am using that approach for "internal" processes. "Client connection" are not distributed globally. >> >> Best Regards, >> Dmitry >> >> On May 31, 2013, at 1:12 PM, Morgan Segalis wrote: >> >>> Hi, >>> >>> Actually right now, I have this implementation : >>> >>> Each node has a clientpool >>> Each client pool manage an ets table, where all client are inserted / removed / searched >>> >>> If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. >>> >>> Advantage of this implementation : no synchronization required if one node goes down and back up again? >>> Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec >>> >>> >>> Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : >>> >>>> Please keep in mind that this is the result of a total of about 5 seconds of thinking. >>>> >>>> You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. >>>> >>>> Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. >>>> >>>> >>>> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: >>>> Hi everyone, >>>> >>>> I'm currently looking for better ways to do a clustered connected client list. >>>> >>>> On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. >>>> >>>> Each row in the ets table is for now a simple {"id", } >>>> >>>> I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). >>>> >>>> If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). >>>> If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. >>>> >>>> What would be in your opinion the best way to do that ? >>>> >>>> It is a messaging system, so it has to handle massive message passing through processes. >>>> >>>> Thank you for your help. >>>> >>>> Morgan. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From msegalis@REDACTED Fri May 31 13:06:33 2013 From: msegalis@REDACTED (Morgan Segalis) Date: Fri, 31 May 2013 13:06:33 +0200 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <347E399F-F486-4E4D-A2E8-45E248B7C3C4@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> <347E399F-F486-4E4D-A2E8-45E248B7C3C4@gmail.com> Message-ID: <93C3A955-97C5-4655-8A1C-08012D461035@gmail.com> Hi, But this won't let me choose which cluster the user should connect according to the number of connected client through the gateway, right ? If at first connection the user is redirect to cluster A, and after a reconnection the user is redirected to cluster B, how can a consistent Hash work in my case ? Best regards, Morgan. Le 31 mai 2013 ? 13:01, Dmitry Kolesnikov a ?crit : > Hello, > > Nop, this is a sole purposes of consistent hashing to allow dynamically add/remove nodes. > > - Dmitry > > On May 31, 2013, at 1:52 PM, Morgan Segalis wrote: > >> Hi Dmitry, >> >> I have though about consistent hashing, >> >> The only issue is that consistent hashing will work if we have a fixed number of cluster, If we add dynamically another cluster, the hash won't gives me the same cluster? >> I might be wrong? >> >> Actually right now I have a gateway, which will choose the cluster on which there is the less number of connected clients, and redirect the client to this one. Working like a load balancer among other things the gateway does. >> >> Best regards, >> Morgan. >> >> >> Le 31 mai 2013 ? 12:30, Dmitry Kolesnikov a ?crit : >> >>> Hello, >>> >>> You current implementation starts to suffer from performance due to large number of messages to discover process location. >>> You have to define a formal rule about "id" and its relation to node where processes exists. Essentially I am talking about consistent hashing. >>> >>> To be honest, I am not getting what is wrong with ETS and gproc. I am using a similar approach for my cluster management. I am using P2P methodology >>> where local tables get sync periodically + updates on local table is replicated to cluster members. Each node is capable to observe the status of cluster members. Once node is disconnected the table is clean-up. However, I am using that approach for "internal" processes. "Client connection" are not distributed globally. >>> >>> Best Regards, >>> Dmitry >>> >>> On May 31, 2013, at 1:12 PM, Morgan Segalis wrote: >>> >>>> Hi, >>>> >>>> Actually right now, I have this implementation : >>>> >>>> Each node has a clientpool >>>> Each client pool manage an ets table, where all client are inserted / removed / searched >>>> >>>> If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. >>>> >>>> Advantage of this implementation : no synchronization required if one node goes down and back up again? >>>> Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec >>>> >>>> >>>> Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : >>>> >>>>> Please keep in mind that this is the result of a total of about 5 seconds of thinking. >>>>> >>>>> You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. >>>>> >>>>> Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. >>>>> >>>>> >>>>> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: >>>>> Hi everyone, >>>>> >>>>> I'm currently looking for better ways to do a clustered connected client list. >>>>> >>>>> On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. >>>>> >>>>> Each row in the ets table is for now a simple {"id", } >>>>> >>>>> I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). >>>>> >>>>> If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). >>>>> If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. >>>>> >>>>> What would be in your opinion the best way to do that ? >>>>> >>>>> It is a messaging system, so it has to handle massive message passing through processes. >>>>> >>>>> Thank you for your help. >>>>> >>>>> Morgan. >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 31 13:38:49 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 31 May 2013 14:38:49 +0300 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <93C3A955-97C5-4655-8A1C-08012D461035@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> <347E399F-F486-4E4D-A2E8-45E248B7C3C4@gmail.com> <93C3A955-97C5-4655-8A1C-08012D461035@gmail.com> Message-ID: <02ABDDE7-38F4-4482-8433-B6806BBEBEAF@gmail.com> Hello, The consisted hash principles would handle load balancing as well. Let's assume you have clusters/nodes: A, B, C, D. The you have a connection with identifier ID. The connection identifier is ALWAYS mapped to same node. E.g. If you have a client identified by user name "User1" it would be place to node A. If "User1" get re-connceted it is still assigned to A. Load balancing comes from fact that "users" are randomly (uniform) are mapped to nodes A, B, C, D If you add a new node/cluster E then essentially some of user needs to be migrated to that node but it is resolved via shard/virtual nodes concept... You can play around with consistent hashing. I do have an utility implementation: https://github.com/fogfish/feta/blob/master/src/ring.erl e.g. R0 = ring:new([{modulo, 2}, {shard, 4}]). R1 = ring:join(1, a, ring:join(2, b, ring:join(3, c, ring:join(4, d, R)))). ring:whereis(ring:address(<<"alex">>, R1), R1) ring:whereis(ring:address(<<"bob">>, R1), R1). - Dmitry On May 31, 2013, at 2:06 PM, Morgan Segalis wrote: > Hi, > > But this won't let me choose which cluster the user should connect according to the number of connected client through the gateway, right ? > > If at first connection the user is redirect to cluster A, and after a reconnection the user is redirected to cluster B, how can a consistent Hash work in my case ? > > Best regards, > > Morgan. > > Le 31 mai 2013 ? 13:01, Dmitry Kolesnikov a ?crit : > >> Hello, >> >> Nop, this is a sole purposes of consistent hashing to allow dynamically add/remove nodes. >> >> - Dmitry >> >> On May 31, 2013, at 1:52 PM, Morgan Segalis wrote: >> >>> Hi Dmitry, >>> >>> I have though about consistent hashing, >>> >>> The only issue is that consistent hashing will work if we have a fixed number of cluster, If we add dynamically another cluster, the hash won't gives me the same cluster? >>> I might be wrong? >>> >>> Actually right now I have a gateway, which will choose the cluster on which there is the less number of connected clients, and redirect the client to this one. Working like a load balancer among other things the gateway does. >>> >>> Best regards, >>> Morgan. >>> >>> >>> Le 31 mai 2013 ? 12:30, Dmitry Kolesnikov a ?crit : >>> >>>> Hello, >>>> >>>> You current implementation starts to suffer from performance due to large number of messages to discover process location. >>>> You have to define a formal rule about "id" and its relation to node where processes exists. Essentially I am talking about consistent hashing. >>>> >>>> To be honest, I am not getting what is wrong with ETS and gproc. I am using a similar approach for my cluster management. I am using P2P methodology >>>> where local tables get sync periodically + updates on local table is replicated to cluster members. Each node is capable to observe the status of cluster members. Once node is disconnected the table is clean-up. However, I am using that approach for "internal" processes. "Client connection" are not distributed globally. >>>> >>>> Best Regards, >>>> Dmitry >>>> >>>> On May 31, 2013, at 1:12 PM, Morgan Segalis wrote: >>>> >>>>> Hi, >>>>> >>>>> Actually right now, I have this implementation : >>>>> >>>>> Each node has a clientpool >>>>> Each client pool manage an ets table, where all client are inserted / removed / searched >>>>> >>>>> If an user A sends a message to user B, it will check first the local ets, if not found, will ask to all nodes if they found user B in their ets table, if user B is connected to another node, this node will return the process pid. >>>>> >>>>> Advantage of this implementation : no synchronization required if one node goes down and back up again? >>>>> Disadvantage of this implementation : I guess the number of message / sec is much higher of number of connection/disconnection / sec >>>>> >>>>> >>>>> Le 31 mai 2013 ? 00:36, Chris Hicks a ?crit : >>>>> >>>>>> Please keep in mind that this is the result of a total of about 5 seconds of thinking. >>>>>> >>>>>> You could have a coordinator on each node which is responsible for communicating with the coordinators on all of the other connected nodes. Your ETS entries would need to be expanded to keep track of the node that the user is connected on as well. The coordinators track the joining/leaving of the cluster of all other nodes and will purge the ETS table of any entries that belong to any recently downed node. As long as you don't have hard real-time requirements, which if you do you're using the wrong tool anyway, then you can come up with a bunch of ways to group together updates between coordinators to make sure they don't get overloaded. >>>>>> >>>>>> Without a lot more details on the exact sort of metrics your system needs to be able to handle it's all really just a guessing game, in the end. >>>>>> >>>>>> >>>>>> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis wrote: >>>>>> Hi everyone, >>>>>> >>>>>> I'm currently looking for better ways to do a clustered connected client list. >>>>>> >>>>>> On every server I have an ets table, in which I insert / delete / search the process of each client connecting to the server. >>>>>> >>>>>> Each row in the ets table is for now a simple {"id", } >>>>>> >>>>>> I have tried the gproc module from Ulf Wiger, but when a cluster goes down, everything goes wrong? (especially if it is the elected leader). >>>>>> >>>>>> If a cluster goes down, other clusters should consider that every client connected on the said cluster are actually not connected (even if is just a simple connection error between clusters). >>>>>> If it goes back online, back on the nodes() list, other clusters should consider clients on this cluster back online. >>>>>> >>>>>> What would be in your opinion the best way to do that ? >>>>>> >>>>>> It is a messaging system, so it has to handle massive message passing through processes. >>>>>> >>>>>> Thank you for your help. >>>>>> >>>>>> Morgan. >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 31 13:47:05 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 31 May 2013 14:47:05 +0300 Subject: [erlang-questions] managing subprocesses In-Reply-To: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> References: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> Message-ID: <47BE1CE4-CB57-4DF7-8860-D0796756D145@gmail.com> Hello, I've found this library useful for interaction shell commands: https://github.com/gleber/sh If you are talking about daemons and supervise them then I've made a simple wrapper to supervise "external" processes and attache then to supervise tree https://github.com/fogfish/feta/blob/master/src/osproc.erl e.g. supervisor:start_child(Sup, { jaba, {osproc, start_link, [java, jaba_spec()]}, permanent, 1000, worker, dynamic }) jaba_spec() -> ["-mx300m", "-cp", ?] - Dmitry On May 30, 2013, at 10:52 PM, Timo Schrappe wrote: > Hello, > > I am wondering that there isn't a great way for managing and parsing subprocesses (through os:cmd/1) and their output. > I spent some time with google and found erlexec which basically will do my job here. But it feels a bit cheesy to use an external module (or is it called application in erlang?) to use such a fundamental functionality. > > I want to manage some shell scripts which will start some e.g. daemons. > > > What is the erly way to do that kind of things? Im specially interested in the exit code from the bash scripts. > > > - Timo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sean@REDACTED Fri May 31 13:47:59 2013 From: sean@REDACTED (Sean Allen) Date: Fri, 31 May 2013 07:47:59 -0400 Subject: [erlang-questions] Etudes for Erlang Discussion Group in NYC Message-ID: Hi All, We are hosting a series of meetups in NYC to go through the Etudes for Erlang series in hopes of getting more people learning Erlang. We'd love if you could help spread the word. Info is on the NYC Erlang meetup group site: http://www.meetup.com/Erlang-NYC/events/122146332/ -Sean- -------------- next part -------------- An HTML attachment was scrubbed... URL: From publicityifl@REDACTED Fri May 31 14:02:09 2013 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Fri, 31 May 2013 05:02:09 -0700 (PDT) Subject: [erlang-questions] Call for Papers IFL 2013 Message-ID: <51a89141.8757b40a.5a9b.0a90@mx.google.com> =============================================================================== VACANCY : 1x Phd Student in domain specific type error diagnosis for Haskell =============================================================================== The activities of the Software Systems division at Utrecht University include research on programming methodologies, compiler construction, and program analysis, validation, and verification. For information about the research group of Software Technology, see: http://www.cs.uu.nl/wiki/Center Financed by the Netherlands Organisation for Scientific Research (NWO), we currently have a job opening for: * 1x PhD researcher (Ph D student) Software Technology Domain-specific languages (DSLs) have the potential both to reduce the effort of programming, and to result in programs that are easier to understand and maintain. For various good reasons, researchers have proposed to embed DSLs (then called EDSLs) into a general purpose host language. An important disadvantage of such an embedding is that it is very hard to make type error diagnosis domain-aware, because inconsistencies are by default explained in terms of the host language. We are currently looking for a highly motivated Ph D student to investigate this problem in the context of the functional language Haskell. The basic approach is to scale the concept of specialized type rules as developed by (Heeren, Hage and Swierstra, ICFP '03, see link below) for Haskell '98 to modern day Haskell with all of its type system extensions. The work is both technically challenging, i.e., how do you ensure that modifications to the type diagnositic process do not inadvertently change the type system, and practically immediately useful: making domain-specific type error diagnosis a reality for a full sized language such as Haskell is likely to have a pervasive influence on the field of domain-specific languages, and the language Haskell. The ICFP '03 paper can be found at http://www.cs.uu.nl/people/jur/scriptingthetypeinferencer.pdf A project paper that describes the context and aims of the current project can be found here: http://www.cs.uu.nl/people/jur/tfp2013_submission_2.pdf At first, the work will be prototyped in our own Utrecht Haskell Compiler. If succesfull, the work will also make its way into the GHC. We expect the candidate to communicate the results academically, to present the work at scientific conferences, to supervise Master students, and to assist in teaching courses at Bachelor or Master level. --------------------------------- What we are looking for --------------------------------- The candidate should have an MSc in Computer Science, be highly motivated, speak and write English very well, and be proficient in producing scientific reports. Knowledge of and experience with at least one of the following two areas is essential: * functional programming, and Haskell in particular * type system concepts Furthermore, we expect the candidate to be able to reason formally. Experience in compiler construction is expected to be useful in this project. --------------------------------- What we offer --------------------------------- You are offered a full-time position for 4 years. The gross salary is in the range between ? 2083,- and maximum ? 2664,- per month. The salary is supplemented with a holiday bonus of 8% and an end-of-year bonus of 8,3% per year. In addition we offer: a pension scheme, a partially paid parental leave, flexible employment conditions. Conditions are based on the Collective Labour Agreement Dutch Universities. We aim to start November 1, 2013 at the latest, but preferably sooner. --------------------------------- In order to apply --------------------------------- To apply please attach a letter of motivation, a curriculum vitae, and (email) addresses of two referees. Make sure to also include a transcript of the courses you have followed (at bachelor and master level), with the grades you obtained, and to include a sample of your scientific writing, e.g., the pdf of your master thesis. It is possible to apply for this position if you are close to obtaining your Master's. In that case include a letter of your supervisor with an estimate of your progress, and do not forget to include at least a sample of your technical writing skills. Application closes on the 20th of June 2013. For application, visit http://www.cs.uu.nl/vacatures/en/583630.html and follow the link to the official job application page at the bottom. --------------- Contact person --------------- For further information you can direct your inquiries to: Dr. Jurriaan Hage Phone: (+31) 30 253 3283 e-mail: J.Hage@REDACTED website: http://www.cs.uu.nl/wiki/Hage/WebHome From schrappe.t@REDACTED Fri May 31 14:40:16 2013 From: schrappe.t@REDACTED (Timo Schrappe) Date: Fri, 31 May 2013 14:40:16 +0200 Subject: [erlang-questions] managing subprocesses In-Reply-To: References: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> Message-ID: <60FC5EE4-960B-4D6C-B64B-CE9615813ADC@thirdman.de> On May 31, 2013, at 2:01 PM, David Welton wrote: >> I am wondering that there isn't a great way for managing and parsing subprocesses (through os:cmd/1) and their output. > > It does not seem to be of interest to the Erlang maintainers. > >> I spent some time with google and found erlexec which basically will do my job here. But it feels a bit cheesy to use an external module (or is it called application in erlang?) to use such a fundamental functionality. > > Erlexec works pretty well, and is actively developed. Do not misunderstand me. My problem was that I could not find a way to get the exit status with erlang itself. > > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From schrappe.t@REDACTED Fri May 31 14:41:11 2013 From: schrappe.t@REDACTED (Timo Schrappe) Date: Fri, 31 May 2013 14:41:11 +0200 Subject: [erlang-questions] managing subprocesses In-Reply-To: <51A7B219.1020501@gmail.com> References: <56C1B069-BCD7-4AFB-A4E1-2F0FF107E9F7@thirdman.de> <51A7B219.1020501@gmail.com> Message-ID: <13684C53-2D4C-44F8-A23A-5E6796D9A2E0@thirdman.de> The "&& echo 0 || echo 1" thing is indeed smart but still a dirty hack. open_port works for me! 14> P = erlang:open_port({spawn, "asdsad23"}, [exit_status]). #Port<0.601> 15> sh: line 0: exec: asdsad23: not found 15> flush(). Shell got {#Port<0.601>,{exit_status,127}} ok Thanks On May 30, 2013, at 10:10 PM, Michael Truog wrote: > If you want to do it manually, you would be using: > http://www.erlang.org/doc/man/erlang.html#open_port-2 > > Note that you can also do something like (if you only care about the exit code): > > shell(Command, Arguments) -> > CommandSuffix = " && echo 0 || echo 1", > [Result | _] = os:cmd(lists:flatten( > io_lib:format(Command ++ CommandSuffix, Arguments))), > if > Result =:= $0 -> > ok; > Result =:= $1 -> > io:format("\"~s\" failed!~n", [io_lib:format(Command, Arguments)]), > error > end. > > > > On 05/30/2013 12:52 PM, Timo Schrappe wrote: >> Hello, >> >> I am wondering that there isn't a great way for managing and parsing subprocesses (through os:cmd/1) and their output. >> I spent some time with google and found erlexec which basically will do my job here. But it feels a bit cheesy to use an external module (or is it called application in erlang?) to use such a fundamental functionality. >> >> I want to manage some shell scripts which will start some e.g. daemons. >> >> >> What is the erly way to do that kind of things? Im specially interested in the exit code from the bash scripts. >> >> >> - Timo >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Fri May 31 16:36:07 2013 From: mattevans123@REDACTED (Matthew Evans) Date: Fri, 31 May 2013 10:36:07 -0400 Subject: [erlang-questions] Preallocate memory Message-ID: Hi Group What is the best way (if it's possible) to preallocate memory on startup for the Erlang VM? On our setup we have a low priority Java app that we don't want to consume RAM that is needed for the higher priority Erlang VM. Thanks Matt Sent from my iPhone From jleivent@REDACTED Fri May 31 16:44:26 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Fri, 31 May 2013 10:44:26 -0400 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <51A8507C.5070303@cs.ntua.gr> References: <51A7D001.6050707@comcast.net> <20130531070348.GC451@aluminium.local> <51A8507C.5070303@cs.ntua.gr> Message-ID: <51A8B74A.8050908@comcast.net> On 05/31/2013 03:25 AM, Kostis Sagonas wrote: > ... > Since I've also stumbled across something analogous in the past, IMO the > proper solution is to extend the API of these opaque data structures > (these modules) by providing is_empty/1 functions for them. I feel better knowing that I wasn't just missing some obvious is_empty substitute. I also agree with your sentiment about keeping opaque structures "opaque" - and the documentation implies a similar sentiment ("The representation of a dictionary|set is not defined."). In an off-list post, it was pointed out that the structures that are missing is_empty predicates (dict, orddict, sets, ordsets) are also missing any API function to peek at or take an arbitrary element (similar to queue:peek or queue:out) - which could be used as an is_empty substitute (a more useful one at that). Another problem with relying on either size(X) or the representation itself is that dict and orddict (likewise set and ordset) have identical APIs apparently to allow for substitution of one for the other with minimal code churn (other than the == vs. =:= issue). But their representations (and the performance of size) differ. > I am sure the OTP folks would welcome a patch. Maybe that's the answer. But, the lack of an is_empty predicate for dict/sets has existed in Erlang for so long!? -- Jonathan From bchesneau@REDACTED Fri May 31 16:57:42 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 31 May 2013 16:57:42 +0200 Subject: [erlang-questions] STUN server Message-ID: is there any implementation of a STUN server in erlang around? - benoit From ingham.k@REDACTED Fri May 31 17:00:43 2013 From: ingham.k@REDACTED (Igor Karymov) Date: Fri, 31 May 2013 08:00:43 -0700 (PDT) Subject: [erlang-questions] STUN server In-Reply-To: References: Message-ID: You may take a look here: https://github.com/processone/ejabberd/tree/master/src/stun ???????, 31 ??? 2013 ?., 18:57:42 UTC+4 ???????????? Benoit Chesneau ???????: > > is there any implementation of a STUN server in erlang around? > > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Fri May 31 17:04:45 2013 From: lemenkov@REDACTED (Peter Lemenkov) Date: Fri, 31 May 2013 19:04:45 +0400 Subject: [erlang-questions] STUN server In-Reply-To: References: Message-ID: 2013/5/31 Benoit Chesneau : > is there any implementation of a STUN server in erlang around? If all you need is just STUN (not TURN for example) then you may take a look at this one I've built some time ago: * https://github.com/lemenkov/erlstund STUN isn't that hard to implement (apart from STUN message type encoding, see http://tools.ietf.org/html/rfc5389#section-6 ) and you can build your own quite fast. -- With best regards, Peter Lemenkov. From mononcqc@REDACTED Fri May 31 17:19:57 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 31 May 2013 11:19:57 -0400 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <51A7D001.6050707@comcast.net> References: <51A7D001.6050707@comcast.net> Message-ID: <20130531151956.GA48778@ferdair.local> Most of these data structures have a 'new' or 'empty' function call that creates an empty/blank data structure. I then just compare things like MyDict =:= dict:new() to know if it's empty. I'm not aware of a case where this doesn't work, but it makes a very subtle assumption that *might* break opacity: it assumes that the data structure doesn't keep internal statistics or traces of content it has seen before but is no longer there. Regards, Fred. On 05/30, Jonathan Leivent wrote: > I noticed that a lot of the opaque data structures don't have an > is_empty predicate. What is the best way to test if one of these is > empty? Preferably an O(1) test, not size(X)==0 (unless that is > O(1)), because the structure might be quite large. > > One way I thought of is to use fold on a function that just throws: > > is_empty(Dict) -> > catch dict:fold(fun (_,_) -> throw(false) end, true, Dict). > > But that is disturbingly ugly. Is there a preferred way to do this? > > Thanks. > Jonathan Leivent > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jleivent@REDACTED Fri May 31 17:33:43 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Fri, 31 May 2013 11:33:43 -0400 Subject: [erlang-questions] Best way to test if empty dict/set/etc.? In-Reply-To: <20130531151956.GA48778@ferdair.local> References: <51A7D001.6050707@comcast.net> <20130531151956.GA48778@ferdair.local> Message-ID: <51A8C2D7.2070409@comcast.net> On 05/31/2013 11:19 AM, Fred Hebert wrote: > Most of these data structures have a 'new' or 'empty' function call > that creates an empty/blank data structure. > > I then just compare things like MyDict =:= dict:new() to know if it's > empty. I'm not aware of a case where this doesn't work, but it makes a > very subtle assumption that *might* break opacity: it assumes that the > data structure doesn't keep internal statistics or traces of content it > has seen before but is no longer there. I wasn't sure about whether the equality test would always work, either. Does the documentation ever mention whether or not equality tests on opaque structures is expected to work? However, for sets/ordsets, maybe this is both O(1) and opaque: is_empty(S) -> sets:is_subset(S, sets:new()). -- Jonathan From bchesneau@REDACTED Fri May 31 18:08:55 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 31 May 2013 18:08:55 +0200 Subject: [erlang-questions] STUN server In-Reply-To: References: Message-ID: On Fri, May 31, 2013 at 5:04 PM, Peter Lemenkov wrote: > 2013/5/31 Benoit Chesneau : >> is there any implementation of a STUN server in erlang around? > > If all you need is just STUN (not TURN for example) then you may take > a look at this one I've built some time ago: > > * https://github.com/lemenkov/erlstund > > STUN isn't that hard to implement (apart from STUN message type > encoding, see http://tools.ietf.org/html/rfc5389#section-6 ) and you > can build your own quite fast. Thanks, Interestingly your project goes in the direction I want , ie implementing the rfc 5245 (ICE) . Do you plan to continue to work on it? - benoit From bchesneau@REDACTED Fri May 31 18:10:20 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 31 May 2013 18:10:20 +0200 Subject: [erlang-questions] STUN server In-Reply-To: References: Message-ID: On Fri, May 31, 2013 at 5:00 PM, Igor Karymov wrote: > You may take a look here: > https://github.com/processone/ejabberd/tree/master/src/stun I completely forgot that ejabberd had that. Thanks! > > ???????, 31 ??? 2013 ?., 18:57:42 UTC+4 ???????????? Benoit Chesneau > ???????: >> >> is there any implementation of a STUN server in erlang around? >> >> >> - benoit >> _______________________________________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions