From technion@REDACTED Fri Sep 1 02:13:17 2017 From: technion@REDACTED (Technion) Date: Fri, 1 Sep 2017 00:13:17 +0000 Subject: [erlang-questions] How safe is it to leave an open SSL port on the public internet? In-Reply-To: <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> References: <55FFE7D1-782C-4FEF-B23D-BE292BCA5574@gmail.com> <1DAAF3BE-D958-45C8-AA75-2F71EC5D9E68@gmail.com> <20170829221959.GA51868@ferdmbp.local> <20170829222354.GB51868@ferdmbp.local>, <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> Message-ID: Hi, Since this is a discussion around SSL, you're choices are down to what is a part of the TLS spec. Those choices are precisely between RSA, and ECC. In the upcoming TLS1.3, RSA has been dropped. GPG only just introduced ECC support. The highly trusted libsodium uses ECC. A well respected "best practices" guide places ECC in the recommended section: https://gist.github.com/atoponce/07d8d4c833873be2f68c34f9afc5a78a There are valid concerns around the future impact of quantum computing here, but it's currently the best option. It's certainly not true to say "the NSA has deprecated ECC", several current NIST standards recommend ECC moving forward. The only real debate is supporting the dubious NIST curves, or the alternate 25519 we've seen in TLS 1.3. I wouldn't suggest for current, practical discussions there needs to be a warning against ECC. ________________________________ From: erlang-questions-bounces@REDACTED on behalf of code wiget Sent: Thursday, 31 August 2017 4:03 AM To: Fred Hebert Cc: Erlang-Questions Questions Subject: Re: [erlang-questions] How safe is it to leave an open SSL port on the public internet? Also, Fred, I re-read your post and wanted to either start a quick discussion/warn you about elliptic curves. According to the NSA: "the growth of elliptic curve use has bumped up against the fact of continued progress in the research on quantum computing, which has made it clear that elliptic curve cryptography is not the long term solution many once hoped it would be.? The NSA has deprecated ECC, whether or not that means that some foreign actor has a crack or if they are that worried about quantum computing is to be seen, but for now it seems like we should be moving away from ECC. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Fri Sep 1 02:22:07 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 31 Aug 2017 20:22:07 -0400 (EDT) Subject: [erlang-questions] Building a DAW in Erlang In-Reply-To: References: Message-ID: <1504225327.54672404@apps.rackspace.com> Hey Joe, Great project! Sadly, for me, I can't help with the technical issues facing you. But, here are two wacky experiments that may or may not yield interesting results. But first, some background: Way back in my Stanford graduate student days friends and I got together for several sessions to play with audio recording. We were interested in creating the most far-our sounds possible--- anything goes from banging on oven racks to looping and messing with tape speeds. Pure play. We came out with a cacophony of eery sounds. A year later I produced a film on Robert Frost's childhood in San Francisco. But one scene defied our composer. We need something, but music didn't seem right. Then I remembered our cacophy tape. We found an interesting section, mixed it into the sound track at a low level, and it was perfect. So, Experiment A: 1. Create a sample of N Erlang terms. 2. Run them through erlang:term_to_binary/1. 3. Play them back through audio. You may need headers to play the audio, but you'd know more about that than I. Thought (and maybe do) Experiment B: 1. Is it possible to write an Erlang term that generates an approximation of white noise? 2. Is it possible to write an Erlang term that generates short sequences of basic wave forms--- sine, triangle, square? 3. Is there a programmatic way to edit the generated binaries to improve aesthetic quality? Love to hear if you come up with anything. All the best, LRP -----Original Message----- From: "Joe Armstrong" Sent: Thursday, August 31, 2017 7:11am To: "Erlang" Subject: [erlang-questions] Building a DAW in Erlang Hello, I want to build a "proof of concept" DAW (Digital Audio Workstation) Why? - just for fun. DAWs involve complex GUIs, complex audio processing, and complex man-machine interactions - I'd like to make the DAW from small well defined isolated communicating components. I was wondering about audio - has anybody ideas about this. I want to build a synthesizer with as much Erlang as possible and as little C as necessary - and even if I use C I'd like reference algorithms in Erlang to validate the C. Has anybody implemented any digital audio filters in Erlang? The Synthesizer would have no GUI (The GUI is a separate issue). Having got to the point where I can generate raw PCM samples I need to output these - preferably send them to a socket, with an audio server that reads and outputs the samples - has anybody done this? Initially I'll do this on mac but would eventually like it to run on windows and linux. All idea are very welcome /Joe _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Fri Sep 1 03:11:29 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Sep 2017 13:11:29 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: References: <2691824.Matf7LgGbk@changa> <20170830082912.GA29138@erix.ericsson.se> <13223505.cVJpAap13P@changa> <77c901fb-88b1-f2e6-23c0-a2eeaaa690e4@ninenines.eu> Message-ID: <0339cc4e-501b-1873-1549-63f83286e417@cs.otago.ac.nz> Sometimes people program against the documentation. Sometimes people program against the behaviour. Sometimes people program against their expectations and the hell with the docs and the code. In fact you are never going to please everyone. My case study: The Quintus Prolog documentation had always said "do not add or remove clauses of an interpreted predicate while it is running; the behaviour is not defined." In major release 3, as part of implementing modules, we defined the behaviour to be "every call acts as if it made a snapshot of the predicate on entry and used that snapshot; changes to the predicate will affect later calls but not earlier ones". One customer complained that we had broken his code. We not only pointed out that the manual had warned against what he was doing, I rewrote his code to run 700 times faster. (I kid you not.) Of course we lost that customer. I wish there was a blanket rule I could recommend. There is one, of course, and that is "test before you release". In this particular case, it's quite hard to test because the bad outcome is extremely rare. (Although I do have a photocopy of the library of an old programming language where the Gaussian random number generator could not possibly have worked, because 0 turned up once every 2^16 calls.) There is another rule, which is that changes to the documentation and changes to the code both need to be clearly communicated in release nodes, and as a user, it's my job to *read* the release notes. The irony here, of course, is that Erlang copied as183 from Prolog, and I wrote the Prolog version of the Wichmann-Hill 3-cycle generator, and it doesn't take any care to avoid 0.0, and that's one reason why I was glad to see it replaced... From ok@REDACTED Fri Sep 1 04:57:49 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Sep 2017 14:57:49 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59A856EC.1030000@gmail.com> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> Message-ID: <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> On 1/09/17 6:35 AM, Michael Truog wrote: > As I argued in the original pull request for these recent 20.0 random > number changes, a uniform distribution is much more intuitive if it is > inclusive: [0,1] Intuitive? For integers, I'll grant that. For reals? Not so much. I certainly cannot grant "MUCH more intuitive". I've had occasion to do (random next * 2 - 1) arcTanh, which of course breaks down if *either* 0 or 1 is returned by (random next). > For example, if you are dealing with probabilities, it is simpler to > think in percentages from 0.00 to 1.00 Actually, when I'm dealing with probabilities, I never think about them as percentages. Now the interesting thing here is this. Suppose you want to get a true [false] outcome with probability p [1-p]. Then random next < p does the job perfectly, but ONLY if 1 is excluded. The trick of generating a random integer from 1 to N by doing (in C): (int)(random() * N) + 1 can of course give you N+1 if random() can return 1.0, and this is a thing I very often do. (Yes, if 0.0 is excluded, the probability of getting 1 is very slightly skewed, but it's _very_ slightly.) > An example from the python documentation is at > https://docs.python.org/3/library/random.html#random.uniform though they > have ambiguity about the highest value due to a rounding problem they have. Oh, the bit where they say "The end-point value b may or may not be included in the range." Worst of both worlds. You cannot rely on it being included and you cannot rely on it being excluded. Let's face it, the usual expectation is that a uniform random number generator will return a value in the half-open range [0,1). I have uses for (0.0, 1.0). Michael Truog has uses for [0.0,1.0], although I wasn't able to tell from a quick scan of his code what they are. I could personally live with a warning in the documentation that says that the random number generator could return 0.0, and here's a little loop you might use to avoid that, and another suggestion in the code about how to get the result Michael Truog wants. I just want it to be obvious that it's dangerous to assume that the result will not be 0. By the way, given that a common way to make random floats is to generate a bitvector, consider (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. You will notice that the spacing between the values is *almost* uniform, but not at the end. From ok@REDACTED Fri Sep 1 05:40:00 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Sep 2017 15:40:00 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170831203007.GE51868@ferdmbp.local> References: <2691824.Matf7LgGbk@changa> <20170830082912.GA29138@erix.ericsson.se> <13223505.cVJpAap13P@changa> <77c901fb-88b1-f2e6-23c0-a2eeaaa690e4@ninenines.eu> <8a3ec5fb-27a6-78dc-aa60-e9650c1445b4@ninenines.eu> <20170831203007.GE51868@ferdmbp.local> Message-ID: <6bafab95-e7f2-9c33-1bb0-66367fd5b7a1@cs.otago.ac.nz> On 1/09/17 8:30 AM, Fred Hebert wrote: > I think 'it depends' truly *is* the best way to go about it. +1. The universal expert's answer. (The other is, "some do, some don't.") I'm going to nitpick gently. > - A function does what is in the doc, but also does a bit more at the > same time. Do you fix by removing the additional functionality people > may now rely on, or by updating the doc to match the implementation? There is no pressing need to do either. Perhaps adding a note to the documentation that "In some releases this function may accept additional arguments and appear to do sensible things. This is subject to change." is all that is required. > - The documentation specifies that by sending the atom 'tsl1.3' you can > set up a TLS 1.3 connection, but the implementation only accepts > 'tsl1.3' and crashes on 'tsl1.3'. Do you not update the documentation > for what was a typo, or do you add support for 'tsl1.3' as a > parameter? You do both, and you add a note to the documentation that "Earlier releases incorrectly accepted 'tsl1.3' instead of 'tls1.3'. This will be corrected in release N+2." > - A function for socket handling returns values such as `{error, emfile > | enfile | econnrefused}`. Someone finds out that the syscalls it > relays data to also may return `{error, eperm | eaccess}` on top of > what was specified before. Do you swallow the errors and mask them, or > update the docs? Or is it not just a bug in the docs? This one requires serious digging to find out. I've been there, though not with sockets. Discovering that the C program you wrote for "UNIX" version x reports an error in "UNIX" version y that doesn't even exist in version x is a pain. When I was working at Quintus I spent weeks trawling through Unix manuals carefully checking the error returns of every system call we used, and tried to figure out what to do for each. My aim was to classify errors as - the program should not have done this (EINVAL &c) - the resource doesn't exist (ENOENT, ECHILD, ...) when it should or does exist (EEXIST) when it shouldn't - the resource named does exist but is the wrong kind of resource (ENOTDIR, EISDIR, ENOTBLK, ...) - the program doesn't have permission to do this (EPERM, ...) - the resource is busy (EBUSY, EDEADLK, ...) - some resource ran out (ENOMEM, ENFILE, EMFILE, EFBIG, ENOSPC,...) - something went wrong in the system (ENETDOWN, EHOSTDOWN, ...) I eventually had to stop, but did write a little script using the C preprocessor to dump all the macros obtained by including and find all the E[A-Z0-9]+ ones, and check them against the list of known ones. It was a lot of work that I hoped POSIX would render unnecessary in the future. It didn't. So this raises the quality-of-documentation issue, "Why did we ever document this as only raising these specific errors?" > - A function's doc says it sorts in a stable manner but it does not. > Which one should you change? There seems to be no winner on this one. There is a very clear winner: fix the code. When quicksort was invented, it was known to do more comparisons than merge sort. Hoare's problem was that he needed to sort a bunch of numbers on a machine with 256 words of data memory, so that (a) he *really* couldn't afford extra memory, and (b) comparisons were cheap. Sedgewick's PhD thesis (which I did read once) did a very thorough examination of quicksort performance and several variations on it, BUT assumed that a comparison cost 1/4 as much as a memory reference. Most comparisons of quicksort vs mergesort I've seen were misleading because they compared numbers rather than strings or richer data and because they compared a relatively smart quicksort implementation against a pretty dump merge sort. You can, for example, - do *optimal* sorting for small subfiles - not only ping-pong between the two arrays but run alternating passes backwards (this gets a bit more leverage out of the cache) - do 4-way merge instead of 2-way merge (does as many comparisons but is nicer to the memory subsystem) and you can use a variant of the natural merge like samsort to exploit existing order. If you want an efficient algorithm with minimal space overhead, there's a variant of heapsort that gets NlgN + O(N) comparisons worst case, which beats quicksort. > - A function logs information while it operates on data, but the > documentation makes no reference to it. Someone working with it in a > specific environment has issues with that behaviour.* The documentation and the code both have to be changed. The logging has to be made conditional, and the documentation has to mention it. > > * In adding shell history to Erlang, we got tripped up on disk_log doing > just that. We added a new option to force it to be silent when needed, > for example, so both the code and the doc required a fix! Ah, so I agreed with you on that one. For the 2nd half of my concurrent programming course, I have strongly recommended LYSE, so it's nice to find myself on your side... From ok@REDACTED Fri Sep 1 05:44:41 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Sep 2017 15:44:41 +1200 Subject: [erlang-questions] Building a DAW in Erlang In-Reply-To: <1504225327.54672404@apps.rackspace.com> References: <1504225327.54672404@apps.rackspace.com> Message-ID: <21cb1b25-6914-1650-1f9a-5feff1c48d4c@cs.otago.ac.nz> On 1/09/17 12:22 PM, lloyd@REDACTED wrote: > > 1. Is it possible to write an Erlang term that generates an approximation of white noise? > 2. Is it possible to write an Erlang term that generates short sequences of basic wave forms--- sine, triangle, square? > 3. Is there a programmatic way to edit the generated binaries to improve aesthetic quality? Since the external term encoding of a binary includes the bytes of that binary verbatim, yes, yes, and yes. See BINARY_EXT in http://erlang.org/doc/apps/erts/erl_ext_dist.html From ok@REDACTED Fri Sep 1 06:57:22 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Sep 2017 16:57:22 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170831123008.GA73645@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> Message-ID: <5feab048-c234-8abc-c19d-ebfc21317673@cs.otago.ac.nz> For what it's worth, here is an extract from the R documentation. runif(n, min = 0, max = 1) # min and max have defaults ... 'runif' will not generate either of the extreme values unless 'max = min' or 'max-min' is small compared to 'min', and in particular not for the default arguments. From mjtruog@REDACTED Fri Sep 1 07:29:34 2017 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 31 Aug 2017 22:29:34 -0700 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> Message-ID: <59A8F03E.9040400@gmail.com> On 08/31/2017 07:57 PM, Richard A. O'Keefe wrote: > > On 1/09/17 6:35 AM, Michael Truog wrote: >> As I argued in the original pull request for these recent 20.0 random >> number changes, a uniform distribution is much more intuitive if it is >> inclusive: [0,1] > > Intuitive? For integers, I'll grant that. For reals? Not so much. > I certainly cannot grant "MUCH more intuitive". > I've had occasion to do (random next * 2 - 1) arcTanh, which of > course breaks down if *either* 0 or 1 is returned by (random next). A uniform distribution should be uniformly distributed. I understand the woes of floating-point prevent perfect uniform distribution, but we could at least try to pay attention to the limits involved, and if we did, that would make the idea much more intuitive. My belief is that the [0,1) distribution is the most common because it is the easiest to implement given the IEEE floating point standard format. However, I would also like to be proven wrong, to have more faith in the current situation. >> For example, if you are dealing with probabilities, it is simpler to >> think in percentages from 0.00 to 1.00 > > Actually, when I'm dealing with probabilities, I never think > about them as percentages. Now the interesting thing here > is this. Suppose you want to get a true [false] outcome > with probability p [1-p]. Then random next < p does the > job perfectly, but ONLY if 1 is excluded. I see this as much simpler when it is possible to have random =< p , not that it matters much in this context, only when things get more complex. > > > The trick of generating a random integer from 1 to N by > doing (in C): (int)(random() * N) + 1 can of course give > you N+1 if random() can return 1.0, and this is a thing I very > often do. (Yes, if 0.0 is excluded, the probability of getting > 1 is very slightly skewed, but it's _very_ slightly.) > >> An example from the python documentation is at >> https://docs.python.org/3/library/random.html#random.uniform though they >> have ambiguity about the highest value due to a rounding problem they have. > > Oh, the bit where they say "The end-point value b may or may not be > included in the range." Worst of both worlds. You cannot rely on it > being included and you cannot rely on it being excluded. > > Let's face it, the usual expectation is that a uniform random number > generator will return a value in the half-open range [0,1). > > I have uses for (0.0, 1.0). > Michael Truog has uses for [0.0,1.0], although I wasn't able to tell > from a quick scan of his code what they are. I have some examples that can make this desire a bit clearer: https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 % use Box-Muller transformation to generate Gaussian noise % (G. E. P. Box and Mervin E. Muller, % A Note on the Generation of Random Normal Deviates, % The Annals of Mathematical Statistics (1958), % Vol. 29, No. 2 pp. 610?611) X1 = random(), X2 = PI2 * random(), K = StdDev * math:sqrt(-2.0 * math:log(X1)), Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), sleep(Result2), https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L204-L210 X = random(), if X =< Percent -> erlang:exit(monkey_chaos); true -> ok end, with: random() -> quickrand:strong_float(). These are code segments used for the CloudI service configuration options monkey_latency and monkey_chaos so that normal distribution latency values and random service deaths can occur, respectively (with the more common names as Latency Monkey and Chaos Monkey, but the words switched to make the concepts easier to find and associate). For the Box-Muller transformation, it really does want a definite range [0,1] and it helps make the monkey_chaos service death easier to understand at a glance. > > I could personally live with a warning in the documentation that says > that the random number generator could return 0.0, and here's a little > loop you might use to avoid that, and another suggestion in the code > about how to get the result Michael Truog wants. > > I just want it to be obvious that it's dangerous to assume that the > result will not be 0. > > By the way, given that a common way to make random floats is to > generate a bitvector, consider > (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. > You will notice that the spacing between the values is *almost* > uniform, but not at the end. > I agree, but I still think the use of the word uniform here is better suited to the extremes. We know it is IEEE floating-point, so we know it is inexact. From raimo+erlang-questions@REDACTED Fri Sep 1 09:58:17 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 09:58:17 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <77c901fb-88b1-f2e6-23c0-a2eeaaa690e4@ninenines.eu> References: <2691824.Matf7LgGbk@changa> <20170830082912.GA29138@erix.ericsson.se> <13223505.cVJpAap13P@changa> <77c901fb-88b1-f2e6-23c0-a2eeaaa690e4@ninenines.eu> Message-ID: <20170901075817.GA87875@erix.ericsson.se> On Thu, Aug 31, 2017 at 03:42:06PM +0200, Lo?c Hoguin wrote: > On 08/30/2017 12:33 PM, zxq9 wrote: > > On 2017?08?30? ??? 10:29:12 Raimo Niskanen wrote: > > > >> It is in general safer to change the documentation to match the reality. > > > > Wow. > > I certainly hope this is not the general policy for OTP. We program > against the documentation. The documentation *is* our reality. > > It also seems it's not even listed in the release notes. We program > against the documentation, if the documentation has a breaking changes > it would be great to know about it. I had no idea that statement would be so flammable. :-) I simply wanted to point out that from the point of view of a developer of a mature product like Erlang/OTP it is has happened too many times that a subtle behaviour change breaks something for a customer. And that is something that programmers writing new code often do not appreciate since they simply want the libraries to be "right" where it is a very reasonable view that the documentation defines what is "right". I also realize that in this particular case to stop returning 0.0 from rand:uniform() would also have been a safe choice since that would be almost impossible to detect and almost certainly cause no harm. And no, I did not state an OTP policy. We decide from case to case. > > -- > Lo?c Hoguin > https://ninenines.eu -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Sep 1 10:41:15 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 10:41:15 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59A8F03E.9040400@gmail.com> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> Message-ID: <20170901084115.GB87875@erix.ericsson.se> On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > On 08/31/2017 07:57 PM, Richard A. O'Keefe wrote: > > > > On 1/09/17 6:35 AM, Michael Truog wrote: > >> As I argued in the original pull request for these recent 20.0 random > >> number changes, a uniform distribution is much more intuitive if it is > >> inclusive: [0,1] > > > > Intuitive? For integers, I'll grant that. For reals? Not so much. > > I certainly cannot grant "MUCH more intuitive". > > I've had occasion to do (random next * 2 - 1) arcTanh, which of > > course breaks down if *either* 0 or 1 is returned by (random next). > > A uniform distribution should be uniformly distributed. I understand the woes of floating-point prevent perfect uniform distribution, but we could at least try to pay attention to the limits involved, and if we did, that would make the idea much more intuitive. If I try to be philosophical, picking a random number in the range 0.0 to 1.0 of real numbers, the probability of getting a number exactly 0.0 (or exactly 1.0) is infinitely low. Therefore the range (0.0,1.0) is more natural. > > My belief is that the [0,1) distribution is the most common because it is the easiest to implement given the IEEE floating point standard format. However, I would also like to be proven wrong, to have more faith in the current situation. I think that is very possible. We can not forget the fact that digital floating point numbers will always be some kind of integer values in disguise. : > > I have some examples that can make this desire a bit clearer: > > https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 > > % use Box-Muller transformation to generate Gaussian noise > % (G. E. P. Box and Mervin E. Muller, > % A Note on the Generation of Random Normal Deviates, > % The Annals of Mathematical Statistics (1958), > % Vol. 29, No. 2 pp. 610?611) > X1 = random(), > X2 = PI2 * random(), > K = StdDev * math:sqrt(-2.0 * math:log(X1)), math:log(X1) will badarith if X1 =:= 0.0. You need a generator for X1 that does not return 0.0, just as RO'K says. > Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), > Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), If random() for X2 is in [0.0,1.0] then both 0.0 and 1.0 will produce the same value after math:cos(X2) or math:sin(X2), which I am convinced will bias the result since that particular value will have twice the probability compared to all other values. I think you should use a generator for X2 that only can return one of the endpoints. Actually, it seems a generator for (0.0,1.0] would be more appropriate here... > sleep(Result2), > > > https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L204-L210 > > X = random(), > if > X =< Percent -> > erlang:exit(monkey_chaos); > true -> > ok > end, In this kind of code, I think that (when thinking integers, since we are talking about integers in disguise) half open intervals are more correct. The interval [0.0,0.1] contains say N+1 numbers, the interval [0.0,0.2] contains 2*N+1 nubers so subtracting the first interval from the second would get the interval (1.0,2.0) which have N numbers. So you get a bias because you include both endpoints. In this case I believe more in a generator that gives [0.0,1.0) and the test X < Percent, since that is what I would have written using integers to avoid off-by-one errors. > > with: > random() -> > quickrand:strong_float(). > > These are code segments used for the CloudI service configuration options monkey_latency and monkey_chaos so that normal distribution latency values and random service deaths can occur, respectively (with the more common names as Latency Monkey and Chaos Monkey, but the words switched to make the concepts easier to find and associate). For the Box-Muller transformation, it really does want a definite range [0,1] and it helps make the monkey_chaos service death easier to understand at a glance. Please explain why the Box-Muller transformation needs a definite range [0.0,1.0]. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Sep 1 10:49:58 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 10:49:58 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> Message-ID: <20170901084958.GC87875@erix.ericsson.se> On Fri, Sep 01, 2017 at 02:57:49PM +1200, Richard A. O'Keefe wrote: : > > I could personally live with a warning in the documentation that says > that the random number generator could return 0.0, and here's a little > loop you might use to avoid that, and another suggestion in the code > about how to get the result Michael Truog wants. > > I just want it to be obvious that it's dangerous to assume that the > result will not be 0. That can surely merit a warning, even though the interval is documented. > > By the way, given that a common way to make random floats is to > generate a bitvector, consider > (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. > You will notice that the spacing between the values is *almost* > uniform, but not at the end. That sounds interesting but I do not understand. Is that Elixir code? -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Sep 1 10:54:39 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 10:54:39 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59A8F03E.9040400@gmail.com> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> Message-ID: <20170901085439.GD87875@erix.ericsson.se> On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: : > > I have some examples that can make this desire a bit clearer: > > https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 > > % use Box-Muller transformation to generate Gaussian noise > % (G. E. P. Box and Mervin E. Muller, > % A Note on the Generation of Random Normal Deviates, > % The Annals of Mathematical Statistics (1958), > % Vol. 29, No. 2 pp. 610?611) > X1 = random(), > X2 = PI2 * random(), > K = StdDev * math:sqrt(-2.0 * math:log(X1)), > Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), > Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), > sleep(Result2), Why not use rand:normal/3? It uses the Ziggurat Method and is supposed to be much faster and numerically more stable than the basic Box-Muller method. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From hugo@REDACTED Fri Sep 1 11:36:37 2017 From: hugo@REDACTED (Hugo Mills) Date: Fri, 1 Sep 2017 09:36:37 +0000 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901084115.GB87875@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901084115.GB87875@erix.ericsson.se> Message-ID: <20170901093637.GF30990@carfax.org.uk> On Fri, Sep 01, 2017 at 10:41:15AM +0200, Raimo Niskanen wrote: > On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > > On 08/31/2017 07:57 PM, Richard A. O'Keefe wrote: > > > > > > On 1/09/17 6:35 AM, Michael Truog wrote: > > >> As I argued in the original pull request for these recent 20.0 random > > >> number changes, a uniform distribution is much more intuitive if it is > > >> inclusive: [0,1] > > > > > > Intuitive? For integers, I'll grant that. For reals? Not so much. > > > I certainly cannot grant "MUCH more intuitive". > > > I've had occasion to do (random next * 2 - 1) arcTanh, which of > > > course breaks down if *either* 0 or 1 is returned by (random next). > > > > A uniform distribution should be uniformly distributed. I understand the woes of floating-point prevent perfect uniform distribution, but we could at least try to pay attention to the limits involved, and if we did, that would make the idea much more intuitive. > > If I try to be philosophical, picking a random number in the range > 0.0 to 1.0 of real numbers, the probability of getting a number exactly 0.0 > (or exactly 1.0) is infinitely low. Therefore the range (0.0,1.0) is more > natural. Mathematically, there's a distinction. What you've just described is that in a random variable over the interval [0.0, 1.0], 0.0 and 1.0 happen *almost never* (which is a very specific technical term), and that values in the open interval (0.0, 1.0) occur *almost surely*. Being discrete, the computer implementation based on floating point numbers ensures that the probability of getting 0.0 or 1.0 in that case is measurably non-zero, whereas in the ideal version over the reals, above, it is infinitesimally small. In that distinction lie most of the problems that people are talking about here, I think. > > My belief is that the [0,1) distribution is the most common > > because it is the easiest to implement given the IEEE floating > > point standard format. However, I would also like to be proven > > wrong, to have more faith in the current situation. > I think that is very possible. From my relatively limited practical experience, either I've wanted [0, 1) or I don't care. Example: Red = int(random() * 256) where I don't want the value 256, because it's out of range for my 8-bit graphics mode, but I do want the probability of 255 to be the same as every other value. So I want [0, 1) as my range. Alternatively: P = random(), if P =< 0.3 -> ...; P =< 0.7 -> ...; P > 0.7 -> ... end where, in general, I don't care if I could get 0.0 or 1.0 or not, because the differences are immeasurably small for all practical purposes. I think it's clear to me that _several_ functions are needed for different cases, with fully-closed, fully-open and half-open intervals. IMO, the fully-closed and half-open are probably the most useful (and, modulo any floating-point issues which I'm not qualified to talk about, [0,1) can be turned into (0,1] with 1-random_halfopen()). With a fully-closed interval, it should be possible to write helpers for generating the other three by simply calling random_closed() again if you get an undesirable end-point. You can't easily extend the range of the half-open or open intervals to give you the closed ones. So I'd say at minimum, there should be a function giving the closed interval. Whether the "test and retry" approach is the best implementation or not is a matter for discussion, as is the question of whether all or some of these functions should be in the standard lib, or they are expected to be hacked together on an as-needed basis. Hugo. -- Hugo Mills | Anyone who says their system is completely secure hugo@REDACTED carfax.org.uk | understands neither systems nor security. http://carfax.org.uk/ | PGP: E2AB1DE4 | Bruce Schneier -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From mjtruog@REDACTED Fri Sep 1 12:53:37 2017 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 1 Sep 2017 03:53:37 -0700 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901084115.GB87875@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901084115.GB87875@erix.ericsson.se> Message-ID: <59A93C31.4050006@gmail.com> On 09/01/2017 01:41 AM, Raimo Niskanen wrote: >> I have some examples that can make this desire a bit clearer: >> >> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 >> >> % use Box-Muller transformation to generate Gaussian noise >> % (G. E. P. Box and Mervin E. Muller, >> % A Note on the Generation of Random Normal Deviates, >> % The Annals of Mathematical Statistics (1958), >> % Vol. 29, No. 2 pp. 610?611) >> X1 = random(), >> X2 = PI2 * random(), >> K = StdDev * math:sqrt(-2.0 * math:log(X1)), > math:log(X1) will badarith if X1 =:= 0.0. You need a generator for X1 > that does not return 0.0, just as RO'K says. > >> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), >> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), > If random() for X2 is in [0.0,1.0] then both 0.0 and 1.0 will produce the > same value after math:cos(X2) or math:sin(X2), which I am convinced will > bias the result since that particular value will have twice the probability > compared to all other values. I think you should use a generator for X2 > that only can return one of the endpoints. > > Actually, it seems a generator for (0.0,1.0] would be more appropriate > here... > >> sleep(Result2), >> >> with: >> random() -> >> quickrand:strong_float(). >> >> These are code segments used for the CloudI service configuration options monkey_latency and monkey_chaos so that normal distribution latency values and random service deaths can occur, respectively (with the more common names as Latency Monkey and Chaos Monkey, but the words switched to make the concepts easier to find and associate). For the Box-Muller transformation, it really does want a definite range [0,1] and it helps make the monkey_chaos service death easier to understand at a glance. > Please explain why the Box-Muller transformation needs a definite range > [0.0,1.0]. That was my understanding after not having modified that routine for a decent amount of time, though I must be mistaken. I will need to fix this source code and regret not seeing these problems in the Box-Muller transformation source code. Thank you for pointing them out. At least this shows a need for a (0.0,1.0] function. Thanks, Michael From mjtruog@REDACTED Fri Sep 1 13:00:59 2017 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 1 Sep 2017 04:00:59 -0700 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901085439.GD87875@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901085439.GD87875@erix.ericsson.se> Message-ID: <59A93DEB.6000205@gmail.com> On 09/01/2017 01:54 AM, Raimo Niskanen wrote: > On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > : >> I have some examples that can make this desire a bit clearer: >> >> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 >> >> % use Box-Muller transformation to generate Gaussian noise >> % (G. E. P. Box and Mervin E. Muller, >> % A Note on the Generation of Random Normal Deviates, >> % The Annals of Mathematical Statistics (1958), >> % Vol. 29, No. 2 pp. 610?611) >> X1 = random(), >> X2 = PI2 * random(), >> K = StdDev * math:sqrt(-2.0 * math:log(X1)), >> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), >> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), >> sleep(Result2), > Why not use rand:normal/3? > > It uses the Ziggurat Method and is supposed to be much faster and > numerically more stable than the basic Box-Muller method. > The Box-Muller is simpler and producing 2 results instead of 1 . I believe I looked at the source code for rand:normal/3 and expected the Box-Muller to be faster only because it creates 2 results, though I should check that. I will have to investigate it more. From arif@REDACTED Fri Sep 1 13:48:15 2017 From: arif@REDACTED (arif@REDACTED) Date: Fri, 01 Sep 2017 11:48:15 +0000 Subject: [erlang-questions] common test not working in OTP R20 Message-ID: <464854be0365860d1ed4510a2e809b2f@riseup.net> I just can't get common test to run on built-from-source OTP 20.0 on a linux machine. Trying to run just the skeleton of a small common test suite, some_SUITE, I get a function_clause error: failed: {error, {ct_framework,init_tc, {function_clause, [{ct_hooks,init_tc, [some_SUITE,my_test_case, [{tc_logfile, ".."}, {tc_group_properties,[]}, {tc_group_path,[]}, {data_dir, ".."}, {priv_dir, ""}]], [{file,"ct_hooks.erl"},{line,81}]}, {ct_framework,ct_suite_init,4, [{file,"ct_framework.erl"},{line,280}]}, {test_server_sup,framework_call,4, [{file,"test_server_sup.erl"},{line,772}]}, {test_server,run_test_case_eval,9, [{file,"test_server.erl"},{line,920}]}]}}}, {ct_framework, init_tc} I have looked through the source and, for example, in common_test-1.11 we had init_tc(_Mod, TC, Config) -> call(fun call_generic/3, Config, [pre_init_per_testcase, TC]). However, in common_test-1.15 (OTP 20.0), we have: init_tc(Mod, {init_per_testcase,TC}, Config) -> call(fun call_generic_fallback/3, Config, [pre_init_per_testcase, Mod, TC]); init_tc(Mod, {end_per_testcase,TC}, Config) -> call(fun call_generic_fallback/3, Config, [pre_end_per_testcase, Mod, TC]); init_tc(Mod, TC = error_in_suite, Config) -> call(fun call_generic_fallback/3, Config, [pre_init_per_testcase, Mod, TC]). Would this mean that in ct_framework:init_tc ct_suite_init(Suite,HookFunc,PostInitHook,Config) when is_list(Config) -> case ct_hooks:init_tc(Suite,HookFunc,Config) of ... HookFunc should be {init_per_testcase, TC} and not TC, as it appears to be happening? I have also tried the pre-built OTP R20.0 for windows, and that works fine. What am I doing wrong? Thanks and best regards Arif From frank.muller.erl@REDACTED Fri Sep 1 13:58:17 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 01 Sep 2017 11:58:17 +0000 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> Message-ID: Can someone help on this please? Thanks /Frank Le mer. 30 ao?t 2017 at 07:21, Frank Muller wrote : > Hi Prakash, > > Thanks, but I doubt it's that easy. > > In C, you've to call fflush or set the stdout to unbuffer mode. > > /Frank > > Le mer. 30 ao?t 2017 ? 06:51, Prakash Parmar > wrote : > >> Hi Frank, >> >> >> >> As per my experience, Like in C, keeping newline "~n" in io:format will >> flush it to screen. >> >> >> >> /Prakash >> >> >> >> -----Original Message----- >> From: "Frank Muller" >> Sent: Wednesday, August 30, 2017 9:58am >> To: "Erlang-Questions Questions" >> Subject: [erlang-questions] Flush "stdout" buffer >> >> Hi guys, >> I'd like to know if there's an equivalent function to C fflush(stdout) ? >> I've a small Erlang program which prints data to stdout. After every >> io:format call, I'd like to ensure output is flushed to screen. >> Thanks in adavance. >> /Frank >> >> >> >> ******* DISCLAIMER: This email and any files transmitted with it are >> privileged and confidential information and intended solely for the use of >> the individual or entity to which they are addressed. If you are not the >> intended recipient, please notify the sender by e-mail and delete the >> original message. Further, you are not to copy, disclose, or distribute >> this e-mail or its contents to any other person and any such actions are >> unlawful. This e-mail may contain viruses. Torry Harris Business Solutions >> has taken every reasonable precaution to minimize this risk, but is not >> liable for any damage you may sustain as a result of any virus in this >> e-mail. The recipient should check this email and any attachments for the >> presence of viruses. THBS reserves the right to monitor and review the >> content of all messages sent to or from this e-mail address******** >> >> ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Sep 1 14:06:44 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 14:06:44 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59A93C31.4050006@gmail.com> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901084115.GB87875@erix.ericsson.se> <59A93C31.4050006@gmail.com> Message-ID: <20170901120644.GA11300@erix.ericsson.se> On Fri, Sep 01, 2017 at 03:53:37AM -0700, Michael Truog wrote: > On 09/01/2017 01:41 AM, Raimo Niskanen wrote: > >> I have some examples that can make this desire a bit clearer: > >> > >> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 > >> > >> % use Box-Muller transformation to generate Gaussian noise > >> % (G. E. P. Box and Mervin E. Muller, > >> % A Note on the Generation of Random Normal Deviates, > >> % The Annals of Mathematical Statistics (1958), > >> % Vol. 29, No. 2 pp. 610?611) > >> X1 = random(), > >> X2 = PI2 * random(), > >> K = StdDev * math:sqrt(-2.0 * math:log(X1)), > > math:log(X1) will badarith if X1 =:= 0.0. You need a generator for X1 > > that does not return 0.0, just as RO'K says. > > > >> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), > >> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), > > If random() for X2 is in [0.0,1.0] then both 0.0 and 1.0 will produce the > > same value after math:cos(X2) or math:sin(X2), which I am convinced will > > bias the result since that particular value will have twice the probability > > compared to all other values. I think you should use a generator for X2 > > that only can return one of the endpoints. > > > > Actually, it seems a generator for (0.0,1.0] would be more appropriate > > here... > > > >> sleep(Result2), > >> > >> with: > >> random() -> > >> quickrand:strong_float(). > >> > >> These are code segments used for the CloudI service configuration options monkey_latency and monkey_chaos so that normal distribution latency values and random service deaths can occur, respectively (with the more common names as Latency Monkey and Chaos Monkey, but the words switched to make the concepts easier to find and associate). For the Box-Muller transformation, it really does want a definite range [0,1] and it helps make the monkey_chaos service death easier to understand at a glance. > > Please explain why the Box-Muller transformation needs a definite range > > [0.0,1.0]. > > That was my understanding after not having modified that routine for a decent amount of time, though I must be mistaken. I will need to fix this source code and regret not seeing these problems in the Box-Muller transformation source code. Thank you for pointing them out. At least this shows a need for a (0.0,1.0] function. Yes. That is easily produced by (pointed out earlier in this thread): 1.0 - rand:uniform() > > Thanks, > Michael -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Sep 1 14:13:46 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 14:13:46 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59A93DEB.6000205@gmail.com> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901085439.GD87875@erix.ericsson.se> <59A93DEB.6000205@gmail.com> Message-ID: <20170901121346.GB11300@erix.ericsson.se> On Fri, Sep 01, 2017 at 04:00:59AM -0700, Michael Truog wrote: > On 09/01/2017 01:54 AM, Raimo Niskanen wrote: > > On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > > : > >> I have some examples that can make this desire a bit clearer: > >> > >> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 > >> > >> % use Box-Muller transformation to generate Gaussian noise > >> % (G. E. P. Box and Mervin E. Muller, > >> % A Note on the Generation of Random Normal Deviates, > >> % The Annals of Mathematical Statistics (1958), > >> % Vol. 29, No. 2 pp. 610?611) > >> X1 = random(), > >> X2 = PI2 * random(), > >> K = StdDev * math:sqrt(-2.0 * math:log(X1)), > >> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), > >> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), > >> sleep(Result2), > > Why not use rand:normal/3? > > > > It uses the Ziggurat Method and is supposed to be much faster and > > numerically more stable than the basic Box-Muller method. > > > The Box-Muller is simpler and producing 2 results instead of 1 . I believe I looked at the source code for rand:normal/3 and expected the Box-Muller to be faster only because it creates 2 results, though I should check that. I will have to investigate it more. Simpler - yes. The basic benchmark in rand_SUITE indicates that rand:normal() is only about 50% slower than rand:uniform(1 bsl 58) (internal word size), which I think is a very good number. The Box-Muller transform method needs 4 calls to the 'math' module for non-trivial floating point functions i.e log(), sqrt(), cos() and sin(), which is why I think that "must" be slower. But I have also not measured... :-/ Looking forward to hear your results! -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Sep 1 14:53:38 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 14:53:38 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901093637.GF30990@carfax.org.uk> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901084115.GB87875@erix.ericsson.se> <20170901093637.GF30990@carfax.org.uk> Message-ID: <20170901125338.GC11300@erix.ericsson.se> On Fri, Sep 01, 2017 at 09:36:37AM +0000, Hugo Mills wrote: > On Fri, Sep 01, 2017 at 10:41:15AM +0200, Raimo Niskanen wrote: > > On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > > > On 08/31/2017 07:57 PM, Richard A. O'Keefe wrote: > > > > > > > > On 1/09/17 6:35 AM, Michael Truog wrote: > > > >> As I argued in the original pull request for these recent 20.0 random > > > >> number changes, a uniform distribution is much more intuitive if it is > > > >> inclusive: [0,1] > > > > > > > > Intuitive? For integers, I'll grant that. For reals? Not so much. > > > > I certainly cannot grant "MUCH more intuitive". > > > > I've had occasion to do (random next * 2 - 1) arcTanh, which of > > > > course breaks down if *either* 0 or 1 is returned by (random next). > > > > > > A uniform distribution should be uniformly distributed. I understand the woes of floating-point prevent perfect uniform distribution, but we could at least try to pay attention to the limits involved, and if we did, that would make the idea much more intuitive. > > > > If I try to be philosophical, picking a random number in the range > > 0.0 to 1.0 of real numbers, the probability of getting a number exactly 0.0 > > (or exactly 1.0) is infinitely low. Therefore the range (0.0,1.0) is more > > natural. > > Mathematically, there's a distinction. What you've just described > is that in a random variable over the interval [0.0, 1.0], 0.0 and 1.0 > happen *almost never* (which is a very specific technical term), and > that values in the open interval (0.0, 1.0) occur *almost surely*. > > Being discrete, the computer implementation based on floating point > numbers ensures that the probability of getting 0.0 or 1.0 in that > case is measurably non-zero, whereas in the ideal version over the > reals, above, it is infinitesimally small. In that distinction lie > most of the problems that people are talking about here, I think. Precisely! > > > > My belief is that the [0,1) distribution is the most common > > > because it is the easiest to implement given the IEEE floating > > > point standard format. However, I would also like to be proven > > > wrong, to have more faith in the current situation. > > > I think that is very possible. > > From my relatively limited practical experience, either I've wanted > [0, 1) or I don't care. Example: > > Red = int(random() * 256) > > where I don't want the value 256, because it's out of range for my > 8-bit graphics mode, but I do want the probability of 255 to be the > same as every other value. So I want [0, 1) as my range. > > Alternatively: > > P = random(), > if > P =< 0.3 -> ...; > P =< 0.7 -> ...; > P > 0.7 -> ... > end > > where, in general, I don't care if I could get 0.0 or 1.0 or not, > because the differences are immeasurably small for all practical > purposes. Especially since decimal numbers below 1.0 have no exact representation as IEEE floating point numbers. > > I think it's clear to me that _several_ functions are needed for > different cases, with fully-closed, fully-open and half-open > intervals. IMO, the fully-closed and half-open are probably the most > useful (and, modulo any floating-point issues which I'm not qualified > to talk about, [0,1) can be turned into (0,1] with > 1-random_halfopen()). There should be no issues in this case with the updated algorithms in the 'rand' module that produce numbers on the form N * 2^-53, and due to the fact that IEEE floating point arithmetics is defined to produce correctly rounded results for the simple operations +, -, *, /, ... > > With a fully-closed interval, it should be possible to write > helpers for generating the other three by simply calling > random_closed() again if you get an undesirable end-point. You can't > easily extend the range of the half-open or open intervals to give you > the closed ones. So I'd say at minimum, there should be a function > giving the closed interval. The closed interval [0.0,1.0] may be the most general for the reasons you mention. Unfortunately it is cumbersome to implement "efficiently". You would have to generate integers [0, 1+2^53] and to do that with a 58 bit generator (the default) you generate one number and have roughly a 1/32 chance of landing in the high interval where you need to retry. The amortized impact of this is only about 3% slower, but the code has to do all the testing for these corner cases which adds up to maybe 20% slower. But once in a million it will take 4 full attempts or more to get a number, which is rather annoying, and the worst case is infinity (but will never happen ;-). Therefore the half open interval [0.0,1.0) is probably the most useful one, and the question is if the closed interval [0.0,1.0] (and the open interval (0.0,1.0) is worth to implement... > > Whether the "test and retry" approach is the best implementation > or not is a matter for discussion, as is the question of whether all I have never heard of a better alternative than "test and retry" when you are limiting the interval, except possibly when the probability for retry is high (approaching 50%) - then you may consider to generate twice as many bits and do a simple 'mod' operation for the result; the skew would be impossible to notice. The "test and retry" method is still expected to be as fast or faster, amortized, but the "generate double and 'mod'" method has more or less fixed execution time. > or some of these functions should be in the standard lib, or they are > expected to be hacked together on an as-needed basis. Yes. That is the question. > > Hugo. > > -- > Hugo Mills | Anyone who says their system is completely secure > hugo@REDACTED carfax.org.uk | understands neither systems nor security. > http://carfax.org.uk/ | > PGP: E2AB1DE4 | Bruce Schneier -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Fri Sep 1 14:57:04 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 1 Sep 2017 14:57:04 +0200 Subject: [erlang-questions] How safe is it to leave an open SSL port on the public internet? In-Reply-To: <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> References: <55FFE7D1-782C-4FEF-B23D-BE292BCA5574@gmail.com> <1DAAF3BE-D958-45C8-AA75-2F71EC5D9E68@gmail.com> <20170829221959.GA51868@ferdmbp.local> <20170829222354.GB51868@ferdmbp.local> <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> Message-ID: <8021c22d-c101-f8f9-4ad2-f84275783f30@ninenines.eu> On 08/30/2017 08:03 PM, code wiget wrote: > Also, Fred, I re-read your post and wanted to either start a quick > discussion/warn you about elliptic curves. According to the NSA: "the > growth of elliptic curve use has bumped up against the fact of continued > progress in the research on quantum computing, which has made it clear > that *elliptic curve cryptography is not the long term solution many > once hoped it would be.**?* > * > * > The NSA has deprecated ECC, whether or not that means that some foreign > actor has a crack or if they are that worried about quantum computing is > to be seen, but for now it seems like we should be moving away from ECC. Surely the NSA's bigger concern is that they can't crack it today, rather than it being too weak in the future. -- Lo?c Hoguin https://ninenines.eu From per@REDACTED Fri Sep 1 14:58:44 2017 From: per@REDACTED (Per Hedeland) Date: Fri, 1 Sep 2017 14:58:44 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> Message-ID: On 2017-09-01 13:58, Frank Muller wrote: > Can someone help on this please? Well, what is the problem that you want to solve? Erlang doesn't really have an equivalent to C stdio, and anything you write with io:format/1,2,3 to "standard output" is sent immediately to the OS via a write(2) (or typically writev(2)) call. Are you seeing something else? Try this: erl -noshell -eval 'io:format("foo "),timer:sleep(2000),io:format("bar~n"),timer:sleep(2000),erlang:halt(0)' --Per > Thanks > /Frank > > Le mer. 30 ao?t 2017 at 07:21, Frank Muller > wrote : > > Hi Prakash, > > Thanks, but I doubt it's that easy. > > In C, you've to call fflush or set the stdout to unbuffer mode. > > /Frank > > Le mer. 30 ao?t 2017 ? 06:51, Prakash Parmar > wrote : > > Hi Frank, > > As per my experience, Like in C, keeping newline "~n" in io:format will flush it to screen. > > /Prakash > > -----Original Message----- > From: "Frank Muller" > > Sent: Wednesday, August 30, 2017 9:58am > To: "Erlang-Questions Questions" > > Subject: [erlang-questions] Flush "stdout" buffer > > Hi guys, > I'd like to know if there's an equivalent function to C fflush(stdout) ? > I've a small Erlang program which prints data to stdout. After every io:format call, I'd like to ensure output is flushed to screen. > Thanks in adavance. > /Frank > > > > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are > addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its > contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, > but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves > the right to monitor and review the content of all messages sent to or from this e-mail address******** > > ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.******** > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zxq9@REDACTED Fri Sep 1 15:03:10 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 01 Sep 2017 22:03:10 +0900 Subject: [erlang-questions] How safe is it to leave an open SSL port on the public internet? In-Reply-To: <8021c22d-c101-f8f9-4ad2-f84275783f30@ninenines.eu> References: <55FFE7D1-782C-4FEF-B23D-BE292BCA5574@gmail.com> <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> <8021c22d-c101-f8f9-4ad2-f84275783f30@ninenines.eu> Message-ID: <3840471.JKSsV42BRn@changa> On 2017?09?01? ??? 14:57:04 Lo?c Hoguin wrote: > On 08/30/2017 08:03 PM, code wiget wrote: > > Also, Fred, I re-read your post and wanted to either start a quick > > discussion/warn you about elliptic curves. According to the NSA: "the > > growth of elliptic curve use has bumped up against the fact of continued > > progress in the research on quantum computing, which has made it clear > > that *elliptic curve cryptography is not the long term solution many > > once hoped it would be.**?* > > * > > * > > The NSA has deprecated ECC, whether or not that means that some foreign > > actor has a crack or if they are that worried about quantum computing is > > to be seen, but for now it seems like we should be moving away from ECC. > > Surely the NSA's bigger concern is that they can't crack it today, > rather than it being too weak in the future. The NSA's biggest concern is that once they realize they have a lead on cracking something there is an instant (and very well founded) fear reaction that someone else must have already achieved this, but never announced it. That is the #1 priority of the NSA in every fiber of its being. Secondarily, they have to keep trying to crack things, but for practical operational purposes HUMINT trumps SIGINT in almost every way, every day, and they aren't blind to this. Cracks are only a small part of the NSA's game; the vast majority of what they collect is given to them willfully, one way or another. -Craig From essen@REDACTED Fri Sep 1 15:13:18 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 1 Sep 2017 15:13:18 +0200 Subject: [erlang-questions] How safe is it to leave an open SSL port on the public internet? In-Reply-To: <3840471.JKSsV42BRn@changa> References: <55FFE7D1-782C-4FEF-B23D-BE292BCA5574@gmail.com> <18BC0FBC-2A70-4D1B-A3F8-5E022B4DF45C@gmail.com> <8021c22d-c101-f8f9-4ad2-f84275783f30@ninenines.eu> <3840471.JKSsV42BRn@changa> Message-ID: <0f273b59-6621-a542-3fb6-d6d9987aeeaa@ninenines.eu> On 09/01/2017 03:03 PM, zxq9 wrote: > On 2017?09?01? ??? 14:57:04 Lo?c Hoguin wrote: >> On 08/30/2017 08:03 PM, code wiget wrote: >>> Also, Fred, I re-read your post and wanted to either start a quick >>> discussion/warn you about elliptic curves. According to the NSA: "the >>> growth of elliptic curve use has bumped up against the fact of continued >>> progress in the research on quantum computing, which has made it clear >>> that *elliptic curve cryptography is not the long term solution many >>> once hoped it would be.**?* >>> * >>> * >>> The NSA has deprecated ECC, whether or not that means that some foreign >>> actor has a crack or if they are that worried about quantum computing is >>> to be seen, but for now it seems like we should be moving away from ECC. >> >> Surely the NSA's bigger concern is that they can't crack it today, >> rather than it being too weak in the future. > > The NSA's biggest concern is that once they realize they have a lead on cracking something there is an instant (and very well founded) fear reaction that someone else must have already achieved this, but never announced it. That is the #1 priority of the NSA in every fiber of its being. In a post-Snowden world it should be obvious that this is not always the case. Anyway I just wanted to provide some healthy skepticism. -- Lo?c Hoguin https://ninenines.eu From raimo+erlang-questions@REDACTED Fri Sep 1 16:52:24 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 1 Sep 2017 16:52:24 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170831123008.GA73645@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> Message-ID: <20170901145224.GA28397@erix.ericsson.se> On Thu, Aug 31, 2017 at 02:30:08PM +0200, Raimo Niskanen wrote: : > Should I make a pull request of this? > > https://github.com/erlang/otp/compare/OTP-20.0...RaimoNiskanen:raimo/stdlib/rand-uniformNZ > > Is the name uniformNZ good enough? > Are uniform floats complete enough with this addition? I have updated the suggestion above. Comments? -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From lloyd@REDACTED Fri Sep 1 21:30:47 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Fri, 1 Sep 2017 15:30:47 -0400 (EDT) Subject: [erlang-questions] Building a DAW in Erlang In-Reply-To: <21cb1b25-6914-1650-1f9a-5feff1c48d4c@cs.otago.ac.nz> References: <1504225327.54672404@apps.rackspace.com> <21cb1b25-6914-1650-1f9a-5feff1c48d4c@cs.otago.ac.nz> Message-ID: <1504294247.065413595@apps.rackspace.com> Hi, So, great ignorance speculating here, if we can generate basic wave forms with Erlang terms then, next, we need to be able to edit, mix, amplify, attenuate, play with phase relationships, etc., etc. My superficial Google search didn't come up with much. This exchange touches on the issues: https://stackoverflow.com/questions/1569035/what-makes-erlang-suitable-for-soft-real-time-applications But it does feel like a vast field for bottom-up play. Shame that there are so few hours in the day. All the best, Lloyd -----Original Message----- From: "Richard A. O'Keefe" Sent: Thursday, August 31, 2017 11:44pm To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Building a DAW in Erlang On 1/09/17 12:22 PM, lloyd@REDACTED wrote: > > 1. Is it possible to write an Erlang term that generates an approximation of white noise? > 2. Is it possible to write an Erlang term that generates short sequences of basic wave forms--- sine, triangle, square? > 3. Is there a programmatic way to edit the generated binaries to improve aesthetic quality? Since the external term encoding of a binary includes the bytes of that binary verbatim, yes, yes, and yes. See BINARY_EXT in http://erlang.org/doc/apps/erts/erl_ext_dist.html _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Sat Sep 2 07:12:09 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 2 Sep 2017 08:12:09 +0300 Subject: [erlang-questions] Building a DAW in Erlang In-Reply-To: <1504294247.065413595@apps.rackspace.com> References: <1504225327.54672404@apps.rackspace.com> <21cb1b25-6914-1650-1f9a-5feff1c48d4c@cs.otago.ac.nz> <1504294247.065413595@apps.rackspace.com> Message-ID: Just short thing: if you decide to play audio in Mac, you will suffer from "main thread problem". Almost all specific mac API calls require you to call them from "main thread" of application that is inaccessible by erlang code (nif or driver). It was so several years ago and I don't know if this has changed. About performance, stability and so else. We in Flussonic have a serious challenge with sending UDP. We need to prepare and send UDP packets (mpegts) so that they form a very good and stable stream with stable bitrate and inter-packet time. This is sent to DVB headend that sends video to cable or even to satellite. Problem is that this task sometimes require submillisecond timer precision that is inaccessible in erlang. Our code works ok with dozens of channels but without submillisecond precision. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Sep 2 07:22:36 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 2 Sep 2017 08:22:36 +0300 Subject: [erlang-questions] Erlang HTTP client libraries- pros/cons In-Reply-To: References: Message-ID: HTTP client over external port is the most expensive way in all terms: 1) programming. It is REALLY hard to debug it. Was it launched under valgrind? If no, then there are at least 5 horrible memory leaks and memory corruptions per each screen of code that haven't been discovered yet 2) deploying. Deploying NIF is a pain because you need to have build farm for each architecture that you target to. For example, we deploy flussonic under suse 10, debian 7, debian8/ubuntu16, arm7, arm8, windows, elbrus 2k. All these platforms are different and you cannot rely on cross compiler. Good luck with building repeatable infrastructure for compiling under windows. If you have erlang code, you can compile under mac and launch under windows because guys from OTP team have already done all dirty job. Just read their manual and follow it. 3) speed. It will be slow in all terms. High latency due to multiple OS process scheduling: read in one process, then write to pipe and send further. Do you think that linux pipe is a "good and optimized" thing? It is not. What if we speak about high traffic? 1 gigabit of input will become several gigabits of _useless_ copying data. So I do not understand what can give libcurl that cannot be achieved in plain erlang. It is definitely not about high traffic speed because plain vanilla lhttpc can download 10 Gbit/s over fiber without any extra modifications. On Thu, Aug 31, 2017 at 10:22 AM, Paul Oliver wrote: > On Thu, Aug 31, 2017 at 6:20 PM Taras Halturin wrote: > >> I think, Max means that you choose most expensive way to deal with it and >> it's not about efficiency of http handling but about efficiency of aim >> achieving :) >> >> > The most expensive way in terms of what? If not speed do you mean > development effort? Given that the aim is to use libcurl then the choice is > a port executable or some sort of NIF. When using a port executable I don't > have to worry about it crashing my VM and all I pay is the price of port > communications. If I use a NIF I have to concern myself with making sure my > NIF code and the code in libcurl doesn't crash my VM. That's a lot more > development time and risk. > > Cheers, > Paul. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Sat Sep 2 08:06:43 2017 From: puzza007@REDACTED (Paul Oliver) Date: Sat, 02 Sep 2017 06:06:43 +0000 Subject: [erlang-questions] Erlang HTTP client libraries- pros/cons In-Reply-To: References: Message-ID: On Sat, Sep 2, 2017 at 5:22 PM Max Lapshin wrote: > HTTP client over external port is the most expensive way in all terms: > > 1) programming. It is REALLY hard to debug it. Was it launched under > valgrind? If no, then there are at least 5 horrible memory leaks and memory > corruptions per each screen of code that haven't been discovered yet > And yet you wrote https://github.com/maxlapshin/csv_reader (which I use btw). Running port executables protects the VM from any bugs in the C code, at least. It's almost like Erlang in that regard! Valgrind was used during Katipo's development. > > 2) deploying. Deploying NIF is a pain because you need to have build farm > for each architecture that you target to. For example, we deploy flussonic > under suse 10, debian 7, debian8/ubuntu16, arm7, arm8, windows, elbrus 2k. > All these platforms are different and you cannot rely on cross compiler. > Good luck with building repeatable infrastructure for compiling under > windows. > > If you have erlang code, you can compile under mac and launch under > windows because guys from OTP team have already done all dirty job. Just > read their manual and follow it. > This I will concede can be an issue with NIFs. It's not an issue for my use case, however, and can be overcome by using things like Docker. > > 3) speed. It will be slow in all terms. High latency due to multiple OS > process scheduling: read in one process, then write to pipe and send > further. Do you think that linux pipe is a "good and optimized" thing? It > is not. > What if we speak about high traffic? 1 gigabit of input will become > several gigabits of _useless_ copying data. > > So I do not understand what can give libcurl that cannot be achieved in > plain erlang. It is definitely not about high traffic speed because plain > vanilla lhttpc can download 10 Gbit/s over fiber without any extra > modifications. > > I think perhaps you have tunnel vision based on your specific HTTP use case. I'm not sure what you mean by "in all terms". In https://github.com/lpgauth/httpc_bench (d)lhttpc peaks at 93178 reqs/s whereas Katipo peaks at 107900. You may not require the HTTP compatibility that *22K* commits to libcurl provides. Your custom version of lhttpc may well be able to sustain 10Gbit/s over fiber - that is not my use case so I've no idea if Katipo could do the same. > > On Thu, Aug 31, 2017 at 10:22 AM, Paul Oliver wrote: > >> On Thu, Aug 31, 2017 at 6:20 PM Taras Halturin >> wrote: >> >>> I think, Max means that you choose most expensive way to deal with it >>> and it's not about efficiency of http handling but about efficiency of aim >>> achieving :) >>> >>> >> The most expensive way in terms of what? If not speed do you mean >> development effort? Given that the aim is to use libcurl then the choice is >> a port executable or some sort of NIF. When using a port executable I don't >> have to worry about it crashing my VM and all I pay is the price of port >> communications. If I use a NIF I have to concern myself with making sure my >> NIF code and the code in libcurl doesn't crash my VM. That's a lot more >> development time and risk. >> >> Cheers, >> Paul. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From palani@REDACTED Fri Sep 1 14:36:59 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Fri, 1 Sep 2017 18:06:59 +0530 Subject: [erlang-questions] How to convert sql return data to json In-Reply-To: References: Message-ID: Hi guys, I tried with this modules https://github.com/talentdeficit/jsx, https://github.com/davisp/jiffy But its need fieldname. My Sql driver returns data like below *for checking Sql result {{metadata,2,0, {server_status,false,true,false,false,true, false,false,false,false,false,false,false}, [{field_metadata,"def","zurik","erlang", "erlang","name","name",33,135,253, <<0,0>>, 0,[]}, {field_metadata,"def","zurik","erlang", "erlang","id","id",63,11,3, <<0,0>>, 0,[]}], []}, [["some",457],["some",457],["data2sosfasfme",42]]}* *Please find my table structure below* +----------------+------+ | name | id | +----------------+------+ | some | 457 | | some | 457 | | data2sosfasfme | 42 | +----------------+------+ Its that idea to get field name On 31 August 2017 at 18:43, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > The way Erlang systems handle JSON data is the following: > > 1. In Erlang, every value passed around is an "Erlang Term". > 2. A subset of these terms are representable in JSON. > 3. JSON encoders such as 'jiffy' or 'jsx' are able to take the valid > subset from 2 and convert it into a binary containing the equivalent JSON > representation. > > Your problem can be solved by taking the data you obtain from MySQL and > then convert the data into a form which follows the valid JSON subset and > has the structure you want. Then call something like `jsx:encode(Data)` on > your `Data`. > > Some background on 2 above: Erlang terms can contain tuples such as {ok, > 37} or {error, not_connected}. These do not have a representation in JSON > (since it lacks the equivalent of a symbol/atom and tuples). So most JSON > encoders reject Erlang terms containing tuples as subterms. The details > depend on the encoder though. > > I can't remember if MySQL is able to return a result set directly as JSON, > and if it were, you have to check that the driver you use can handle such a > result as well--I can't help you much in this case, unfortunately. > > Hopefully this will get you started on a solution! > > On Thu, Aug 31, 2017 at 2:32 PM Palanikumar Gopalakrishnan < > palani@REDACTED> wrote: > >> I want to get data from mysql and convert them into json format. I tried >> with this >> >> module https://sourceforge.net/projects/erlmysql/files/ >> latest/download?source=directory >> >> >> Previously i tried this modules, >> https://github.com/Eonblast/Emysql >> >> This module give support for json conversion. But It was not support for >> Transaction. >> >> So I need to convert Sql return data to Json format. Please guide me to >> solve this challenge >> >> >> >> >> >> >> -- >> >> *Warm Regards,* >> >> *Palanikumar Gopalakrishnan *[image: ?] >> *Developer* >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hachreak@REDACTED Fri Sep 1 23:38:45 2017 From: hachreak@REDACTED (Leo) Date: Fri, 1 Sep 2017 23:38:45 +0200 Subject: [erlang-questions] Building a DAW in Erlang In-Reply-To: References: Message-ID: 2017-08-31 13:11 GMT+02:00 Joe Armstrong : > Hello, > > I want to build a "proof of concept" DAW (Digital Audio Workstation) > > Hi Joe, I think a good point where you can start is this: http://lac.linuxaudio.org/ They condensate a good number of audio expert happy to help you. :) I went some years ago to the conference and it was really cool! Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb_duffy@REDACTED Sat Sep 2 01:13:06 2017 From: jb_duffy@REDACTED (John Duffy) Date: Sat, 2 Sep 2017 00:13:06 +0100 (BST) Subject: [erlang-questions] NIF Query Message-ID: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> Hi I'm trying to follow the Interoperability Tutorial Users Guide (Version 9.0) to create a module containing some simple NIFS. I have managed to do this, however I have needed to digress from the documentation as follows... 1.) if my module name doesn't match the filename in the call to erlang:load_nif("filename", 0) I get a compile error. This is not as per the example in the documentation, i.e. -module(complex6) & erlang:load_nif("complex6_nif, 0). 2.) The appears to be a "flags" parameter missing in the {"foo", 1, foo_nif} structure in the documentation which generates a compile error... static ErlNifFunc nif_funcs[] = { {"foo", 1, foo_nif}, {"bar", 1, bar_nif} }; I have changed this to that below, which compiles, but what is the nature/value of "flags"? static ErlNifFunc nif_funcs[] = { {"foo", 1, foo_nif, 0}, {"bar", 1, bar_nif, 0} }; I may be missing something, or the Tutorial may need updating? Kind regards John -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Sep 2 08:42:43 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 02 Sep 2017 06:42:43 +0000 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> Message-ID: Hi Per, Hmmm ... So If i understand it well, we've no control on that after all. The data can still be buffered on the kernel side. Thanks /Frank Le ven. 1 sept. 2017 ? 14:58, Per Hedeland a ?crit : > On 2017-09-01 13:58, Frank Muller wrote: > > Can someone help on this please? > > Well, what is the problem that you want to solve? Erlang doesn't really > have an equivalent to C stdio, and anything you write with io:format/1,2,3 > to "standard output" is sent immediately to the OS via a write(2) (or > typically writev(2)) call. Are you seeing something else? Try this: > > erl -noshell -eval 'io:format("foo > "),timer:sleep(2000),io:format("bar~n"),timer:sleep(2000),erlang:halt(0)' > > --Per > > > Thanks > > /Frank > > > > Le mer. 30 ao?t 2017 at 07:21, Frank Muller > wrote : > > > > Hi Prakash, > > > > Thanks, but I doubt it's that easy. > > > > In C, you've to call fflush or set the stdout to unbuffer mode. > > > > /Frank > > > > Le mer. 30 ao?t 2017 ? 06:51, Prakash Parmar < > prakashkumar_parmar@REDACTED > wrote > : > > > > Hi Frank, > > > > As per my experience, Like in C, keeping newline "~n" in > io:format will flush it to screen. > > > > /Prakash > > > > -----Original Message----- > > From: "Frank Muller" frank.muller.erl@REDACTED>> > > Sent: Wednesday, August 30, 2017 9:58am > > To: "Erlang-Questions Questions" > > > Subject: [erlang-questions] Flush "stdout" buffer > > > > Hi guys, > > I'd like to know if there's an equivalent function to C > fflush(stdout) ? > > I've a small Erlang program which prints data to stdout. After > every io:format call, I'd like to ensure output is flushed to screen. > > Thanks in adavance. > > /Frank > > > > > > > > ******* DISCLAIMER: This email and any files transmitted with it > are privileged and confidential information and intended solely for the use > of the individual or entity to which they are > > addressed. If you are not the intended recipient, please notify > the sender by e-mail and delete the original message. Further, you are not > to copy, disclose, or distribute this e-mail or its > > contents to any other person and any such actions are unlawful. > This e-mail may contain viruses. Torry Harris Business Solutions has taken > every reasonable precaution to minimize this risk, > > but is not liable for any damage you may sustain as a result of > any virus in this e-mail. The recipient should check this email and any > attachments for the presence of viruses. THBS reserves > > the right to monitor and review the content of all messages sent > to or from this e-mail address******** > > > > ******* DISCLAIMER: This email and any files transmitted with it > are privileged and confidential information and intended solely for the use > of the individual or entity to which they are addressed. If you are not the > intended recipient, please notify the sender by e-mail and delete the > original message. Further, you are not to copy, disclose, or distribute > this e-mail or its contents to any other person and any such actions are > unlawful. This e-mail may contain viruses. Torry Harris Business Solutions > has taken every reasonable precaution to minimize this risk, but is not > liable for any damage you may sustain as a result of any virus in this > e-mail. The recipient should check this email and any attachments for the > presence of viruses. THBS reserves the right to monitor and review the > content of all messages sent to or from this e-mail address.******** > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sat Sep 2 08:54:51 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 02 Sep 2017 15:54:51 +0900 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: Message-ID: <1964369.dYEJ2dEnGB@changa> On 2017?09?02? ??? 06:42:43 Frank Muller wrote: > Hi Per, > > Hmmm ... So If i understand it well, we've no control on that after all. > > The data can still be buffered on the kernel side. Right, WE don't have control over it, but the runtime already does, and as far as I can tell, fflush() is called every place it would be meaningful to do so already. https://github.com/erlang/otp/search?utf8=%E2%9C%93&q=fflush This is how things go when working with a MUCH higher level runtime than you seem to be accustimed to. -Craig From zxq9@REDACTED Sat Sep 2 08:59:19 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 02 Sep 2017 15:59:19 +0900 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <1964369.dYEJ2dEnGB@changa> References: <1964369.dYEJ2dEnGB@changa> Message-ID: <4288902.SZVoNJSefp@changa> On 2017?09?02? ??? 15:54:51 zxq9 wrote: > This is how things go when working with a MUCH higher level runtime than you seem to be accustimed to. Tone doesn't transmit well in email. I'd like to emphasize that this is not an indictment or condescending remark. Letting go of the details really does take a lot of getting used to. -Craig From frank.muller.erl@REDACTED Sat Sep 2 09:00:57 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 02 Sep 2017 07:00:57 +0000 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <4288902.SZVoNJSefp@changa> References: <1964369.dYEJ2dEnGB@changa> <4288902.SZVoNJSefp@changa> Message-ID: Thanks Craig. /Frank Le sam. 2 sept. 2017 ? 08:59, zxq9 a ?crit : > On 2017?09?02? ??? 15:54:51 zxq9 wrote: > > This is how things go when working with a MUCH higher level runtime than > you seem to be accustimed to. > > Tone doesn't transmit well in email. > I'd like to emphasize that this is not an indictment or condescending > remark. > Letting go of the details really does take a lot of getting used to. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sat Sep 2 09:18:45 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 02 Sep 2017 16:18:45 +0900 Subject: [erlang-questions] Generally agreed file extension... Message-ID: <1824276.vnx5mV0nYa@changa> We have .erl, .hrl, .beam, .ez and .app file extensions in common use. Is there any generally agreed upon file extension for files that contain data formatted for use by file:consult/2? -Craig From zxq9@REDACTED Sat Sep 2 09:24:15 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 02 Sep 2017 16:24:15 +0900 Subject: [erlang-questions] Generally agreed file extension... In-Reply-To: <1824276.vnx5mV0nYa@changa> References: <1824276.vnx5mV0nYa@changa> Message-ID: <1620612.Is2QSK6Ag9@changa> On 2017?09?02? ??? 16:18:45 zxq9 wrote: > file:consult/2? Er... file:consult/1 -Craig From mjtruog@REDACTED Sat Sep 2 10:07:23 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 2 Sep 2017 01:07:23 -0700 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> Message-ID: <59AA66BB.4020906@gmail.com> On 09/01/2017 04:58 AM, Frank Muller wrote: > Can someone help on this please? > If you use stdout as an Erlang port like: STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), erlang:port_command(STDOUT, "unbuffered"). Then you don't need to flush stdout and you could probably flush stdout by just using erlang:port_command(STDOUT, ""). Doing output this way bypasses the io server and is useful for doing things like progress bars in Erlang source code. Best Regards, Michael From palani@REDACTED Sat Sep 2 11:14:58 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Sat, 2 Sep 2017 14:44:58 +0530 Subject: [erlang-questions] Want to convert tuple format into map Message-ID: Hi Guys, I want to convert tuple format into map. This is format of tuple *{{metadata,2,0, {server_status,false,true,false,false,true,false,false, false,false,false,false,false}, [ { field_metadata,"def","zurik","erlang","erlang","name", "name",33,135,253, <<0,0>>, 0, [] }, { field_metadata,"def","zurik","erlang","erlang","id", "id",63,11,3, <<0,0>>, 0, [] } ], [] }, [ ["some",457], ["some",457], ["data2sosfasfme",42] ] }* I want to convert this format into map. Please help me to solve this challenge -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sat Sep 2 11:55:51 2017 From: per@REDACTED (Per Hedeland) Date: Sat, 2 Sep 2017 11:55:51 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> Message-ID: <7d571fea-72f8-98a7-9e99-1d7316dac45b@hedeland.org> On 2017-09-02 08:42, Frank Muller wrote: > Hi Per, > > Hmmm ... So If i understand it well, we've no control on that after all. > The data can still be buffered on the kernel side. Yes, just like it can when you call fflush() in C. C stdio implements a user-level buffer where the result of your fprintf() etc call goes - fflush() does a write(2)/writev(2) call with the contents of that buffer. That's as far as you get with *any* language executing outside the kernel, and for non-disk I/O it's really all that you need - for disk I/O you may want to consider fsync(2) in rare cases, available in Erlang as file:sync/1, but irrelevant for non-disk I/O. You still haven't said what observed problem you want to solve... --Per From mikpelinux@REDACTED Sat Sep 2 12:01:43 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 2 Sep 2017 12:01:43 +0200 Subject: [erlang-questions] Generally agreed file extension... In-Reply-To: <1824276.vnx5mV0nYa@changa> References: <1824276.vnx5mV0nYa@changa> Message-ID: <22954.33159.707264.568376@gargle.gargle.HOWL> zxq9 writes: > We have .erl, .hrl, .beam, .ez and .app file extensions in common use. > > Is there any generally agreed upon file extension for files that contain data formatted for use by file:consult/2? We use ".eterm" for those. I don't know how common that convention is. From per@REDACTED Sat Sep 2 12:10:22 2017 From: per@REDACTED (Per Hedeland) Date: Sat, 2 Sep 2017 12:10:22 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <1964369.dYEJ2dEnGB@changa> References: <1964369.dYEJ2dEnGB@changa> Message-ID: On 2017-09-02 08:54, zxq9 wrote: > On 2017t0902? ?? 06:42:43 Frank Muller wrote: >> Hi Per, >> >> Hmmm ... So If i understand it well, we've no control on that after all. >> >> The data can still be buffered on the kernel side. > > Right, WE don't have control over it, but the runtime already does, and as far as I can tell, fflush() is called every place it would be meaningful to do so already. Well, as stated, that is probably correct, but as I at least implied, the VM doesn't even *use* stdio for the I/O operations requested by Erlang code. Basically all your "fflush" hits are in test / debug / port-program code that doesn't even run in the VM. --Per From eric.pailleau@REDACTED Sat Sep 2 12:26:07 2017 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 2 Sep 2017 12:26:07 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <7d571fea-72f8-98a7-9e99-1d7316dac45b@hedeland.org> References: <1504068697.147330003@webmail.thbs.com> <7d571fea-72f8-98a7-9e99-1d7316dac45b@hedeland.org> Message-ID: Hi, From Erlang perspective, you can get the message queue length of io server handling your write. io servers are erlang processes consuming an io messages queue. To do so, you need to get the group leader pid of the writing process. _______________________________________________ group_leader() -> pid() Returns the process identifier of the group leader for the process evaluating the function. Every process is a member of some process group and all groups have a group leader. All I/O from the group is channeled to the group leader. When a new process is spawned, it gets the same group leader as the spawning process. Initially, at system startup, init is both its own group leader and the group leader of all processes. _______________________________________________ then you can ask the number of messages still in queue of group leader, using erlang:process_info/2 + message_queue_len note that it is not however a guarantee that all data was flushed if message queue = 0, because last one message maybe currently being treated, and as other mentionned, you don't have control on what underlaying OS is doing low level. >> >> Hmmm ... So If i understand it well, we've no control on that after all. >> The data can still be buffered on the kernel side. From per@REDACTED Sat Sep 2 13:08:36 2017 From: per@REDACTED (Per Hedeland) Date: Sat, 2 Sep 2017 13:08:36 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <59AA66BB.4020906@gmail.com> References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> Message-ID: <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> On 2017-09-02 10:07, Michael Truog wrote: > On 09/01/2017 04:58 AM, Frank Muller wrote: >> Can someone help on this please? >> > If you use stdout as an Erlang port like: > > STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), > erlang:port_command(STDOUT, "unbuffered"). Can you actually observe a difference *in buffering* between doing that and just io:format("unbuffered"). ? > Then you don't need to flush stdout You also don't need to flush after a call to io:format/1 as above. > and you could probably flush stdout by just using erlang:port_command(STDOUT, ""). I can't believe that it would have any effect whatsoever. > Doing output this way bypasses the io server True. But the io server doesn't *buffer*, does it? Obviously there is some amount of message passing and processing that you avoid by going directly to a file descriptor port, but I really don't think you can observe that difference in interactive use. If you are creating an application for non-interactive use that will write lots of data on its "stdout", your method is probably preferable though. > and is useful for doing things like progress bars in Erlang source code. I think the very trivial -eval snippet I posted in another reply (io:format/1 interleaved with timer:sleep/1) clearly shows that you can do progress bars just fine with io:format/1 too. But besides the obvious, such that your progress bar will be printed on the remote node instead of locally when running a remote shell, and that you lose the functionality of io:format/2, I noticed a difference that I hadn't foreseen: If you are at a (e.g.) shell prompt, your method will just print on the current line *after* the prompt, while io:format/1 will result in "printing before" the prompt, i.e. the line is redrawn with the prompt at the end. Both are pretty ugly but in different ways:-) though, and I guess a progress bar is typically printed when you have given a "command" and are waiting for the result - i.e. *not* at a prompt - in which case there is again no observable difference. --Per From scott_ribe@REDACTED Sat Sep 2 13:21:46 2017 From: scott_ribe@REDACTED (scott ribe) Date: Sat, 2 Sep 2017 05:21:46 -0600 Subject: [erlang-questions] Erlang HTTP client libraries- pros/cons In-Reply-To: References: Message-ID: On Sep 1, 2017, at 11:22 PM, Max Lapshin wrote: > > So I do not understand what can give libcurl that cannot be achieved in plain erlang. It is definitely not about high traffic speed because plain vanilla lhttpc can download 10 Gbit/s over fiber without any extra modifications. From fast servers. It bogs down with lots of connections, such that when pointed at a wide selection of real-world servers, it's quite slow. -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From scott_ribe@REDACTED Sat Sep 2 13:24:18 2017 From: scott_ribe@REDACTED (scott ribe) Date: Sat, 2 Sep 2017 05:24:18 -0600 Subject: [erlang-questions] Erlang HTTP client libraries- pros/cons In-Reply-To: References: Message-ID: <4D2CF3A1-E143-451A-9256-0735CF173A50@elevated-dev.com> On Sep 2, 2017, at 5:21 AM, scott ribe wrote: > > On Sep 1, 2017, at 11:22 PM, Max Lapshin wrote: >> >> So I do not understand what can give libcurl that cannot be achieved in plain erlang. It is definitely not about high traffic speed because plain vanilla lhttpc can download 10 Gbit/s over fiber without any extra modifications. > > From fast servers. It bogs down with lots of connections, such that when pointed at a wide selection of real-world servers, it's quite slow. Replying to myself... It's 5AM here, and I was talking about httpc, not lhttpc. -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From mononcqc@REDACTED Sat Sep 2 14:28:28 2017 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 2 Sep 2017 08:28:28 -0400 Subject: [erlang-questions] Generally agreed file extension... In-Reply-To: <1824276.vnx5mV0nYa@changa> References: <1824276.vnx5mV0nYa@changa> Message-ID: <20170902122827.GN51868@ferdmbp.local> On 09/02, zxq9 wrote: >We have .erl, .hrl, .beam, .ez and .app file extensions in common use. > >Is there any generally agreed upon file extension for files that contain data formatted for use by file:consult/2? > I've used filename.consult quite a few times. Otherwise, the .config format for Erlang is close to that. In the most cases though, people appear to use extensions that explain what the file contains and not its format, oddly enough. sys.config uses the format, although it asks for a full list of terms: [{App1, CfgList1, {App2, CfgList2, ... {AppN, CfgListN}]. the .app (and .app.src) files use the consult format as well, and so do various ones like rebar.config, rebar.lock, relx.config, elvis.config, common_test's .spec files, etc. From vinoski@REDACTED Sat Sep 2 15:24:45 2017 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 2 Sep 2017 09:24:45 -0400 Subject: [erlang-questions] NIF Query In-Reply-To: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> References: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> Message-ID: On Fri, Sep 1, 2017 at 7:13 PM, John Duffy wrote: > Hi > > I'm trying to follow the Interoperability Tutorial Users Guide (Version > 9.0) to create a module containing some simple NIFS. I have managed to do > this, however I have needed to digress from the documentation as follows... > > 1.) if my module name doesn't match the filename in the call to > erlang:load_nif("filename", 0) I get a compile error. This is not as per > the example in the documentation, i.e. -module(complex6) & > erlang:load_nif("complex6_nif, 0). > Can you supply the details of the error you're getting? I followed the tutorial on my Mac and had no problems. 2.) The appears to be a "flags" parameter missing in the {"foo", 1, > foo_nif} structure in the documentation which generates a compile error... > > static ErlNifFunc nif_funcs[] = { > > {"foo", 1, foo_nif}, > {"bar", 1, bar_nif} > }; > > > I have changed this to that below, which compiles, but what is the nature/value of "flags"? > > > static ErlNifFunc nif_funcs[] = { > > {"foo", 1, foo_nif, 0}, > {"bar", 1, bar_nif, 0} > }; > > > > I may be missing something, or the Tutorial may need updating? > The flags field is described in http://erlang.org/doc/man/erl_nif.html -- it's currently used for values specific to dirty NIFs, which are also described on that page. For a regular NIF like the one in the tutorial, flags should be 0. Since the nif_funcs variable in the tutorial is defined as static, the C compiler is guaranteed to initialize flags to 0 because an initial value for it isn't specified. Whether you get a problem from the flags field or not depends on your compiler and the options you pass to it. For example, copying and pasting the complex6_nif.c from the tutorial into a file and compiling it as follows on my Mac produced no errors or warnings: clang -o complex6_nif.so complex.c complex6_nif.c -I/usr/local/lib/erlang/usr/include -bundle -flat_namespace -undefined suppress But adding the -Wmissing-field-initializers option results in warnings: complex6_nif.c:27:23: warning: missing field 'flags' initializer [-Wmissing-field-initializers] {"foo", 1, foo_nif}, ^ complex6_nif.c:28:23: warning: missing field 'flags' initializer [-Wmissing-field-initializers] {"bar", 1, bar_nif} ^ 2 warnings generated. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Sat Sep 2 16:34:01 2017 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 2 Sep 2017 14:34:01 +0000 Subject: [erlang-questions] io:columns() call from escripts Message-ID: Hi, We have a number of escripts used as engineering debug tools that make use of io:columns() to properly format output. This was working fine with R17, however we recently upgraded to R19 and now the call fails with {error,enotsup}. Has anything changed between R17 and R19 to break things? I'm investigating the build environment, but it works fine when we run it within the Erlang shell, just not within an escript. Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Sep 2 18:18:19 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 02 Sep 2017 16:18:19 +0000 Subject: [erlang-questions] Want to convert tuple format into map In-Reply-To: References: Message-ID: The tools you need are recursion and pattern matching. Build a set of functions which match on your tuples, uses recursion to compute subtuples into maps, and then builds a map out of the results at this level. Suppose we have T = {x, {data, 34, 35}, {data, 21, 22}}, We can then write a tool conv({x, L, R}) -> #{ type => x, left => conv(L), right => conv(R) }; conv({data, V1, V2}) -> #{ type => data, value1 => V1, value2 => V2 }. which can do the conversion by calling itself recursively. These ideas should be applicable to your case as well. On Sat, Sep 2, 2017 at 11:15 AM Palanikumar Gopalakrishnan < palani@REDACTED> wrote: > > > Hi Guys, > > I want to convert tuple format into map. This is format of > tuple > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *{{metadata,2,0, > {server_status,false,true,false,false,true,false,false, > false,false,false,false,false}, [ > { > field_metadata,"def","zurik","erlang","erlang","name", > "name",33,135,253, <<0,0>>, > 0, [] > }, { > field_metadata,"def","zurik","erlang","erlang","id", > "id",63,11,3, <<0,0>>, > 0, [] > } ], [] > }, [ ["some",457], > ["some",457], ["data2sosfasfme",42] > ] }* > I want to convert this format into map. Please help me to solve this > challenge > > > > -- > > *Warm Regards,* > > *Palanikumar Gopalakrishnan *[image: ?] > *Developer* > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Sep 2 19:15:11 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 2 Sep 2017 10:15:11 -0700 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> Message-ID: <59AAE71F.3050800@gmail.com> On 09/02/2017 04:08 AM, Per Hedeland wrote: > On 2017-09-02 10:07, Michael Truog wrote: >> On 09/01/2017 04:58 AM, Frank Muller wrote: >>> Can someone help on this please? >>> >> If you use stdout as an Erlang port like: >> >> STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), >> erlang:port_command(STDOUT, "unbuffered"). > > Can you actually observe a difference *in buffering* between doing that > and just > > io:format("unbuffered"). Yes, it really does matter. You will notice that the io module doesn't have any flush function, so you are unable to force any buffering in the Erlang VM to flush. However, if you really care about stdout in unbuffered mode, you need to use the Erlang port as described above. That is similar to using fcntl to set file descriptor 1 with O_NONBLOCK. Programming languages normally have a way of setting stdout (and stderr, though stderr is really suppose to always be in unbuffered mode) in an unbuffered mode so all the output is available as quickly as possible. An example for C++ is: setbuf(stdout, NULL); std::cout.setf(std::ios::unitbuf); If you have pipes setup to handle stdout/stderr from a forked OS process that is executing a programming language, you want to have stdout and stderr in unbuffered mode to ensure the output is available without any extra delays (e.g., for logging). Otherwise, it is problematic if the OS process crashes, because you may never receive a stacktrace (if one is provided) or any other output if it is available. That is why each external (running in an OS process, not in the Erlang VM) CloudI API implementation (https://github.com/CloudI/CloudI/tree/develop/src/api) puts stdout and stderr into unbuffered mode. In Erlang source code we can't really put the io server into an unbuffered mode, and there is no reason to really, because it is simpler to bypass the io server by making an Erlang port, as shown above. Best Regards, Michael From frank.muller.erl@REDACTED Sat Sep 2 19:34:35 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 02 Sep 2017 17:34:35 +0000 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <59AAE71F.3050800@gmail.com> References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> <59AAE71F.3050800@gmail.com> Message-ID: Ok, thanks for the clarification. /Frank Le sam. 2 sept. 2017 ? 19:15, Michael Truog a wrote : > On 09/02/2017 04:08 AM, Per Hedeland wrote: > > On 2017-09-02 10:07, Michael Truog wrote: > >> On 09/01/2017 04:58 AM, Frank Muller wrote: > >>> Can someone help on this please? > >>> > >> If you use stdout as an Erlang port like: > >> > >> STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), > >> erlang:port_command(STDOUT, "unbuffered"). > > > > Can you actually observe a difference *in buffering* between doing that > > and just > > > > io:format("unbuffered"). > > Yes, it really does matter. You will notice that the io module doesn't > have any flush function, so you are unable to force any buffering in the > Erlang VM to flush. However, if you really care about stdout in unbuffered > mode, you need to use the Erlang port as described above. That is similar > to using fcntl to set file descriptor 1 with O_NONBLOCK. Programming > languages normally have a way of setting stdout (and stderr, though stderr > is really suppose to always be in unbuffered mode) in an unbuffered mode so > all the output is available as quickly as possible. An example for C++ is: > > setbuf(stdout, NULL); > std::cout.setf(std::ios::unitbuf); > > If you have pipes setup to handle stdout/stderr from a forked OS process > that is executing a programming language, you want to have stdout and > stderr in unbuffered mode to ensure the output is available without any > extra delays (e.g., for logging). Otherwise, it is problematic if the OS > process crashes, because you may never receive a stacktrace (if one is > provided) or any other output if it is available. > > That is why each external (running in an OS process, not in the Erlang VM) > CloudI API implementation ( > https://github.com/CloudI/CloudI/tree/develop/src/api) puts stdout and > stderr into unbuffered mode. In Erlang source code we can't really put the > io server into an unbuffered mode, and there is no reason to really, > because it is simpler to bypass the io server by making an Erlang port, as > shown above. > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sat Sep 2 21:17:19 2017 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 2 Sep 2017 20:17:19 +0100 Subject: [erlang-questions] NIF Query In-Reply-To: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> References: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> Message-ID: On 2 September 2017 at 00:13, John Duffy wrote: > 1.) if my module name doesn't match the filename in the call to > erlang:load_nif("filename", 0) I get a compile error. This is not as per the > example in the documentation, i.e. -module(complex6) & > erlang:load_nif("complex6_nif, 0). I don't see how you'd get a *compile* error. Could you clarify? I *have* noticed that the name of the calling module (i.e. the .erl file) must match the name of the NIF as provided in the first argument to ERL_NIF_INIT. That doesn't appear to be documented anywhere. Also note that Erlang will implicitly add the ".so" extension when you call 'erlang:load_nif'. From jb_duffy@REDACTED Sat Sep 2 21:24:35 2017 From: jb_duffy@REDACTED (John Duffy) Date: Sat, 2 Sep 2017 20:24:35 +0100 (BST) Subject: [erlang-questions] NIF Query In-Reply-To: References: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> Message-ID: <333668.28505.1504380275697.JavaMail.defaultUser@defaultHost> Hi Steve Thanks for you reply. Your help has answered my queries... 1.) I had a typo in ERL_NIF_INIT(), the module name parameter was pointing to a NIF function name and not the module name. 2.) I'm compiling with gcc on Debian. With my original compiler options "-Wall -W" the "-W" was generating the uninitialised structure field warning. Without the "-W" flag the warning is suppressed. Many thanks for your help. Kind regards John ----Original message---- >From : vinoski@REDACTED Date : 02/09/2017 - 14:24 (BST) To : jb_duffy@REDACTED Cc : erlang-questions@REDACTED Subject : Re: [erlang-questions] NIF Query On Fri, Sep 1, 2017 at 7:13 PM, John Duffy wrote: Hi I'm trying to follow the Interoperability Tutorial Users Guide (Version 9.0) to create a module containing some simple NIFS. I have managed to do this, however I have needed to digress from the documentation as follows... 1.) if my module name doesn't match the filename in the call to erlang:load_nif("filename", 0) I get a compile error. This is not as per the example in the documentation, i.e. -module(complex6) & erlang:load_nif("complex6_nif, 0). Can you supply the details of the error you're getting? I followed the tutorial on my Mac and had no problems. 2.) The appears to be a "flags" parameter missing in the {"foo", 1, foo_nif} structure in the documentation which generates a compile error... static ErlNifFunc nif_funcs[] = { {"foo", 1, foo_nif}, {"bar", 1, bar_nif} }; I have changed this to that below, which compiles, but what is the nature/value of "flags"? static ErlNifFunc nif_funcs[] = { {"foo", 1, foo_nif, 0}, {"bar", 1, bar_nif, 0} }; I may be missing something, or the Tutorial may need updating? The flags field is described in http://erlang.org/doc/man/erl_nif.html -- it's currently used for values specific to dirty NIFs, which are also described on that page. For a regular NIF like the one in the tutorial, flags should be 0. Since the nif_funcs variable in the tutorial is defined as static, the C compiler is guaranteed to initialize flags to 0 because an initial value for it isn't specified. Whether you get a problem from the flags field or not depends on your compiler and the options you pass to it. For example, copying and pasting the complex6_nif.c from the tutorial into a file and compiling it as follows on my Mac produced no errors or warnings: clang -o complex6_nif.so complex.c complex6_nif.c -I/usr/local/lib/erlang/usr/include -bundle -flat_namespace -undefined suppress But adding the -Wmissing-field-initializers option results in warnings: complex6_nif.c:27:23: warning: missing field 'flags' initializer [-Wmissing-field-initializers] {"foo", 1, foo_nif}, ^ complex6_nif.c:28:23: warning: missing field 'flags' initializer [-Wmissing-field-initializers] {"bar", 1, bar_nif} ^ 2 warnings generated. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb_duffy@REDACTED Sat Sep 2 21:32:02 2017 From: jb_duffy@REDACTED (John Duffy) Date: Sat, 2 Sep 2017 20:32:02 +0100 (BST) Subject: [erlang-questions] NIF Query In-Reply-To: References: <31545993.15542.1504307586593.JavaMail.defaultUser@defaultHost> Message-ID: <30719305.28972.1504380722165.JavaMail.defaultUser@defaultHost> Roger Thanks for your reply. My issues were caused by not specifying the module name correctly in the first argument to ERL_NIF_INIT() and a gcc compiler option "-W" which was generating the uninitialised structure field warning. Kind regards John ----Original message---- >From : roger@REDACTED Date : 02/09/2017 - 20:17 (BST) To : jb_duffy@REDACTED Cc : erlang-questions@REDACTED Subject : Re: [erlang-questions] NIF Query On 2 September 2017 at 00:13, John Duffy wrote: > 1.) if my module name doesn't match the filename in the call to > erlang:load_nif("filename", 0) I get a compile error. This is not as per the > example in the documentation, i.e. -module(complex6) & > erlang:load_nif("complex6_nif, 0). I don't see how you'd get a *compile* error. Could you clarify? I *have* noticed that the name of the calling module (i.e. the .erl file) must match the name of the NIF as provided in the first argument to ERL_NIF_INIT. That doesn't appear to be documented anywhere. Also note that Erlang will implicitly add the ".so" extension when you call 'erlang:load_nif'. From mjtruog@REDACTED Sun Sep 3 01:35:39 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 2 Sep 2017 16:35:39 -0700 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901121346.GB11300@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901085439.GD87875@erix.ericsson.se> <59A93DEB.6000205@gmail.com> <20170901121346.GB11300@erix.ericsson.se> Message-ID: <59AB404B.4040408@gmail.com> On 09/01/2017 05:13 AM, Raimo Niskanen wrote: > On Fri, Sep 01, 2017 at 04:00:59AM -0700, Michael Truog wrote: >> On 09/01/2017 01:54 AM, Raimo Niskanen wrote: >>> On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: >>> : >>>> I have some examples that can make this desire a bit clearer: >>>> >>>> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 >>>> >>>> % use Box-Muller transformation to generate Gaussian noise >>>> % (G. E. P. Box and Mervin E. Muller, >>>> % A Note on the Generation of Random Normal Deviates, >>>> % The Annals of Mathematical Statistics (1958), >>>> % Vol. 29, No. 2 pp. 610?611) >>>> X1 = random(), >>>> X2 = PI2 * random(), >>>> K = StdDev * math:sqrt(-2.0 * math:log(X1)), >>>> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), >>>> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), >>>> sleep(Result2), >>> Why not use rand:normal/3? >>> >>> It uses the Ziggurat Method and is supposed to be much faster and >>> numerically more stable than the basic Box-Muller method. >>> >> The Box-Muller is simpler and producing 2 results instead of 1 . I believe I looked at the source code for rand:normal/3 and expected the Box-Muller to be faster only because it creates 2 results, though I should check that. I will have to investigate it more. > Simpler - yes. > > The basic benchmark in rand_SUITE indicates that rand:normal() is only > about 50% slower than rand:uniform(1 bsl 58) (internal word size), > which I think is a very good number. > > The Box-Muller transform method needs 4 calls to the 'math' module for > non-trivial floating point functions i.e log(), sqrt(), cos() and sin(), > which is why I think that "must" be slower. > > But I have also not measured... :-/ > > Looking forward to hear your results! I have some interesting results. These results use https://github.com/okeuday/erlbench which includes a copy of the source code at https://github.com/okeuday/quickrand : TEST pseudo_randomness N == 10000 (10 runs) 18_bxor_abs get: 1612.7 us ( 1.3) 18_erlang:system_tim get: 1254.1 us ( 1.0) 18_monotonic get: 1372.5 us ( 1.1) 18_os:system_time/1 get: 1221.7 us ( 1.0) 19_os:perf_counter/1 get: 3752.2 us ( 3.1) 20_rand:normal get: 6832.0 us ( 5.6) 20_rand_exrop get: 3949.3 us ( 3.2) 20_rand_exs1024s get: 12073.3 us ( 9.9) 20_rand_exsp get: 3390.4 us ( 2.8) os:timestamp/0 get: 1392.3 us ( 1.1) os_time:perf_counter get: 4109.4 us ( 3.4) quickrand_c:floatR/0 get: 5776.0 us ( 4.7) quickrand_c:floatR/1 get: 5704.3 us ( 4.7) quickrand_c:uni/1 get: 4015.2 us ( 3.3) quickrand_c:uni/2 get: 3960.7 us ( 3.2) quickrand_c_normal/2 get: 9329.5 us ( 7.6) quickrand_c_normal/3 get: 8917.7 us ( 7.3) random_wh06_int:unif get: 10777.5 us ( 8.8) random_wh82:uniform/ get: 4750.0 us ( 3.9) random_wh82_int:unif get: 4866.4 us ( 4.0) The function names that are relevant for a normal distribution are: 20_rand:normal -> rand:normal/0 (when using rand:seed(exsp, _)) 20_rand_exsp -> rand:uniform/1 (when using rand:seed(exsp, _)) quickrand_c:floatR/0 -> quickrand_cache:floatR/0 quickrand_c:floatR/1 -> quickrand_cache:floatR/1 quickrand_c_normal/2 -> quickrand_cache_normal:box_muller/2 quickrand_c_normal/3 -> quickrand_cache_normal:box_muller/3 The rand module exsp algorithm was used here because it is the fastest pseudo-random number generator in the rand module. A rough look at the latency associated with the normal distribution method, ignoring the latency for random number source is: rand:normal/0 3441.6 us = 6832.0 us - (rand:uniform/1 3390.4 us) quickrand_cache_normal:box_muller/2 3553.5 us = 9329.5 us - (quickrand_cache:floatR/0 5776.0 us) quickrand_cache_normal:box_muller/3 3213.4 us = 8917.7 us - (quickrand_cache:floatR/1 5704.3us) So, this helps to show that the latency with both methods is very similar if you ignore the random number generation. However, it likely requires some explanation: The quickrand_cache module is what I am using here for random number generation, which stores cached data from crypto:strong_rand_bytes/1 with a default size of 64KB for the cache. The difference between the functions quickrand_cache_normal:box_muller/2 and quickrand_cache_normal:box_muller/3 is that the first uses the process dictionary while the second uses a state variable. Using the large amount of cached random data, the latency associated with individual calls to crypto:strong_rand_bytes/1 is avoided at the cost of the extra memory consumption, and the use of the cache makes the speed of random number generation similar to the speed of pseudo-random number generation that occurs in the rand module. In CloudI, I instead use quickrand_normal:box_muller/2 to avoid the use of cached data to keep the memory use minimal (the use-case there doesn't require avoiding the latency associated with crypto:strong_rand_bytes/1 because it is adding latency for testing (at https://github.com/CloudI/cloudi_core/blob/299df02e6d22103415c8ba14379e90ca8c3d3b82/src/cloudi_core_i_runtime_testing.erl#L138) and it is best using a cryptographic random source to keep the functionality widely applicable). However, the same function calls occur in the quickrand Box-Muller transformation source code, so the overhead is the same. I used Erlang/OTP 20.0 (without HiPE) using the hardware below: |Core i7 2670QM 2.2GHz 1 cpu, 4 cores/cpu, 2 hts/core L2:4?256KB L3:6MB RAM:8GB:DDR3-1333MHz Sandy Bridge-HE-4 (Socket G2) Best Regards, Michael | -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Sep 3 06:18:43 2017 From: zxq9@REDACTED (zxq9) Date: Sun, 03 Sep 2017 13:18:43 +0900 Subject: [erlang-questions] Generally agreed file extension... In-Reply-To: <20170902122827.GN51868@ferdmbp.local> References: <1824276.vnx5mV0nYa@changa> <20170902122827.GN51868@ferdmbp.local> Message-ID: <3176911.knEZz2XZfN@changa> On 2017?09?02? ??? 08:28:28 you wrote: > On 09/02, zxq9 wrote: > >We have .erl, .hrl, .beam, .ez and .app file extensions in common use. > > > >Is there any generally agreed upon file extension for files that contain data formatted for use by file:consult/2? > > > > I've used filename.consult quite a few times. Otherwise, the .config > format for Erlang is close to that. In the most cases though, people > appear to use extensions that explain what the file contains and not its > format, oddly enough. > > sys.config uses the format, although it asks for a full list of terms: > > [{App1, CfgList1, > {App2, CfgList2, > ... > {AppN, CfgListN}]. > > the .app (and .app.src) files use the consult format as well, and so do > various ones like rebar.config, rebar.lock, relx.config, elvis.config, > common_test's .spec files, etc. In the past I've also often used the extension to name what type of thing it is instead of the format. Was wondering if there was a standardish way but its not a big deal. Some interesting suggestions were Mikael Pettersson's suggestion of using .eterm as universal extension and in #erlang illo suggested following the convention of .erl and .hrl files would yield .trl Any preference among folks on the ML? (Other than the useful tradition of using the extension as a label of what the file is instead of the format) -Craig From per@REDACTED Sun Sep 3 06:22:13 2017 From: per@REDACTED (Per Hedeland) Date: Sun, 3 Sep 2017 06:22:13 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <59AAE71F.3050800@gmail.com> References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> <59AAE71F.3050800@gmail.com> Message-ID: On 2017-09-02 19:15, Michael Truog wrote: > On 09/02/2017 04:08 AM, Per Hedeland wrote: >> On 2017-09-02 10:07, Michael Truog wrote: >>> On 09/01/2017 04:58 AM, Frank Muller wrote: >>>> Can someone help on this please? >>>> >>> If you use stdout as an Erlang port like: >>> >>> STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), >>> erlang:port_command(STDOUT, "unbuffered"). >> >> Can you actually observe a difference *in buffering* between doing that >> and just >> >> io:format("unbuffered"). > > Yes, it really does matter. Unfortunately you didn't answer my question. > You will notice that the io module doesn't have any flush function, Yes, of course it doesn't have any flush function, since it doesn't do any buffering. > so you are unable to force any buffering in the Erlang VM to flush. This gets us into the realm of philosophy - is it possible to be unable to flush buffering that doesn't exist? > However, if you really care about > stdout in unbuffered mode, you need to use the Erlang port as > described above. Please provide *some* justification for that claim. At least a way to observe that buffering is actually happening - ideally a pointer to the specific piece of code that implements it. I don't care much what you do in your code, but I strongly oppose that you lead "newbies" to believe that they have to resort to ugly low-level stuff like the above in order to achieve functionality that is actually the default. > That is similar to using fcntl to set file descriptor 1 with O_NONBLOCK. No, neither that code nor making C stdout unbuffered is even remotely similar to making file descriptor 1 non-blocking. > Programming languages normally have a way of > setting stdout (and stderr, though stderr is really suppose to always be in unbuffered mode) in an unbuffered mode so all the output is available as quickly as possible. Yes, programming languages that have C stdio or a corresponding buffering mechanism normally have a way of turning it off. Erlang doesn't have such a mechanism (for non-disk I/O). [snip irrelevant elaboration on C/C++ programming] > In Erlang source code we can't really put the io server into an unbuffered mode, and there is no reason to really, because it is simpler to bypass the io server by making an Erlang port, as > shown above. No - there is no reason to do it, because it isn't needed. Btw, this doesn't mean that Erlang "standard_io" is somehow "higher level" than C stdio, it's rather the opposite - but it's simpler to use, since you don't need to care about tweaking the buffering with setvbuf() and/or fflush() like you do in C. And finally, yet another demonstration of the lack of buffering in Erlang "standard_io" - a progress bar!:-) Eshell V8.3.5.1 (abort with ^G) 1> F = fun (0,_) -> ok; (N,G) -> io:format("*"),timer:sleep(1000),G(N-1,G) end. #Fun 2> F(20,F). --Per From mjtruog@REDACTED Sun Sep 3 08:30:35 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 2 Sep 2017 23:30:35 -0700 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> <59AAE71F.3050800@gmail.com> Message-ID: <59ABA18B.3000704@gmail.com> On 09/02/2017 09:22 PM, Per Hedeland wrote: > On 2017-09-02 19:15, Michael Truog wrote: >> On 09/02/2017 04:08 AM, Per Hedeland wrote: >>> On 2017-09-02 10:07, Michael Truog wrote: >>>> On 09/01/2017 04:58 AM, Frank Muller wrote: >>>>> Can someone help on this please? >>>>> >>>> If you use stdout as an Erlang port like: >>>> >>>> STDOUT = erlang:open_port({fd, 0, 1}, [out, {line, 256}]), >>>> erlang:port_command(STDOUT, "unbuffered"). >>> >>> Can you actually observe a difference *in buffering* between doing that >>> and just >>> >>> io:format("unbuffered"). >> >> Yes, it really does matter. > > Unfortunately you didn't answer my question. > >> You will notice that the io module doesn't have any flush function, > > Yes, of course it doesn't have any flush function, since it doesn't do > any buffering. When you call a function in the io module, what happens? An io:format/1 function call sends the data in an Erlang message and that Erlang message goes into the group leader's Erlang process message queue, to get received when the Erlang process is ready for it. That use of the Erlang process message queue is a form of buffering, ignoring everything that happens after that in the group leader Erlang process. > >> so you are unable to force any buffering in the Erlang VM to flush. > > This gets us into the realm of philosophy - is it possible to be unable > to flush buffering that doesn't exist? Refer to the explanation above. > >> However, if you really care about stdout in unbuffered mode, you need to use the Erlang port as >> described above. > > Please provide *some* justification for that claim. At least a way to > observe that buffering is actually happening - ideally a pointer to the > specific piece of code that implements it. > > I don't care much what you do in your code, but I strongly oppose that > you lead "newbies" to believe that they have to resort to ugly low-level > stuff like the above in order to achieve functionality that is actually > the default. To allow output to be unbuffered, you need to bypass buffers. If the approach isn't simple enough, that means there is room for improvement in the Erlang programming language. > >> That is similar to using fcntl to set file descriptor 1 with O_NONBLOCK. > > No, neither that code nor making C stdout unbuffered is even remotely > similar to making file descriptor 1 non-blocking. > >> Programming languages normally have a way of setting stdout (and stderr, though stderr is really suppose to always be in unbuffered mode) in an unbuffered mode so all the output is available as quickly as possible. > > Yes, programming languages that have C stdio or a corresponding > buffering mechanism normally have a way of turning it off. Erlang > doesn't have such a mechanism (for non-disk I/O). > > [snip irrelevant elaboration on C/C++ programming] > >> In Erlang source code we can't really put the io server into an unbuffered mode, and there is no reason to really, because it is simpler to bypass the io server by making an Erlang port, as shown above. > > No - there is no reason to do it, because it isn't needed. > > Btw, this doesn't mean that Erlang "standard_io" is somehow "higher > level" than C stdio, it's rather the opposite - but it's simpler to use, > since you don't need to care about tweaking the buffering with setvbuf() > and/or fflush() like you do in C. > > And finally, yet another demonstration of the lack of buffering in > Erlang "standard_io" - a progress bar!:-) > > Eshell V8.3.5.1 (abort with ^G) > 1> F = fun (0,_) -> ok; (N,G) -> io:format("*"),timer:sleep(1000),G(N-1,G) end. > #Fun > 2> F(20,F). > To have a fast responsive progress bar, you need unbuffered output that won't get stuck on other use of io, allowing unimpeded use of the \b character that goes directly to the terminal without special terminal settings. If you don't understand this, I have an example at https://github.com/okeuday/reltool_util/blob/b80cf8ecc423a108402529272ffe00f018bcf5ca/scope#L873-L944 . Best Regards, Michael From jesper.louis.andersen@REDACTED Sun Sep 3 12:33:45 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 03 Sep 2017 10:33:45 +0000 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <59ABA18B.3000704@gmail.com> References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> <59AAE71F.3050800@gmail.com> <59ABA18B.3000704@gmail.com> Message-ID: On Sun, Sep 3, 2017 at 8:30 AM Michael Truog wrote: > > When you call a function in the io module, what happens? An io:format/1 > function call sends the data in an Erlang message and that Erlang message > goes into the group leader's Erlang process message queue, to get received > when the Erlang process is ready for it. That use of the Erlang process > message queue is a form of buffering, ignoring everything that happens > after that in the group leader Erlang process. > > It is definitely asynchronous communication with latency. I don't think it is buffering, however, albeit that is a bit philosophical. As soon as the message arrives, it is sent on. In no place do you delay the write in the hope that more data arrives shortly. If anything it acts like a queue, not a buffer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sun Sep 3 14:51:57 2017 From: per@REDACTED (Per Hedeland) Date: Sun, 3 Sep 2017 14:51:57 +0200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: References: <1504068697.147330003@webmail.thbs.com> <59AA66BB.4020906@gmail.com> <8703adc8-e5f8-b735-878f-eece1c06a925@hedeland.org> <59AAE71F.3050800@gmail.com> <59ABA18B.3000704@gmail.com> Message-ID: On 2017-09-03 12:33, Jesper Louis Andersen wrote: > On Sun, Sep 3, 2017 at 8:30 AM Michael Truog > wrote: > > > When you call a function in the io module, what happens? An io:format/1 function call sends the data in an Erlang message and that Erlang message goes into the group leader's Erlang process > message queue, to get received when the Erlang process is ready for it. That use of the Erlang process message queue is a form of buffering, ignoring everything that happens after that in the > group leader Erlang process. > > > It is definitely asynchronous communication with latency. > > I don't think it is buffering, however, albeit that is a bit philosophical. As soon as the message arrives, it is sent on. In no place do you delay the write in the hope that more data arrives > shortly. If anything it acts like a queue, not a buffer. Exactly. So I guess we just have a bit of miscommunication - personally I haven't previously encountered the use of the term "buffering" for "asynchronous communication with latency", but I don't know if there is an formal definition somewhere. However the thread started with the question of how to do the equivalent of fflush() or "set to unbuffered" for C stdio, which deals with the stdio buffering that *is* precisely "delay the write in the hope that more data arrives shortly" - and will delay the write *forever* in some cases unless you know what you're doing. And this does not exist in Erlang "standard_io". But for the question of how to do output with "maximal synchronicity" and minimal latency, opening a file descriptor port and using erlang:port_command/2,3 is probably a fine answer. --Per From ok@REDACTED Mon Sep 4 02:20:51 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 4 Sep 2017 12:20:51 +1200 Subject: [erlang-questions] Flush "stdout" buffer In-Reply-To: <1964369.dYEJ2dEnGB@changa> References: <1964369.dYEJ2dEnGB@changa> Message-ID: <17d2a687-4ce3-c9c9-72a8-3d56ab4a9184@cs.otago.ac.nz> On 2/09/17 6:54 PM, zxq9 wrote: > On 2017?09?02? ??? 06:42:43 Frank Muller wrote: >> Hi Per, >> >> Hmmm ... So If i understand it well, we've no control on that after all. >> >> The data can still be buffered on the kernel side. > > Right, WE don't have control over it, but the runtime already does, and as far as I can tell, fflush() is called every place it would be meaningful to do so already. For kernel buffering there is fsync(fileno(...)). Of course this does nothing about *device* buffering... From ok@REDACTED Mon Sep 4 02:37:50 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 4 Sep 2017 12:37:50 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901084958.GC87875@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> Message-ID: <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> On 1/09/17 8:49 PM, Raimo Niskanen wrote: >> By the way, given that a common way to make random floats is to >> generate a bitvector, consider >> (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. >> You will notice that the spacing between the values is *almost* >> uniform, but not at the end. > > That sounds interesting but I do not understand. Is that Elixir code? Nope, Smalltalk. I wanted to use rational arithmetic. In fact I did not need to. Here it is in Haskell: > [(x * 256) `div` 15 | x <- [0..15]] [0,17,34,51,68,85,102,119,136,153,170,187,204,221,238,256] Let's push that a bit further. Let's generate all possible 10-bit integers and map them to the range [0..63]. We find again that the gap sizes are not all the same. They can't be. If you consider all vectors of N bits and map them to the range [0..2**M] they *cannot* be uniformly distributed no matter what method you use because (2**M+1) does not divide 2**N. You can fix this by rejecting some of the bit vectors, but that would be asking everyone to pay extra for a result they don't have any particular need for. From ok@REDACTED Mon Sep 4 03:42:19 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 4 Sep 2017 13:42:19 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170901125338.GC11300@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901084115.GB87875@erix.ericsson.se> <20170901093637.GF30990@carfax.org.uk> <20170901125338.GC11300@erix.ericsson.se> Message-ID: <552fd1e1-0c9e-a626-f169-2dfaa91d1f06@cs.otago.ac.nz> On 2/09/17 12:53 AM, Raimo Niskanen wrote: >> P = random(), >> if >> P =< 0.3 -> ...; >> P =< 0.7 -> ...; >> P > 0.7 -> ... >> end > Especially since decimal numbers below 1.0 have no exact representation as > IEEE floating point numbers. There is in fact an IEEE standard for *decimal* floating point numbers. If memory serves me correctly, z/Series and POWER support it. There is a supplement to the C standard for it. Software emulation is available. See decimal32, decimal64, decimal128 in https://en.wikipedia.org/wiki/IEEE_754 or the actual IEEE 754-2008 if you have a copy (which I sadly don't). In any case, P = 10*rand(), if P < 3 -> ... ; P < 7 -> ... ; P >= 7 -> ... end evades that issue. (With the numbers from drand48() the multiplication is exact; with 53-bit random numbers it is not.) From raimo+erlang-questions@REDACTED Mon Sep 4 10:48:15 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 4 Sep 2017 10:48:15 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <59AB404B.4040408@gmail.com> References: <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901085439.GD87875@erix.ericsson.se> <59A93DEB.6000205@gmail.com> <20170901121346.GB11300@erix.ericsson.se> <59AB404B.4040408@gmail.com> Message-ID: <20170904084815.GA43354@erix.ericsson.se> On Sat, Sep 02, 2017 at 04:35:39PM -0700, Michael Truog wrote: > On 09/01/2017 05:13 AM, Raimo Niskanen wrote: > > On Fri, Sep 01, 2017 at 04:00:59AM -0700, Michael Truog wrote: > >> On 09/01/2017 01:54 AM, Raimo Niskanen wrote: > >>> On Thu, Aug 31, 2017 at 10:29:34PM -0700, Michael Truog wrote: > >>> : > >>>> I have some examples that can make this desire a bit clearer: > >>>> > >>>> https://github.com/CloudI/cloudi_core/blob/a1c10a02245f0f4284d701a2ee5f07aad17f6e51/src/cloudi_core_i_runtime_testing.erl#L139-L149 > >>>> > >>>> % use Box-Muller transformation to generate Gaussian noise > >>>> % (G. E. P. Box and Mervin E. Muller, > >>>> % A Note on the Generation of Random Normal Deviates, > >>>> % The Annals of Mathematical Statistics (1958), > >>>> % Vol. 29, No. 2 pp. 610?611) > >>>> X1 = random(), > >>>> X2 = PI2 * random(), > >>>> K = StdDev * math:sqrt(-2.0 * math:log(X1)), > >>>> Result1 = erlang:max(erlang:round(Mean + K * math:cos(X2)), 1), > >>>> Result2 = erlang:max(erlang:round(Mean + K * math:sin(X2)), 1), > >>>> sleep(Result2), > >>> Why not use rand:normal/3? > >>> > >>> It uses the Ziggurat Method and is supposed to be much faster and > >>> numerically more stable than the basic Box-Muller method. > >>> > >> The Box-Muller is simpler and producing 2 results instead of 1 . I believe I looked at the source code for rand:normal/3 and expected the Box-Muller to be faster only because it creates 2 results, though I should check that. I will have to investigate it more. > > Simpler - yes. > > > > The basic benchmark in rand_SUITE indicates that rand:normal() is only > > about 50% slower than rand:uniform(1 bsl 58) (internal word size), > > which I think is a very good number. > > > > The Box-Muller transform method needs 4 calls to the 'math' module for > > non-trivial floating point functions i.e log(), sqrt(), cos() and sin(), > > which is why I think that "must" be slower. > > > > But I have also not measured... :-/ > > > > Looking forward to hear your results! > I have some interesting results. > > These results use https://github.com/okeuday/erlbench which includes a copy of the source code at https://github.com/okeuday/quickrand : > > TEST pseudo_randomness > N == 10000 (10 runs) > 18_bxor_abs get: 1612.7 us ( 1.3) > 18_erlang:system_tim get: 1254.1 us ( 1.0) > 18_monotonic get: 1372.5 us ( 1.1) > 18_os:system_time/1 get: 1221.7 us ( 1.0) > 19_os:perf_counter/1 get: 3752.2 us ( 3.1) > 20_rand:normal get: 6832.0 us ( 5.6) > 20_rand_exrop get: 3949.3 us ( 3.2) > 20_rand_exs1024s get: 12073.3 us ( 9.9) > 20_rand_exsp get: 3390.4 us ( 2.8) > os:timestamp/0 get: 1392.3 us ( 1.1) > os_time:perf_counter get: 4109.4 us ( 3.4) > quickrand_c:floatR/0 get: 5776.0 us ( 4.7) > quickrand_c:floatR/1 get: 5704.3 us ( 4.7) > quickrand_c:uni/1 get: 4015.2 us ( 3.3) > quickrand_c:uni/2 get: 3960.7 us ( 3.2) > quickrand_c_normal/2 get: 9329.5 us ( 7.6) > quickrand_c_normal/3 get: 8917.7 us ( 7.3) > random_wh06_int:unif get: 10777.5 us ( 8.8) > random_wh82:uniform/ get: 4750.0 us ( 3.9) > random_wh82_int:unif get: 4866.4 us ( 4.0) > > The function names that are relevant for a normal distribution are: > 20_rand:normal -> rand:normal/0 (when using rand:seed(exsp, _)) > 20_rand_exsp -> rand:uniform/1 (when using rand:seed(exsp, _)) > quickrand_c:floatR/0 -> quickrand_cache:floatR/0 > quickrand_c:floatR/1 -> quickrand_cache:floatR/1 > quickrand_c_normal/2 -> quickrand_cache_normal:box_muller/2 > quickrand_c_normal/3 -> quickrand_cache_normal:box_muller/3 > > The rand module exsp algorithm was used here because it is the fastest pseudo-random number generator in the rand module. Thank you for sharing these numbers! > > A rough look at the latency associated with the normal distribution method, ignoring the latency for random number source is: > rand:normal/0 > 3441.6 us = 6832.0 us - (rand:uniform/1 3390.4 us) Should not the base value come from rand:uniform/0 instead. I know the difference is not big - rand_SUITE:measure/1 suggests 3%, but it also suggests that rand:normal/0 is about 50% slower than rand:uniform/0 while your numbers suggests 100% slower. Slightly strange... > quickrand_cache_normal:box_muller/2 > 3553.5 us = 9329.5 us - (quickrand_cache:floatR/0 5776.0 us) > quickrand_cache_normal:box_muller/3 > 3213.4 us = 8917.7 us - (quickrand_cache:floatR/1 5704.3us) It is really interesting to see that the calls to the 'math' module does not slow that algorithm down very much (hardly noticable)! > > So, this helps to show that the latency with both methods is very similar if you ignore the random number generation. However, it likely requires some explanation: The quickrand_cache module is what I am using here for random number generation, which stores cached data from crypto:strong_rand_bytes/1 with a default size of 64KB for the cache. The difference between the functions quickrand_cache_normal:box_muller/2 and quickrand_cache_normal:box_muller/3 is that the first uses the process dictionary while the second uses a state variable. Using the large amount of cached random data, the latency associated with individual calls to crypto:strong_rand_bytes/1 is avoided at the cost of the extra memory consumption, and the use of the cache makes the speed of random number generation similar to the speed of pseudo-random number generation that occurs in the rand module. We should add a 'rand' plugin to the 'crypto' module that does this buffered crypto:strong_random_bytes/1 trick. There is something like that in rand_SUITE, but we should really have an official one. I also wonder where the sweet spot is? 64 KB seems like a lot of buffer. > > In CloudI, I instead use quickrand_normal:box_muller/2 to avoid the use of cached data to keep the memory use minimal (the use-case there doesn't require avoiding the latency associated with crypto:strong_rand_bytes/1 because it is adding latency for testing (at https://github.com/CloudI/cloudi_core/blob/299df02e6d22103415c8ba14379e90ca8c3d3b82/src/cloudi_core_i_runtime_testing.erl#L138) and it is best using a cryptographic random source to keep the functionality widely applicable). However, the same function calls occur in the quickrand Box-Muller transformation source code, so the overhead is the same. > > I used Erlang/OTP 20.0 (without HiPE) using the hardware below: > |Core i7 2670QM 2.2GHz 1 cpu, 4 cores/cpu, 2 hts/core > L2:4?256KB L3:6MB RAM:8GB:DDR3-1333MHz > Sandy Bridge-HE-4 (Socket G2) > > Best Regards, > Michael > | Best regards -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Mon Sep 4 11:49:05 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 4 Sep 2017 11:49:05 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> Message-ID: <20170904094905.GB43354@erix.ericsson.se> On Mon, Sep 04, 2017 at 12:37:50PM +1200, Richard A. O'Keefe wrote: > > > On 1/09/17 8:49 PM, Raimo Niskanen wrote: > >> By the way, given that a common way to make random floats is to > >> generate a bitvector, consider > >> (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. > >> You will notice that the spacing between the values is *almost* > >> uniform, but not at the end. > > > > That sounds interesting but I do not understand. Is that Elixir code? > > Nope, Smalltalk. I wanted to use rational arithmetic. In fact I did > not need to. Here it is in Haskell: > > [(x * 256) `div` 15 | x <- [0..15]] > [0,17,34,51,68,85,102,119,136,153,170,187,204,221,238,256] > > Let's push that a bit further. Let's generate all possible 10-bit > integers and map them to the range [0..63]. We find again that > the gap sizes are not all the same. They can't be. If you > consider all vectors of N bits and map them to the range > [0..2**M] they *cannot* be uniformly distributed no matter what > method you use because (2**M+1) does not divide 2**N. You can > fix this by rejecting some of the bit vectors, but that would > be asking everyone to pay extra for a result they don't have any > particular need for. > I see, but do not quite understand what you are getting at. The current left-closed float generator starts with 58 random bits and puts 53 of these into the mantissa in an IEEE 754 double binary float, so that would not be it. I guess it is a generator for the closed interval [0.0,1.0] or the open (0.0,1.0) you talk about. If so: This one-liner generates over [0.0,1.0]: (rand:uniform((1 bsl 53) + 1) -1) * math:pow(2, -53) and it uses an integer range R = (2^53 + 1), which is not dividable by 2. The implementation for that range will generate a 58 bit number and then check if the number is 0 =< X < R*31 and if so return the number mod R (31 repetitions of the range is all that fits completely in 58 bits). If the generated number is R*31 =< X that is in the top truncated interval then we start over with a new number. The implementation may in theory retry forever before finding a good number, but the odds for retry is about 1/32 for each round so the accumulated time is about 32/31 times one round. And only once in a million it will take 4 attempts or more. I discussed a different implementation with Prof. Vigna that is to always generate one word too much and then use mod R on that which would bury the skew in one word of random bits hence the difference in probability between generated numbers should be about (2^58 - 1)/2^58, which would take quite some effort to measure with statistical significance. But he considered that as a bad idea since it would get half the speed for most ranges. So this is an already solved problem, as I see it. We *can* write efficient and good generators for open, closed and half-closed intervals, if we want. So far I have only seen the need for a (0.0,1.0] generator, which can be implemented with: 1.0 - rand:uniform() but in some applications such as 1.0/X and math:log(X) I think that the form N * 2^-53 might be less than optimal, so I have a new suggestion: https://github.com/erlang/otp/compare/OTP-20.0...RaimoNiskanen:raimo/stdlib/rand-uniformNZ This variant never returns exactly 0.0 and have better precision for low values. Comments? Especially about the name. And so far I have not seen any actual need for (0.0,1.0) nor [0.0,1.0]. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From acautin@REDACTED Mon Sep 4 13:02:43 2017 From: acautin@REDACTED (Agustin Cautin) Date: Mon, 4 Sep 2017 13:02:43 +0200 Subject: [erlang-questions] Rebar3 release windows service + vm.args with proto_dist Message-ID: Hello, We are in the process of migrating the generation of a windows release from rebar with reltool to rebar3 with relx, it is working more or less fine up to the point of installing the service where some extra configuration from vm.args is ignored. Other than the name and the cookie it seems that the rest of the file is ignored, what would be the proper way of defining the following properties so it can be properly used by the service: ## distributed erlang TCP port range -kernel inet_dist_listen_min 7000 -kernel inet_dist_listen_max 7020 ## Distribution protocol -proto_dist imem_inet_tcp ## Enable kernel poll and a few async threads +K true I believe the kernel ports can be changed to sys.config but not sure about proto_dist. Regards. Agustin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Mon Sep 4 22:59:46 2017 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Mon, 4 Sep 2017 13:59:46 -0700 Subject: [erlang-questions] How to convert sql return data to json In-Reply-To: References: Message-ID: <36db48a8-154f-ee90-5e86-a9b62c98f7cf@gmail.com> On 09/01/2017 05:36 AM, Palanikumar Gopalakrishnan wrote: > *for checking Sql result {{metadata,2,0, > {server_status,false,true,false,false,true, > false,false,false,false,false,false,false}, > [{field_metadata,"def","zurik","erlang", > "erlang","name","name",33,135,253, > <<0,0>>,0,[]}, > {field_metadata,"def","zurik","erlang", > "erlang","id","id",63,11,3, > <<0,0>>,0,[]}],[]}, > [["some",457],["some",457],["data2sosfasfme",42]]}* So, it *looks* like your driver is returning a tuple containing: * A 'metadata' record which contains (among other things) a server_status record and a list of field_metadata records. * A list containing the data that is the response to your query. The "Getting Started" section of the erlmysql documentation [0] seems to say that the record definitions that you need are contained in "client_records.hrl". Loading that file into your Erlang shell with the rr/1 shell command and re-running the query should make the SQL driver's return value make more sense to you. In order to load those record definitions into your client code, you'll probably need to do something like -include_lib("erlmysql/include/client_records.hrl"). near the top of the file that needs to interact with the SQL driver. [0] http://erlmysql.sourceforge.net/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From lloyd@REDACTED Mon Sep 4 23:49:31 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 4 Sep 2017 17:49:31 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?Problem_installing_esl-erlang=5F19?= =?utf-8?q?=2E3=2E6-1=7Eubuntu=7Etrusty-amd64?= Message-ID: <1504561771.40113009@apps.rackspace.com> Hello, I've done this successfully on various machines half-a-dozen times. But, now, setting up a new development machine, installation of Erlang from the Erlang Solutions download page on my Ubuntu 16.04 system fails. From the ES download page I click on Ubuntu Xenial 64-bit. I select software install. An installation page pops up. But when I click on Install, nothing happens. What am I forgetting or getting wrong? Many thanks, LRP From lloyd@REDACTED Tue Sep 5 01:03:22 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 4 Sep 2017 19:03:22 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?Problem_installing_esl-erlang=5F19?= =?utf-8?q?=2E3=2E6-1=7Eubuntu=7Etrusty-amd64?= In-Reply-To: <1504561771.40113009@apps.rackspace.com> References: <1504561771.40113009@apps.rackspace.com> Message-ID: <1504566202.965815168@apps.rackspace.com> Ah, seems that some dependencies were missing. Installation worked when I executed: sudo apt-get -f install I can't remember running into this before. Best wishes, LRP -----Original Message----- From: lloyd@REDACTED Sent: Monday, September 4, 2017 5:49pm To: "Erlang" Subject: [erlang-questions] Problem installing esl-erlang_19.3.6-1~ubuntu~trusty-amd64 Hello, I've done this successfully on various machines half-a-dozen times. But, now, setting up a new development machine, installation of Erlang from the Erlang Solutions download page on my Ubuntu 16.04 system fails. From the ES download page I click on Ubuntu Xenial 64-bit. I select software install. An installation page pops up. But when I click on Install, nothing happens. What am I forgetting or getting wrong? Many thanks, LRP _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From palani@REDACTED Tue Sep 5 13:32:16 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Tue, 5 Sep 2017 17:02:16 +0530 Subject: [erlang-questions] How to convert sql return data to json In-Reply-To: <36db48a8-154f-ee90-5e86-a9b62c98f7cf@gmail.com> References: <36db48a8-154f-ee90-5e86-a9b62c98f7cf@gmail.com> Message-ID: Hi Kenneth, Thanks for your reply, I already include the* -include_lib("erlmysql/**include/client_records.hrl"). *in my source On 5 September 2017 at 02:29, Kenneth Lakin wrote: > On 09/01/2017 05:36 AM, Palanikumar Gopalakrishnan wrote: > > *for checking Sql result {{metadata,2,0, > > {server_status,false,true,false,false,true, > > false,false,false,false,false,false,false}, > > [{field_metadata,"def","zurik","erlang", > > "erlang","name","name",33,135,253, > > <<0,0>>,0,[]}, > > {field_metadata,"def","zurik","erlang", > > "erlang","id","id",63,11,3, > > <<0,0>>,0,[]}],[]}, > > [["some",457],["some",457],["data2sosfasfme",42]]}* > > So, it *looks* like your driver is returning a tuple containing: > * A 'metadata' record which contains (among other things) a > server_status record and a list of field_metadata records. > * A list containing the data that is the response to your query. > > The "Getting Started" section of the erlmysql documentation [0] seems to > say that the record definitions that you need are contained in > "client_records.hrl". Loading that file into your Erlang shell with the > rr/1 shell command and re-running the query should make the SQL driver's > return value make more sense to you. > > In order to load those record definitions into your client code, you'll > probably need to do something like > > -include_lib("erlmysql/include/client_records.hrl"). > > near the top of the file that needs to interact with the SQL driver. > > [0] http://erlmysql.sourceforge.net/ > > -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Tue Sep 5 18:21:37 2017 From: davidnwelton@REDACTED (David Welton) Date: Tue, 5 Sep 2017 09:21:37 -0700 Subject: [erlang-questions] API design and pgapp Message-ID: Hi, I'm jumping back into the world of Erlang, and among other things, work on pgapp, a higher level interface for Postgres. This came up, lately: https://github.com/epgsql/pgapp/issues/22 Which seems to pit two things against one another: * On one hand, using the process registry to keep track of the Conn (connection) is not very Erlang-y and might introduce errors. * That said, it's also nice for things to be composable in the sense that you can write, say, update_stats() either within the transaction or outside and have it 'just work' without having to provide both an update_stats() and update_stats(Conn). Thoughts? Thanks! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From bchesneau@REDACTED Tue Sep 5 18:58:43 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 5 Sep 2017 18:58:43 +0200 Subject: [erlang-questions] nif resource lifetime Message-ID: How do you manage the lifetime of a nif resource when it has been created and can be used by one another? 1) I an object O1 and expose it as a resource R1 2) I?m created R2 that need the object O1exposed by R1 by passing R1 as an argument to the nif function used to create R2 I would like to make sure that the object is not preemptively killed or garbaged collected. How would you do it? Any feedback is welcome :) - beno?t From nathaniel@REDACTED Tue Sep 5 19:03:16 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Tue, 5 Sep 2017 13:03:16 -0400 Subject: [erlang-questions] API design and pgapp In-Reply-To: References: Message-ID: > This came up, lately: https://github.com/epgsql/pgapp/issues/22 > > Which seems to pit two things against one another: > > * On one hand, using the process registry to keep track of the Conn > (connection) is not very Erlang-y and might introduce errors. > > * That said, it's also nice for things to be composable in the sense > that you can write, say, update_stats() either within the transaction > or outside and have it 'just work' without having to provide both an > update_stats() and update_stats(Conn). I don't think it's a productive addition to that epgsql conversation, but I would write only update_stats/1. A Postgres connection is not a trivial thing like an Erlang process. On a default installation you only get a couple hundred of them and you can see some unpleasant surprises if you go over. Therefore, I think it's a mistake to not explicitly track your connections in the code. From daniel.goertzen@REDACTED Tue Sep 5 19:50:21 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 05 Sep 2017 17:50:21 +0000 Subject: [erlang-questions] nif resource lifetime In-Reply-To: References: Message-ID: When forming R2 you would call enif_keep_resource(&O1), and the R2 destructor would enif_release_resource(&O1). This assures O1 sticks around while R2 is alive even if R1 dies. I'm assuming R2 refers to an O2 which in turn is referring to an O1. On Tue, Sep 5, 2017 at 11:58 AM Benoit Chesneau wrote: > How do you manage the lifetime of a nif resource when it has been created > and can be used by one another? > > 1) I an object O1 and expose it as a resource R1 > 2) I?m created R2 that need the object O1exposed by R1 by passing R1 as an > argument to the nif function used to create R2 > > I would like to make sure that the object is not preemptively killed or > garbaged collected. How would you do it? Any feedback is welcome :) > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji@REDACTED Wed Sep 6 14:30:17 2017 From: kenji@REDACTED (Kenji Rikitake) Date: Wed, 6 Sep 2017 21:30:17 +0900 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170904094905.GB43354@erix.ericsson.se> References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> <20170904094905.GB43354@erix.ericsson.se> Message-ID: Raimo and all: I got late to follow the thread. I think the new function name should be rand:uniform_non_zero/{1,2} because I've rarely seen somethingUPPERCASE names in Erlang functions. (I might be wrong.) On ranges: I'm concerned about how OTP pre-20 (i.e., <= OTP 19.3.x) rand:uniform/1 code might crash or cause bugs when running on the OTP 20 implementation. At least how to write the OTP pre-20-equivalent code should be documented. I have no firm idea on what should be the default behavior on ranges and whether the borders should be inclusive/exclusive to the limit values. In fact, the behavior differs between languages and implementations. Some examples I've found follow: JavaScript math.random(): [0.0, 1.0) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random Python random.uniform(a, b): [a, b] (when a <= b) https://stackoverflow.com/questions/6088077/how-to-get-a-random-number-between-a-float-range C++ std::uniform_real_distribution<> gen(a, b): [a, b) http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution Ruby 2.4.1 Random class: rand(max): [0.0, max) https://ruby-doc.org/core-2.4.1/Random.html "When max is a Float, rand returns a random floating point number between 0.0 and max, including 0.0 and excluding max." R runif(min=0.0, max=1.0): [0.0, 1.0] (See Note) https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Uniform.html Note: "runif will not generate either of the extreme values unless max = min or max-min is small compared to min, and in particular not for the default arguments." MySQL 5.7 RAND(): [0.0, 1.0) https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand PostgreSQL 10 random(): [0.0, 1.0) https://www.postgresql.org/docs/10/static/functions-math.html#functions-math-random-table MS SQL Server: (0.0, 1.0) https://docs.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql "Returns a pseudo-random float value from 0 through 1, exclusive." dSFMT: "[1, 2), [0, 1), (0, 1] and (0, 1)" http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/#dSFMT (dSFMT: SFMT PRNG implementation for double-precision floating point only) It took me an hour to investigate this. Lesson learned: don't take the range definitions for granted. Regards, Kenji Rikitake On Mon, Sep 4, 2017 at 6:49 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Mon, Sep 04, 2017 at 12:37:50PM +1200, Richard A. O'Keefe wrote: > > > > > > On 1/09/17 8:49 PM, Raimo Niskanen wrote: > > >> By the way, given that a common way to make random floats is to > > >> generate a bitvector, consider > > >> (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. > > >> You will notice that the spacing between the values is *almost* > > >> uniform, but not at the end. > > > > > > That sounds interesting but I do not understand. Is that Elixir code? > > > > Nope, Smalltalk. I wanted to use rational arithmetic. In fact I did > > not need to. Here it is in Haskell: > > > [(x * 256) `div` 15 | x <- [0..15]] > > [0,17,34,51,68,85,102,119,136,153,170,187,204,221,238,256] > > > > Let's push that a bit further. Let's generate all possible 10-bit > > integers and map them to the range [0..63]. We find again that > > the gap sizes are not all the same. They can't be. If you > > consider all vectors of N bits and map them to the range > > [0..2**M] they *cannot* be uniformly distributed no matter what > > method you use because (2**M+1) does not divide 2**N. You can > > fix this by rejecting some of the bit vectors, but that would > > be asking everyone to pay extra for a result they don't have any > > particular need for. > > > > I see, but do not quite understand what you are getting at. > > The current left-closed float generator starts with 58 random bits and > puts 53 of these into the mantissa in an IEEE 754 double binary float, > so that would not be it. > > I guess it is a generator for the closed interval [0.0,1.0] or the open > (0.0,1.0) you talk about. If so: > > This one-liner generates over [0.0,1.0]: > (rand:uniform((1 bsl 53) + 1) -1) * math:pow(2, -53) > and it uses an integer range R = (2^53 + 1), which is not dividable by 2. > > The implementation for that range will generate a 58 bit number and then > check if the number is 0 =< X < R*31 and if so return the number mod R > (31 repetitions of the range is all that fits completely in 58 bits). > > If the generated number is R*31 =< X that is in the top truncated interval > then we start over with a new number. > > The implementation may in theory retry forever before finding a good > number, but the odds for retry is about 1/32 for each round so the > accumulated time is about 32/31 times one round. And only once in a > million > it will take 4 attempts or more. > > I discussed a different implementation with Prof. Vigna that is to always > generate one word too much and then use mod R on that which would bury the > skew in one word of random bits hence the difference in probability between > generated numbers should be about (2^58 - 1)/2^58, which would take quite > some effort to measure with statistical significance. But he considered > that as a bad idea since it would get half the speed for most ranges. > > So this is an already solved problem, as I see it. > > We *can* write efficient and good generators for open, closed and > half-closed intervals, if we want. > > So far I have only seen the need for a (0.0,1.0] generator, which can be > implemented with: > 1.0 - rand:uniform() > but in some applications such as 1.0/X and math:log(X) I think that the > form > N * 2^-53 might be less than optimal, so I have a new suggestion: > > https://github.com/erlang/otp/compare/OTP-20.0... > RaimoNiskanen:raimo/stdlib/rand-uniformNZ > > This variant never returns exactly 0.0 and have better precision for low > values. Comments? Especially about the name. > > And so far I have not seen any actual need for (0.0,1.0) nor [0.0,1.0]. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Sep 6 15:33:33 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 6 Sep 2017 15:33:33 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: References: <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> <20170904094905.GB43354@erix.ericsson.se> Message-ID: <20170906133333.GA23762@erix.ericsson.se> On Wed, Sep 06, 2017 at 09:30:17PM +0900, Kenji Rikitake wrote: > Raimo and all: > > I got late to follow the thread. > > I think the new function name should be > rand:uniform_non_zero/{1,2} > because I've rarely seen somethingUPPERCASE > names in Erlang functions. > (I might be wrong.) At least it would be the first (only) in the rand module, so you surely have a point. Another possible name would be rand:uniform_right_closed/{1,2}, or rand:uniform_left_open/{1,2} but the mathematical reference might be a bit obscure. > > On ranges: > > I'm concerned about how OTP pre-20 (i.e., <= OTP 19.3.x) rand:uniform/1 code > might crash or cause bugs when running on the OTP 20 implementation. > At least how to write the OTP pre-20-equivalent code should be documented. If you mean pre-20 code using rand:uniform/1 that later runs on 20.0 or later, then I am not aware of any drastic changes, at least not crashes. Code calling rand:seed/{1,2} will select the explicit algorithm from pre-20 and behave exactly as before. Code not calling rand:seed/{1,2} will as before get a random number sequence it has never seen before. The same applies for code using rand:uniform/0 - it could pre-20 get exactly 0.0 and the same on 20.0 and later. You can, of course, only use the explicit algorithms that existed pre-20 when executing pre-20, and the same algorithms still exists but are not documented on OTP 20.0 and should behave exactly the same. So since there should be nothing more to writing pre-20 code than to read the documentation for the release in question, and that such code should work just as before on 20.0 and later, I do not think there should be any need for documenting that. This should be exactly as expected. > > I have no firm idea on what should be the default behavior on ranges > and whether the borders should be inclusive/exclusive to the limit values. > In fact, the behavior differs between languages and implementations. > Some examples I've found follow: > > JavaScript math.random(): [0.0, 1.0) > https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random > > Python random.uniform(a, b): [a, b] (when a <= b) > https://stackoverflow.com/questions/6088077/how-to-get-a-random-number-between-a-float-range There is a note that the upper bound may depending on rounding not be returned. Really nasty. > > C++ std::uniform_real_distribution<> gen(a, b): [a, b) > http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution > > Ruby 2.4.1 Random class: rand(max): [0.0, max) > https://ruby-doc.org/core-2.4.1/Random.html > "When max is a Float, rand returns a random floating point number > between 0.0 and max, including 0.0 and excluding max." > > R runif(min=0.0, max=1.0): [0.0, 1.0] (See Note) > https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Uniform.html > Note: "runif will not generate either of the extreme values > unless max = min or max-min is small compared to min, > and in particular not for the default arguments." That is a really sloppy definition! > > MySQL 5.7 RAND(): [0.0, 1.0) > https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand > > PostgreSQL 10 random(): [0.0, 1.0) > https://www.postgresql.org/docs/10/static/functions-math.html#functions-math-random-table > > MS SQL Server: (0.0, 1.0) > https://docs.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql > "Returns a pseudo-random float value from 0 through 1, exclusive." > > dSFMT: "[1, 2), [0, 1), (0, 1] and (0, 1)" > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/#dSFMT > (dSFMT: SFMT PRNG implementation for double-precision floating point only) > > It took me an hour to investigate this. > Lesson learned: don't take the range definitions for granted. You serched better than me and did find some that excludes 0.0. Good to know! It seems [0.0, 1.0) dominates but does not rule. Best regards / Raimo > > Regards, > Kenji Rikitake > > On Mon, Sep 4, 2017 at 6:49 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > > > On Mon, Sep 04, 2017 at 12:37:50PM +1200, Richard A. O'Keefe wrote: > > > > > > > > > On 1/09/17 8:49 PM, Raimo Niskanen wrote: > > > >> By the way, given that a common way to make random floats is to > > > >> generate a bitvector, consider > > > >> (0 to: 15) collect: [:each | ((each / 15) * 256) truncated]. > > > >> You will notice that the spacing between the values is *almost* > > > >> uniform, but not at the end. > > > > > > > > That sounds interesting but I do not understand. Is that Elixir code? > > > > > > Nope, Smalltalk. I wanted to use rational arithmetic. In fact I did > > > not need to. Here it is in Haskell: > > > > [(x * 256) `div` 15 | x <- [0..15]] > > > [0,17,34,51,68,85,102,119,136,153,170,187,204,221,238,256] > > > > > > Let's push that a bit further. Let's generate all possible 10-bit > > > integers and map them to the range [0..63]. We find again that > > > the gap sizes are not all the same. They can't be. If you > > > consider all vectors of N bits and map them to the range > > > [0..2**M] they *cannot* be uniformly distributed no matter what > > > method you use because (2**M+1) does not divide 2**N. You can > > > fix this by rejecting some of the bit vectors, but that would > > > be asking everyone to pay extra for a result they don't have any > > > particular need for. > > > > > > > I see, but do not quite understand what you are getting at. > > > > The current left-closed float generator starts with 58 random bits and > > puts 53 of these into the mantissa in an IEEE 754 double binary float, > > so that would not be it. > > > > I guess it is a generator for the closed interval [0.0,1.0] or the open > > (0.0,1.0) you talk about. If so: > > > > This one-liner generates over [0.0,1.0]: > > (rand:uniform((1 bsl 53) + 1) -1) * math:pow(2, -53) > > and it uses an integer range R = (2^53 + 1), which is not dividable by 2. > > > > The implementation for that range will generate a 58 bit number and then > > check if the number is 0 =< X < R*31 and if so return the number mod R > > (31 repetitions of the range is all that fits completely in 58 bits). > > > > If the generated number is R*31 =< X that is in the top truncated interval > > then we start over with a new number. > > > > The implementation may in theory retry forever before finding a good > > number, but the odds for retry is about 1/32 for each round so the > > accumulated time is about 32/31 times one round. And only once in a > > million > > it will take 4 attempts or more. > > > > I discussed a different implementation with Prof. Vigna that is to always > > generate one word too much and then use mod R on that which would bury the > > skew in one word of random bits hence the difference in probability between > > generated numbers should be about (2^58 - 1)/2^58, which would take quite > > some effort to measure with statistical significance. But he considered > > that as a bad idea since it would get half the speed for most ranges. > > > > So this is an already solved problem, as I see it. > > > > We *can* write efficient and good generators for open, closed and > > half-closed intervals, if we want. > > > > So far I have only seen the need for a (0.0,1.0] generator, which can be > > implemented with: > > 1.0 - rand:uniform() > > but in some applications such as 1.0/X and math:log(X) I think that the > > form > > N * 2^-53 might be less than optimal, so I have a new suggestion: > > > > https://github.com/erlang/otp/compare/OTP-20.0... > > RaimoNiskanen:raimo/stdlib/rand-uniformNZ > > > > This variant never returns exactly 0.0 and have better precision for low > > values. Comments? Especially about the name. > > > > And so far I have not seen any actual need for (0.0,1.0) nor [0.0,1.0]. > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ulf@REDACTED Wed Sep 6 17:28:01 2017 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 6 Sep 2017 17:28:01 +0200 Subject: [erlang-questions] pid weirdness on OTP 18 Message-ID: I can't promise I'll be able to reproduce this, as I've never seen it before in my 25 years of programming Erlang. https://github.com/uwiger/gproc/issues/141 Really, the only thing I did was fire up one node, start gproc, then fire up another node on the same machine (a Macbook Pro), ping the first node, start gproc and then issue some gproc commands. Other than the apparent pid duplication, I saw no indication of any problem. Can someone provide a solid explanation or should we just write it off as a ghost in the machine? BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Thu Sep 7 00:11:37 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 06 Sep 2017 22:11:37 +0000 Subject: [erlang-questions] otp edit-compile-test? Message-ID: I am doing some dev on the public_key app and am struggling to find a fast edit-compile-test cycle. I have this ( https://github.com/erlang/otp/wiki/Running-tests)... but it is slow and doesn't seem to be rebuilding my app changes. What are other people doing for this scenario? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Sep 7 00:44:29 2017 From: rvirding@REDACTED (Robert Virding) Date: Thu, 7 Sep 2017 00:44:29 +0200 Subject: [erlang-questions] lists:pmap ? In-Reply-To: References: Message-ID: There are many issues you need to address if you are going to add a "standard" pmap to your library. Some have already been mentioned like how much parallelism and are you going to preserve the list order, which I think is critical. Another one is what to do when you get an error. There are some obvious alternatives: - If you get an error crash all the processes and return an error pretty much in the same way map does - Always return the full list but indicate which elements worked and which didn't, or maybe just return a default "error" value But you could always do something for your specific case. Robert On 30 August 2017 at 13:31, Ulf Wiger wrote: > I discussed the problems with a general pmap in a presentation back in > 2009. > > https://www.infoq.com/presentations/wiger-multicore-erlang > > The purpose wasn't really to argue against a general pmap, but rather to > use it as a simple example to illustrate the problems with treating erlang > processes as 'fibers' rather than independent actors. > > BR, > Ulf W > > 2017-08-22 10:42 GMT+02:00 Ola Andersson A : > >> Reading the discussion about list:mapfind reminded me of a function I >> rediscovered recently. >> It's rpc:pmap/3 that was originally defined for use in a distributed >> environment, spreading out the processing over several nodes. It actually >> works on a single multicore node as well even though it wasn't designed for >> that purpose. >> With the limited tests I have done it seems to significantly outperform >> lists:map/2 and also scale reasonably well. The almost negligible cost of >> spawning erlang processes is still amazing. >> How about adding a lists:pmap/2 function, designed for multicore, in the >> lists module? >> /OLA. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Sep 7 02:02:36 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 7 Sep 2017 12:02:36 +1200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: References: <20170829083528.GA91917@erix.ericsson.se> <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> <20170904094905.GB43354@erix.ericsson.se> Message-ID: <0efe7c07-91f4-f8f3-4cfb-a968d7f2f2c0@cs.otago.ac.nz> On 7/09/17 12:30 AM, Kenji Rikitake wrote: > Raimo and all: > > I got late to follow the thread. > > I think the new function name should be > rand:uniform_non_zero/{1,2} > because I've rarely seen somethingUPPERCASE > names in Erlang functions. > (I might be wrong.) "nonzero" is one word, so rand:uniform_nonzero/{1,2} would be better still. > > JavaScript math.random(): [0.0, 1.0) > https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random > > Python random.uniform(a, b): [a, b] (when a <= b) > https://stackoverflow.com/questions/6088077/how-to-get-a-random-number-between-a-float-range Ad fontes! Check the official documentation. https://docs.python.org/3/library/random.html says quite explicitly "Almost all module functions depend on the basic function random(), which generates a random float uniformly in the semi-open range [0.0, 1.0)." The actual source code for random.uniform(a, b) is def uniform(self, a, b): "Get a random number in the range [a, b) or [a, b] depending on rounding." return a + (b-a) * self.random() Now of course in exact real arithmetic, if 0 <= u < 1 and a < b then a <= (b-a)*u + a < b. So they are using an algorithm that *would* exclude b except for roundoff. And they are leaving it to users to deal with the consequences of the numerical error, and weaselling out of it by blaming the computer. In any case, Python is a [0,1) example. > C++ std::uniform_real_distribution<> gen(a, b): [a, b) > http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution > > Ruby 2.4.1 Random class: rand(max): [0.0, max) > https://ruby-doc.org/core-2.4.1/Random.html > "When max is a Float, rand returns a random floating point number > between 0.0 and max, including 0.0 and excluding max." > > R runif(min=0.0, max=1.0): [0.0, 1.0] (See Note) > https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Uniform.html > Note: "runif will not generate either of the extreme values > unless max = min or max-min is small compared to min, > and in particular not for the default arguments." > > MySQL 5.7 RAND(): [0.0, 1.0) > https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand > > PostgreSQL 10 random(): [0.0, 1.0) > https://www.postgresql.org/docs/10/static/functions-math.html#functions-math-random-table > > MS SQL Server: (0.0, 1.0) > https://docs.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql > "Returns a pseudo-random float value from 0 through 1, exclusive." All of the systems you have mentioned to this point use [0,1) as the building block (or in the case of R, (0,1). > > dSFMT: "[1, 2), [0, 1), (0, 1] and (0, 1)" > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/#dSFMT > (dSFMT: SFMT PRNG implementation for double-precision floating point only) That is to say, it has these functions in the interface: dsfmt_genrand_close1_open2() => [1,2) -- their primitive dsfmt_genrand_close_open() => [0,1) dsfmt_genrand_open_close() => (0,1] dsfmt_genrand_open_open() => (0,1) You *can't* take the range for granted because you have to choose. The one range it does not offer is [0,1]. The key lesson is that none of these systems offers [0,1] as a basic building block, and that to the extent that it is possible to tell, the ones that offer [a,b] as a derived version -- R and Python -- do so as a sloppy implementation of [a,b). For what it's worth, my Smalltalk library now has aRandom next [0,1) nextNonzero (0,1] nextBetween: a andExclusive: b [a,b) nextBetweenExclusive: a and: b (a,b] From frank.muller.erl@REDACTED Thu Sep 7 06:47:48 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 07 Sep 2017 04:47:48 +0000 Subject: [erlang-questions] Progress bar in Erlang Message-ID: Hi guys, I?m looking for a simple progress bar module for Erlang? Found this one, but it?s in Elixir: https://github.com/henrik/progress_bar /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Sep 7 06:49:07 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 07 Sep 2017 04:49:07 +0000 Subject: [erlang-questions] Erlang and sleep mode under Windows Message-ID: Hi I?ve a simple Erlang application under Windows. It runs under erlsrv.exe and all is good. However, I?ve noticed that when the machine enters sleep mode, my app "must be manually restarted" to work again. The reason is still not clear to me, but i suspect the fact that it loses internet connectivity. Questions: 1. Can I detect the sleep mode and recover from it programmatically? 2. To avoid restarting the app, is it possible to restart only the networking part of the VM (not even sure if such a thing exits/feasible)? Thanks in advance. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 7 07:14:16 2017 From: zxq9@REDACTED (zxq9) Date: Thu, 07 Sep 2017 14:14:16 +0900 Subject: [erlang-questions] Progress bar in Erlang In-Reply-To: References: Message-ID: <2402984.EsOT2nKDP5@changa> On 2017?09?07? ??? 04:47:48 Frank Muller wrote: > Hi guys, > > I?m looking for a simple progress bar module for Erlang? > > Found this one, but it?s in Elixir: > https://github.com/henrik/progress_bar The only one I've seen is for Wx GUIs, not text progress bars like the one you linked. I'll probably wind up writing one of these eventually to compliment some text layout tools I've accumulated (which I have not yet extracted out of a few projects into a separate text env library). As a side note... If you decide to write this or translate it to Erlang yourself, keep in mind that writing an output lib in a highly concurrent environment requires (usually) multiplexing output requests from concurrent processes. This is overlooked in many projects so when unexpected things happen (even benign things) the screen turns into a portal of chaos. Abstract the text output device as a process so you can communicate with the "screen" within Erlang via messages and get sequential resource access in a sane way. That way the timing of concurrent output doesn't jam things up. Also, there is the detail of figuring out which sort of terminal you're talking to (Windows just doesn't do anything the way you'd ever expect), which is simple if you just assume its always a unixy terminal but is surprisingly difficult to sort out if you want to develop a complete-ish cross-platform solution (which is why I tend to stick with single-line text output OR leap way over that and go to a GUI environment with Wx). -Craig From sergej.jurecko@REDACTED Thu Sep 7 07:26:51 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Thu, 7 Sep 2017 07:26:51 +0200 Subject: [erlang-questions] Erlang and sleep mode under Windows In-Reply-To: References: Message-ID: <7304302B-FA41-4F41-BA3A-C69E55A66D65@gmail.com> Use erlang:send_after and measure the time it actually took for the message to arrive. If it?s much longer the machine was in sleep mode. Don?t know about the networking issues. You could do init:restart() if it?s really necessary. Regards, Sergej > On 7 Sep 2017, at 06:49, Frank Muller wrote: > > Hi > > I?ve a simple Erlang application under Windows. It runs under erlsrv.exe and all is good. > > However, I?ve noticed that when the machine enters sleep mode, my app "must be manually restarted" to work again. > > The reason is still not clear to me, but i suspect the fact that it loses internet connectivity. > > Questions: > 1. Can I detect the sleep mode and recover from it programmatically? > 2. To avoid restarting the app, is it possible to restart only the networking part of the VM (not even sure if such a thing exits/feasible)? > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 7 08:03:09 2017 From: zxq9@REDACTED (zxq9) Date: Thu, 07 Sep 2017 15:03:09 +0900 Subject: [erlang-questions] Erlang and sleep mode under Windows In-Reply-To: References: Message-ID: <9587531.q2FeV3Pbac@changa> On 2017?09?07? ??? 04:49:07 Frank Muller wrote: > 1. Can I detect the sleep mode and recover from it programmatically? No idea. This is an interesting question, though. I imagine Windows provides some way of figuring this out -- perhaps through a registry query. That said, since Linux, OSX and Windows all have power states it would certainly be useful if the EVM provided some way to subscribe to OS power state changes. -spec os:subscribe(pid(), power_state) -> ok | {error, term()}. (Looking at the spec of that function I am obviously envisioning more uses for that than just power states...) > 2. To avoid restarting the app, is it possible to restart only the > networking part of the VM (not even sure if such a thing exits/feasible)? I don't think there is a way right now to detect sleep mode or "I've just woken back up!" without writing a native Windows service to notify you, but it IS possible to check whether your socket is still connected and programmatically recover from this. Provide a check and reconnect for when a connection goes down and it should handle your network failures for you. Incidentally, that will also make moving between networks work more smoothly. Losing connectivity between sleep intervals or between network transitions and IP reassignments is just a reality of modern client-side coding. -Craig From zxq9@REDACTED Thu Sep 7 08:07:23 2017 From: zxq9@REDACTED (zxq9) Date: Thu, 07 Sep 2017 15:07:23 +0900 Subject: [erlang-questions] Erlang and sleep mode under Windows In-Reply-To: <9587531.q2FeV3Pbac@changa> References: <9587531.q2FeV3Pbac@changa> Message-ID: <2122432.35rHXlLSU4@changa> On 2017?09?07? ??? 15:03:09 zxq9 wrote: > -spec os:subscribe(pid(), power_state) -> ok | {error, term()}. > > (Looking at the spec of that function I am obviously envisioning more uses for that than just power states...) Now that I take a second look at that, it is clear to me that an AHCI subscription and query interface would be immensely useful. A cursory search doesn't bring anything up in this area for Erlang. IMO, this sort of thing really would belong in the runtime. -Craig From zxq9@REDACTED Thu Sep 7 08:14:45 2017 From: zxq9@REDACTED (zxq9) Date: Thu, 07 Sep 2017 15:14:45 +0900 Subject: [erlang-questions] Erlang and sleep mode under Windows In-Reply-To: <2122432.35rHXlLSU4@changa> References: <9587531.q2FeV3Pbac@changa> <2122432.35rHXlLSU4@changa> Message-ID: <1560773.7rPiqc3ssa@changa> On 2017?09?07? ??? 15:07:23 zxq9 wrote: > On 2017?09?07? ??? 15:03:09 zxq9 wrote: > > -spec os:subscribe(pid(), power_state) -> ok | {error, term()}. > > > > (Looking at the spec of that function I am obviously envisioning more uses for that than just power states...) > > Now that I take a second look at that, it is clear to me that an AHCI subscription and query interface would be immensely useful. And ACPI. (I was conflating these two in my mind.) Along with that, any other similar generally uniformly available power/hardware/system status type protocols. I believe the best we can do at the moment is figure out which environment we're in and open a port or make a query (if available) via os:command/1. It might be possible to tail a log file in some environments. Anyway, this seems a lot trickier to work out than having runtime interfaces to AHCI, ACPI, etc. -Craig From dszoboszlay@REDACTED Thu Sep 7 15:39:47 2017 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Thu, 07 Sep 2017 13:39:47 +0000 Subject: [erlang-questions] ssl: TLS-1.2 signature extension In-Reply-To: <738269119.718970.1491216379968.JavaMail.zimbra@tpip.net> References: <738269119.718970.1491216379968.JavaMail.zimbra@tpip.net> Message-ID: Hi, I was backporting the patch (edab501fe ) for this issue (ERL-206 ) to OTP 18. The patch itself does 3 things: 1. Fixes a bug in how TLS version numbers are compared (this part is clear and easy). 2. Makes sure to always send the signature algorithms client hello extension when the client supports TLS 1.2 (this alone is enough to fix interoperability with Microsoft IIS). 3. Sets the record layer version to 0x0301 (TLS 1.2) when the client supports TLS 1.2 (the old behaviour was to use the lowest supported protocol's version number). Personally, I'm not convinced that change 3 is a good idea. I found Ingela quoting RFC 5246 in two places for why is it both necessary and backward compatible: - In ERL-206 : E.1. Compatibility with TLS 1.0/1.1 and SSL 3.0 [...] A TLS 1.2 client who wishes to negotiate with such older servers will send a normal TLS 1.2 ClientHello, containing { 3, 3 } (TLS 1.2) in ClientHello.client_version. If the server does not support this version, it will respond with a ServerHello containing an older version number. If the client agrees to use this version, the negotiation will proceed as appropriate for the negotiated protocol. However, this section of the RFC talks about the version number in the ClientHello, not the record layer version. - In an email from the 9th of August: I investigated it and as it turns out it is not a bug. RFC 5246 says, 7.4.1.4.1 Signature Algorithms[...] Note: this extension is not meaningful for TLS versions prior to 1.2. Clients MUST NOT offer it if they are offering prior versions. However the initial client hello will now be sent with the TLS record protocol with lowest version supported, which it failed to do before and that is why it happened to work before. Extensions are sent for the wanted version and should be ignored if a lower version is negotiated with exception of signature algorithms, due to the sentence above. I understand that the MUST NOT clause says a client cannot both send a signature algorithms extension and offer older versions of TLS (via the record version number). However, I am on the side of Andreas, that violating the RFC seems to be the correct thing to do: 1. OpenSSL (and therefore wget and curl) does ignore the MUST NOT sending: - 0x0301 record layer version - 0x0303 ClientHello version - The signature algorithm extensions 2. The quote from the RFC continues like this: Note: this extension is not meaningful for TLS versions prior to 1.2. Clients MUST NOT offer it if they are offering prior versions. However, even if clients do offer it, the rules specified in [TLSEXT ] require servers to ignore extensions they do not understand. The second sentence pretty much contradicts the first one: clients must not do this, but if they do, it should still work, even with older servers. This explains on what grounds OpenSSL could decide to ignore the MUST NOT clause. 3. We've already encountered some "broken" servers that drop TLS 1.2 connection attempts if they don't contain this specific hello extension. I have a bad feeling that there can be some old and broken servers on the internet too, that do not support TLS 1.2 and would drop a connection attempt if we send version number 0x0303 in the record layer. I couldn't figure out why OpenSSL decided to use the old record layer version and violate the MUST NOT clause, but it would feel safer to do what they do when the RFC is this ambiguous. There are way more OpenSSL users than OTP 20 users, so if their strategy would cause interoperability problems, we would probably know about it already. What do you think? Daniel On Mon, 3 Apr 2017 at 12:46 Andreas Schultz wrote: > Hi, > > I've come across a curious TLS 1.2 connection problem. Erlang 19.x will > not connect to https://validator5.addressdoctor.com/ in the default > configuration or when more than TLS 1.2 is offered. > > After some digging I found out that the server (IIS-8.0) would attempt > to use TLS 1.2, but chokes on the *missing* Signature Algorithms extension. > This is clearly a bug in the server implementation, as it should assume > a default value for the supported signature algorithms. Nevertheless, > every other TLS client works normally. > > Erlang SSL behavior is 100% compliant with RFC-5246: > > > Note: this extension is not meaningful for TLS versions prior to 1.2. > > Clients MUST NOT offer it if they are offering prior versions. > > Tests with OpenSSL and GNU-TLS show that they ignore the "MUST NOT" > from the above clause and include the Signature Algorithms extension > whenever they offer TLS 1.2 regardless of a possible fallback to a > lower version. > > I think Erlang's SSL should do the same. > > There is also an alternative way to understand the clause. It's meaning > depends on "offering prior version". The version offered is the value > from ClientHello.client_version, so as soon as this value is {3, 3}, > a Signature Algorithms extension should be sent. > > > Regards > Andreas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus@REDACTED Thu Sep 7 18:41:20 2017 From: magnus@REDACTED (Magnus Henoch) Date: Thu, 07 Sep 2017 17:41:20 +0100 Subject: [erlang-questions] otp edit-compile-test? In-Reply-To: Daniel Goertzen's message of "Wed\, 06 Sep 2017 22\:11\:37 +0000 \(18 hours\, 25 minutes\, 49 seconds ago\)" Message-ID: Daniel Goertzen writes: > I am doing some dev on the public_key app and am struggling to > find a fast edit-compile-test cycle. I have this ( > https://github.com/erlang/otp/wiki/Running-tests)... but it is > slow and doesn't seem to be rebuilding my app changes. What are > other people doing for this scenario? I arrived at this command line, after identifying the test suite I was interested in: make -j6 && ./otp_build tests && cd release/tests && ../../bin/ct_run -pa ../../lib/common_test/test_server -suite ssl_test/ssl_dist_SUITE Hope this helps, Magnus From davidnwelton@REDACTED Thu Sep 7 22:40:33 2017 From: davidnwelton@REDACTED (David Welton) Date: Thu, 7 Sep 2017 13:40:33 -0700 Subject: [erlang-questions] API design and pgapp In-Reply-To: References: Message-ID: Hi, > I don't think it's a productive addition to that epgsql conversation, but I would write only update_stats/1. A Postgres connection is not a trivial thing like an Erlang process. On a default installation you only get a couple hundred of them and you can see some unpleasant surprises if you go over. Therefore, I think it's a mistake to not explicitly track your connections in the code. You could say the same thing about other resources, like, say, memory. Or write a system that helps the programmer work at a higher level by setting up some 'good defaults' and managing the underlying resources. pgapp is the latter kind of system. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From raimo+erlang-questions@REDACTED Fri Sep 8 14:26:28 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 8 Sep 2017 14:26:28 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <0efe7c07-91f4-f8f3-4cfb-a968d7f2f2c0@cs.otago.ac.nz> References: <20170830064202.GA17946@erix.ericsson.se> <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> <20170904094905.GB43354@erix.ericsson.se> <0efe7c07-91f4-f8f3-4cfb-a968d7f2f2c0@cs.otago.ac.nz> Message-ID: <20170908122628.GA77024@erix.ericsson.se> To conclude =========== It would be convenient to have functions in the rand module that generates on the interval (0.0, 1.0] e.g rand:uniform_nonzero/0 and rand:uniform_nonzero_s/1 or maybe rand:uniform_nz/0 and rand:uniform_nz_s/1. But since it is very easy to use e.g (1.0 - rand:uniform()) I see little value in adding them. Adding hints in the documentation could suffice. However, I think we could add functions with the same names and interval that also has got a different uniformity i.e still uniform, but not equidistant, instead increasing precison towards 0. This would have a greater value. Such a uniformity would work better for some suggested algorithms such as Box-Muller. Ironically, the implementation by Kenji that I changed was a few bits in that direction i.e used the generators' extra bits over 53 (5 or 11) for increased precision, but I want to have at least 53 extra bits which I hope is close enough to infinity. I have implemented such functions in my own GitHub repo, but ran into problems with the number distribution that I suspect is caused by rounding errors in the current implementation of erlang:float/1. So I will try to find the time to investigate that further... / Raimo On Thu, Sep 07, 2017 at 12:02:36PM +1200, Richard A. O'Keefe wrote: > > > On 7/09/17 12:30 AM, Kenji Rikitake wrote: > > Raimo and all: > > > > I got late to follow the thread. > > > > I think the new function name should be > > rand:uniform_non_zero/{1,2} > > because I've rarely seen somethingUPPERCASE > > names in Erlang functions. > > (I might be wrong.) > > "nonzero" is one word, so rand:uniform_nonzero/{1,2} > would be better still. > > > > JavaScript math.random(): [0.0, 1.0) > > https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random > > > > Python random.uniform(a, b): [a, b] (when a <= b) > > https://stackoverflow.com/questions/6088077/how-to-get-a-random-number-between-a-float-range > > Ad fontes! Check the official documentation. > https://docs.python.org/3/library/random.html > says quite explicitly > "Almost all module functions depend on the basic function random(), > which generates a random float uniformly in the semi-open range > [0.0, 1.0)." > > The actual source code for random.uniform(a, b) is > > def uniform(self, a, b): > "Get a random number in the range > [a, b) or [a, b] depending on rounding." > return a + (b-a) * self.random() > > Now of course in exact real arithmetic, if 0 <= u < 1 > and a < b then a <= (b-a)*u + a < b. So they are using > an algorithm that *would* exclude b except for roundoff. > And they are leaving it to users to deal with the > consequences of the numerical error, and weaselling out > of it by blaming the computer. > > In any case, Python is a [0,1) example. > > > C++ std::uniform_real_distribution<> gen(a, b): [a, b) > > http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution > > > > Ruby 2.4.1 Random class: rand(max): [0.0, max) > > https://ruby-doc.org/core-2.4.1/Random.html > > "When max is a Float, rand returns a random floating point number > > between 0.0 and max, including 0.0 and excluding max." > > > > R runif(min=0.0, max=1.0): [0.0, 1.0] (See Note) > > https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Uniform.html > > Note: "runif will not generate either of the extreme values > > unless max = min or max-min is small compared to min, > > and in particular not for the default arguments." > > > > MySQL 5.7 RAND(): [0.0, 1.0) > > https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand > > > > PostgreSQL 10 random(): [0.0, 1.0) > > https://www.postgresql.org/docs/10/static/functions-math.html#functions-math-random-table > > > > MS SQL Server: (0.0, 1.0) > > https://docs.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql > > "Returns a pseudo-random float value from 0 through 1, exclusive." > > All of the systems you have mentioned to this point use [0,1) > as the building block (or in the case of R, (0,1). > > > > dSFMT: "[1, 2), [0, 1), (0, 1] and (0, 1)" > > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/#dSFMT > > (dSFMT: SFMT PRNG implementation for double-precision floating point only) > > That is to say, it has these functions in the interface: > dsfmt_genrand_close1_open2() => [1,2) -- their primitive > dsfmt_genrand_close_open() => [0,1) > dsfmt_genrand_open_close() => (0,1] > dsfmt_genrand_open_open() => (0,1) > > You *can't* take the range for granted because you have to choose. > The one range it does not offer is [0,1]. > > The key lesson is that none of these systems offers [0,1] as a > basic building block, and that to the extent that it is possible > to tell, the ones that offer [a,b] as a derived version -- R and > Python -- do so as a sloppy implementation of [a,b). > > For what it's worth, my Smalltalk library now has > aRandom > next [0,1) > nextNonzero (0,1] > nextBetween: a andExclusive: b [a,b) > nextBetweenExclusive: a and: b (a,b] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From gootik_2003@REDACTED Fri Sep 8 16:05:01 2017 From: gootik_2003@REDACTED (Sasan Hezarkhani) Date: Fri, 8 Sep 2017 14:05:01 +0000 Subject: [erlang-questions] Erlang newsletter and Community Survey Message-ID: Hi all, It's me again with Erlang weekly updates. I thought it's good to update everyone on a few things: 1. First of all this week's updates are here: http://bit.ly/twie-sept8 2. I have setup a mailing list to send the newsletter to. Sign up and you will get an update every Friday :) http://eepurl.com/c2xmWr 3. As mentioned in the post, Mariano is running an Erlang community survey, I think everyone should fill it out and help make a better community: http://bit.ly/state-of-beam-2017 Again, sorry if these posts are getting annoying. Thank you -Sasan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sperber@REDACTED Mon Sep 11 14:47:06 2017 From: sperber@REDACTED (Michael Sperber) Date: Mon, 11 Sep 2017 13:47:06 +0100 Subject: [erlang-questions] Call for Contributions: BOB 2018 - Berlin, Feb 23, 2018 Message-ID: Erlang submissions are very welcome at BOB! BOB Conference 2018 "What happens when we use what's best for a change?" http://bobkonf.de/2018/en/cfp.html Berlin, February 23 Call for Contributions Deadline: October 29, 2017 You are actively engaged in advanced software engineering methods, implement ambitious architectures and are open to cutting-edge innovation? Attend this conference, meet people that share your goals, and get to know the best software tools and technologies available today. We strive to offer a day full of new experiences and impressions that you can use to immediately improve your daily life as a software developer. If you share our vision and want to contribute, submit a proposal for a talk or tutorial! Topics ------ We are looking for talks about best-of-breed software technology, e.g.: - functional programming - persistent data structures and databases - types - formal methods for correctness and robustness - abstractions for concurrency and parallelism - metaprogramming - probabilistic programming - math and programming - controlled side effects - beyond REST and SOAP - effective abstractions for data analytics - ... everything really that isn?t mainstream, but you think should be. Presenters should provide the audience with information that is practically useful for software developers. We?re especially interested in experience reports. But this could also take other forms, e.g.: - introductory talks on technical background - overviews of a given field - demos and how-tos Requirements ------------ We accept proposals for presentations of 45 minutes (40 minutes talk + 5 minutes questions), as well as 90 minute tutorials for beginners. The language of presentation should be either English or German. Your proposal should include (in your presentation language of choice): - an abstract of max. 1500 characters. - a short bio/cv - contact information (including at least email address) - a list of 3-5 concrete ideas of how your work can be applied in a developer?s daily life - additional material (websites, blogs, slides, videos of past presentations, ...) Submit here: https://docs.google.com/forms/d/e/1FAIpQLSdjgwulSMpaITJ6q6cK_ndrfR1FlEs_HQlZy04LnUKC-ArCaQ/viewform?usp=sf_link Organisation - direct questions to contact at bobkonf dot de - proposal deadline: October 29, 2017 - notification: November 13, 2017 - program: December 1, 2017 NOTE: The conference fee will be waived for presenters, but travel expenses will not be covered. Speaker Grants -------------- BOB has Speaker Grants available to support speakers from groups under-represented in technology. We specifically seek women speakers and speakers who are not be able to attend the conference for financial reasons. Details are here: http://bobkonf.de/2018/en/speaker-grants.html Shepherding ----------- The program committee offers shepherding to all speakers. Shepherding provides speakers assistance with preparing their sessions, as well as a review of the talk slides. Program Committee ----------------- (more information here: http://bobkonf.de/2018/en/programmkomitee.html) - Matthias Fischmann, zerobuzz UG - Matthias Neubauer, SICK AG - Nicole Rauch, Softwareentwicklung und Entwicklungscoaching - Michael Sperber, Active Group - Stefan Wehr, factis research Scientific Advisory Board - Annette Bieniusa, TU Kaiserslautern - Torsten Grust, Uni T?bingen - Peter Thiemann, Uni Freiburg From lukas@REDACTED Mon Sep 11 15:32:18 2017 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 11 Sep 2017 15:32:18 +0200 Subject: [erlang-questions] +secio emulator flag In-Reply-To: References: Message-ID: Hello, On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson wrote: > I noticed that the secio emulator flag (scheduler eager check for io) > default value has changed from false to true since ERTS 7.0. Could someone > explain what this means in practice, I do not quite understand the line: > "This, however, also implies that execution of outstanding I/O operation is > not prioritized to the same extent as when false is passed." > The eager check io flag instructs the emulator when it should check for new I/O events (internally called check_io) in the fds that it is currently interested in. Without secio, the schedulers will first finish with all work generated from the previous check_io. If a lot of I/O events where triggered by the check_io, ports may be temporarily prioritized higher than processes in order to be able to do the next check_io sooner. With secio, the next check_io can be done before all events of the previous check_io are consumed. When this feature is enabled, the temporary higher prioritization of ports is removed. This is the change in priority that the documentation is trying to explain. > Is there any special cases where you would prefer to still have it set to > false. > I see no reason to not have secio enabled. Lukas (sorry for the late reply, I meant to reply to this before but forgot) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Sep 11 15:36:07 2017 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 11 Sep 2017 15:36:07 +0200 Subject: [erlang-questions] Improved IO polling in erts In-Reply-To: References: Message-ID: I've pushed a number of bug fixes to the branch on github. Again I would like to encourage you all to try the changes out and see if you can notice any differences in your system. Lukas On Tue, Aug 29, 2017 at 8:11 PM, Lukas Larsson wrote: > Hello everyone, > > I've just opened up a new PR at github containing a solution to the > scalability problems that the current polling mechanism in erts has. You > can read more about it in the PR here: https://github.com/ > erlang/otp/pull/1552. > > Our internal measurements have shown great potential, but creating > realistic benchmarks is very hard. I would therefore encourage as many as > possible to test and see how the changes proposed effects your > applications. Please come back with any findings/questions, be they good or > bad. As this is an essential part of many applications, we want to make the > new implementation as good as can be. > > Lukas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Sep 11 17:38:42 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 11 Sep 2017 15:38:42 +0000 Subject: [erlang-questions] Lager dropping messages Message-ID: Hi guys, I?m seeing this annoying warning in my logs: [warning] lager_error_logger_h dropped 107 messages in the last second that exceeded the limit of 50 messages/sec. How can i set Lager (I?m using lager v2 from trunk) and increase this limit of 50msg/sec (ex. set it 300)? Thank you. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From lbakken@REDACTED Mon Sep 11 18:02:40 2017 From: lbakken@REDACTED (Luke Bakken) Date: Mon, 11 Sep 2017 09:02:40 -0700 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: Hi Frank, I think this is what you're looking for: https://github.com/erlang-lager/lager#overload-protection On Mon, Sep 11, 2017 at 8:38 AM, Frank Muller wrote: > Hi guys, > > I?m seeing this annoying warning in my logs: > [warning] lager_error_logger_h dropped 107 messages in the last second that > exceeded the limit of 50 messages/sec. > > How can i set Lager (I?m using lager v2 from trunk) and increase this limit > of 50msg/sec (ex. set it 300)? > > > Thank you. > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Staff Software Engineer Pivotal / RabbitMQ From jesper.louis.andersen@REDACTED Mon Sep 11 20:27:00 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 11 Sep 2017 18:27:00 +0000 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: One thing to be aware of here: When your Erlang instance boots, it logs quite much. Often so much that this overload protection will trigger. If your node() is dying early on in the boot phase, consider bumping this count to 3000 or so. On Mon, Sep 11, 2017 at 7:05 PM Luke Bakken wrote: > Hi Frank, > > I think this is what you're looking for: > https://github.com/erlang-lager/lager#overload-protection > > On Mon, Sep 11, 2017 at 8:38 AM, Frank Muller > wrote: > > Hi guys, > > > > I?m seeing this annoying warning in my logs: > > [warning] lager_error_logger_h dropped 107 messages in the last second > that > > exceeded the limit of 50 messages/sec. > > > > How can i set Lager (I?m using lager v2 from trunk) and increase this > limit > > of 50msg/sec (ex. set it 300)? > > > > > > Thank you. > > /Frank > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Staff Software Engineer > Pivotal / RabbitMQ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Sep 11 20:51:54 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 11 Sep 2017 18:51:54 +0000 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: Thanks for the guidance guys. Kind of lost with my new lager's settings: lager, [{error_logger_hwm,30}, {async_threshold,500}, {async_threshold_window,250}, I'm still seeing dropped messages :-/ What the correct settings for let's say 3000 as suggested by Jesper? /Frank wrote : > One thing to be aware of here: When your Erlang instance boots, it logs > quite much. Often so much that this overload protection will trigger. If > your node() is dying early on in the boot phase, consider bumping this > count to 3000 or so. > > > On Mon, Sep 11, 2017 at 7:05 PM Luke Bakken wrote: > >> Hi Frank, >> >> I think this is what you're looking for: >> https://github.com/erlang-lager/lager#overload-protection >> >> On Mon, Sep 11, 2017 at 8:38 AM, Frank Muller >> wrote: >> > Hi guys, >> > >> > I?m seeing this annoying warning in my logs: >> > [warning] lager_error_logger_h dropped 107 messages in the last second >> that >> > exceeded the limit of 50 messages/sec. >> > >> > How can i set Lager (I?m using lager v2 from trunk) and increase this >> limit >> > of 50msg/sec (ex. set it 300)? >> > >> > >> > Thank you. >> > /Frank >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> >> >> >> -- >> Staff Software Engineer >> Pivotal / RabbitMQ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Mon Sep 11 21:35:24 2017 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 11 Sep 2017 15:35:24 -0400 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: <20170911193523.GH49540@vm.hijacked.us> On Mon, Sep 11, 2017 at 06:51:54PM +0000, Frank Muller wrote: > Thanks for the guidance guys. Kind of lost with my new lager's settings: > > lager, > [{error_logger_hwm,30}, > {async_threshold,500}, > {async_threshold_window,250}, > > I'm still seeing dropped messages :-/ > > What the correct settings for let's say 3000 as suggested by Jesper? > > /Frank You will want to significantly raise the error_logger HWM, the HWM is how many messages lager will pop from the error_logger mailbox before starting to discard them. Also, you might want to try lager 3.x, I landed some fixes to the error_logger discard code a few months ago: https://github.com/erlang-lager/lager/pull/411 Andrew From andrew@REDACTED Mon Sep 11 21:33:20 2017 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 11 Sep 2017 15:33:20 -0400 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: <20170911193320.GG49540@vm.hijacked.us> On Mon, Sep 11, 2017 at 06:51:54PM +0000, Frank Muller wrote: > Thanks for the guidance guys. Kind of lost with my new lager's settings: > > lager, > [{error_logger_hwm,30}, > {async_threshold,500}, > {async_threshold_window,250}, > > I'm still seeing dropped messages :-/ > > What the correct settings for let's say 3000 as suggested by Jesper? > > /Frank You will want to significantly raise the error_logger HWM, the HWM is how many messages lager will pop from the error_logger mailbox before starting to discard them. Also, you might want to try lager 3.x, I landed some fixes to the error_logger discard code a few months ago: https://github.com/erlang-lager/lager/pull/411 Andrew From andrew@REDACTED Mon Sep 11 21:10:19 2017 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 11 Sep 2017 15:10:19 -0400 Subject: [erlang-questions] Lager dropping messages In-Reply-To: References: Message-ID: <20170911191019.GF49540@vm.hijacked.us> On Mon, Sep 11, 2017 at 06:51:54PM +0000, Frank Muller wrote: > Thanks for the guidance guys. Kind of lost with my new lager's settings: > > lager, > [{error_logger_hwm,30}, > {async_threshold,500}, > {async_threshold_window,250}, > > I'm still seeing dropped messages :-/ > > What the correct settings for let's say 3000 as suggested by Jesper? > > /Frank You will want to significantly raise the error_logger HWM, the HWM is how many messages lager will pop from the error_logger mailbox before starting to discard them. Also, you might want to try lager 3.x, I landed some fixes to the error_logger discard code a few months ago: https://github.com/erlang-lager/lager/pull/411 Andrew From mjtruog@REDACTED Mon Sep 11 22:09:26 2017 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 11 Sep 2017 13:09:26 -0700 Subject: [erlang-questions] +secio emulator flag In-Reply-To: References: Message-ID: <59B6ED76.4080806@gmail.com> On 09/11/2017 06:32 AM, Lukas Larsson wrote: > Hello, > > On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson > wrote: > > I noticed that the secio emulator flag (scheduler eager check for io) default value has changed from false to true since ERTS 7.0. Could someone explain what this means in practice, I do not quite understand the line: "This, however, also implies that execution of outstanding I/O operation is not prioritized to the same extent as when false is passed." > > > The eager check io flag instructs the emulator when it should check for new I/O events (internally called check_io) in the fds that it is currently interested in. > > Without secio, the schedulers will first finish with all work generated from the previous check_io. If a lot of I/O events where triggered by the check_io, ports may be temporarily prioritized higher than processes in order to be able to do the next check_io sooner. > > With secio, the next check_io can be done before all events of the previous check_io are consumed. When this feature is enabled, the temporary higher prioritization of ports is removed. This is the change in priority that the documentation is trying to explain. > > Is there any special cases where you would prefer to still have it set to false. > > > I see no reason to not have secio enabled. > I have tested in the past to determine that secio can make Erlang source code performance poor, if it isn't utilizing an Erlang port type, though the testing was with Erlang/OTP 17.4 (https://github.com/CloudI/loadtests/raw/master/tests/http_req/loadtest/results_v1_4_0/201412_summary.pdf). Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Sep 12 06:10:32 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 12 Sep 2017 04:10:32 +0000 Subject: [erlang-questions] Lager dropping messages In-Reply-To: <20170911191019.GF49540@vm.hijacked.us> References: <20170911191019.GF49540@vm.hijacked.us> Message-ID: Working now ({error_logger_hwm,3000}), thanks. /Frank On Mon, Sep 11, 2017 at 06:51:54PM +0000, Frank Muller wrote: > > Thanks for the guidance guys. Kind of lost with my new lager's settings: > > > > lager, > > [{error_logger_hwm,30}, > > {async_threshold,500}, > > {async_threshold_window,250}, > > > > I'm still seeing dropped messages :-/ > > > > What the correct settings for let's say 3000 as suggested by Jesper? > > > > /Frank > > You will want to significantly raise the error_logger HWM, the HWM is > how many messages lager will pop from the error_logger mailbox before > starting to discard them. > > Also, you might want to try lager 3.x, I landed some fixes to the > error_logger discard code a few months ago: > > https://github.com/erlang-lager/lager/pull/411 > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From karlsson.rm@REDACTED Tue Sep 12 19:23:52 2017 From: karlsson.rm@REDACTED (Mikael Karlsson) Date: Tue, 12 Sep 2017 19:23:52 +0200 Subject: [erlang-questions] +secio emulator flag In-Reply-To: <59B6ED76.4080806@gmail.com> References: <59B6ED76.4080806@gmail.com> Message-ID: Thanks Lukas and Michael, > (sorry for the late reply, I meant to reply to this before but forgot) No problem, maybe your personal secio flag also was set to true and something else came in between :-) . The use case I have here is a single (tcp) connection with high load. I think that there are other bottlenecks with this scenario since one can only attach one scheduler/erlang process to this, if I understand it right, but nevertheless would setting secio to false give the attached erlang process more time to finish the work in this case? Best Regards Mikael 2017-09-11 22:09 GMT+02:00 Michael Truog : > On 09/11/2017 06:32 AM, Lukas Larsson wrote: > > Hello, > > On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson > wrote: >> >> I noticed that the secio emulator flag (scheduler eager check for io) >> default value has changed from false to true since ERTS 7.0. Could someone >> explain what this means in practice, I do not quite understand the line: >> "This, however, also implies that execution of outstanding I/O operation is >> not prioritized to the same extent as when false is passed." > > > The eager check io flag instructs the emulator when it should check for new > I/O events (internally called check_io) in the fds that it is currently > interested in. > > Without secio, the schedulers will first finish with all work generated from > the previous check_io. If a lot of I/O events where triggered by the > check_io, ports may be temporarily prioritized higher than processes in > order to be able to do the next check_io sooner. > > With secio, the next check_io can be done before all events of the previous > check_io are consumed. When this feature is enabled, the temporary higher > prioritization of ports is removed. This is the change in priority that the > documentation is trying to explain. > >> >> Is there any special cases where you would prefer to still have it set to >> false. > > > I see no reason to not have secio enabled. > > I have tested in the past to determine that secio can make Erlang source > code performance poor, if it isn't utilizing an Erlang port type, though the > testing was with Erlang/OTP 17.4 > (https://github.com/CloudI/loadtests/raw/master/tests/http_req/loadtest/results_v1_4_0/201412_summary.pdf). > > Best Regards, > Michael From mjtruog@REDACTED Tue Sep 12 19:49:13 2017 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 12 Sep 2017 10:49:13 -0700 Subject: [erlang-questions] +secio emulator flag In-Reply-To: References: <59B6ED76.4080806@gmail.com> Message-ID: <59B81E19.2030908@gmail.com> On 09/12/2017 10:23 AM, Mikael Karlsson wrote: > Thanks Lukas and Michael, > >> (sorry for the late reply, I meant to reply to this before but forgot) > No problem, maybe your personal secio flag also was set to true and > something else came in between :-) . > > The use case I have here is a single (tcp) connection with high load. > I think that there are other bottlenecks with this scenario since one > can only attach one scheduler/erlang process to this, if I understand > it right, but nevertheless would setting secio to false give the > attached erlang process more time to finish the work in this case? My impression is that secio set to true will help if you really do have lots of throughput coming through some number of Erlang ports (sockets or other file descriptors), though it may reduce the performance of Erlang source code that doesn't need to use Erlang ports, so, heavy use of an Erlang process that benefits from spending as much time on a scheduler as possible, though that is based on older testing. The default changed from false to true, so it appears that most use should benefit from keeping it set to true. Ideally, you could loadtest with the setting set either way, to see which is best for your use-case. Best Regards, Michael From karlsson.rm@REDACTED Tue Sep 12 21:51:43 2017 From: karlsson.rm@REDACTED (Mikael Karlsson) Date: Tue, 12 Sep 2017 21:51:43 +0200 Subject: [erlang-questions] +secio emulator flag In-Reply-To: <59B81E19.2030908@gmail.com> References: <59B6ED76.4080806@gmail.com> <59B81E19.2030908@gmail.com> Message-ID: 2017-09-12 19:49 GMT+02:00 Michael Truog wrote: > My impression is that secio set to true will help if you really do have lots > of throughput coming through some number of Erlang ports (sockets or other > file descriptors), though it may reduce the performance of Erlang source > code that doesn't need to use Erlang ports, so, heavy use of an Erlang > process that benefits from spending as much time on a scheduler as possible, > though that is based on older testing. The default changed from false to > true, so it appears that most use should benefit from keeping it set to > true. Ideally, you could loadtest with the setting set either way, to see > which is best for your use-case. Thanks for making it clear. Best Regards Mikael From eshikafe@REDACTED Tue Sep 12 22:10:24 2017 From: eshikafe@REDACTED (austin aigbe) Date: Tue, 12 Sep 2017 21:10:24 +0100 Subject: [erlang-questions] Erlang VM in Rust Message-ID: Hello, Any consideration for implementing the Erlang VM in Rust? Regards, Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Tue Sep 12 22:25:48 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Tue, 12 Sep 2017 22:25:48 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: I tried to begin once. Here's the catch: - Rust had no unions at the time of my experiments (added in 1.19 - July 2017) - Rust has no bit fields (some libraries exist trying to alleviate this) - Erlang process heap is created and maintained by abusing pointers and bits in them. Casting integers to pointers and back i believe is possible but will give you some pain. - A really fast VM loop is done using C/C++ goto (void*) extension, in Rust you will probably have to use old good switch/case (slower) When you overcome these base problems, it should be possible to proceed carefully :) 2017-09-12 22:10 GMT+02:00 austin aigbe : > Hello, > > Any consideration for implementing the Erlang VM in Rust? > > Regards, > Austin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Sep 13 00:27:00 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 13 Sep 2017 00:27:00 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: Any thoughts on the benefits existing Erlang programs would get from a VM implemented in Rust? It may or may not simplify working on the VM and may or may not make it more solid. But what would it improve for the programs written in Erlang themselves? Cheers, On 09/12/2017 10:10 PM, austin aigbe wrote: > Hello, > > Any consideration for implementing the Erlang VM in Rust? > > Regards, > Austin > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From zxq9@REDACTED Wed Sep 13 00:41:43 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 07:41:43 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <9714375.VumhWqLY8o@changa> On 2017?09?13? ??? 00:27:00 Lo?c Hoguin wrote: > Any thoughts on the benefits existing Erlang programs would get from a > VM implemented in Rust? > > It may or may not simplify working on the VM and may or may not make it > more solid. But what would it improve for the programs written in Erlang > themselves? In an ideal world, nothing. I would hope that the only benefit to those coding in Erlang is that maintenance would become more of a sure thing for the OTP maintainers and that, in particular, new feature implementation and performance improvements would become more obvious and safer to implement over time. Those benefits would be beneficial only indirectly. That said... I really can't imagine such a large, old, well documented, well explored runtime known to such a large team would be rewritten to a new language. I can imagine that some of us (probably including some of the OTP team itself) would enjoy working on a rewrite in Rust that is an alternative to, but not a replacement for, the current runtime. Competing runtimes occasionally expose hidden benefits. But a blanket replacement would be a massive, end-of-Netscape type upheaval. It is not as though Erlang is implemented in an obfuscated style of C++ that suffers from fundamental structure and human incompatability issues. -Craig From puzza007@REDACTED Wed Sep 13 00:48:08 2017 From: puzza007@REDACTED (Paul Oliver) Date: Tue, 12 Sep 2017 22:48:08 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <9714375.VumhWqLY8o@changa> References: <9714375.VumhWqLY8o@changa> Message-ID: On Wed, Sep 13, 2017 at 10:42 AM zxq9 wrote: > On 2017?09?13? ??? 00:27:00 Lo?c Hoguin wrote: > > Any thoughts on the benefits existing Erlang programs would get from a > > VM implemented in Rust? > > > > It may or may not simplify working on the VM and may or may not make it > > more solid. But what would it improve for the programs written in Erlang > > themselves? > > In an ideal world, nothing. I would hope that the only benefit to > those coding in Erlang is that maintenance would become more of a sure > thing for the OTP maintainers and that, in particular, new feature > implementation and performance improvements would become more obvious > and safer to implement over time. Those benefits would be beneficial > only indirectly. > > That said... I really can't imagine such a large, old, well documented, > well explored runtime known to such a large team would be rewritten to > a new language. I can imagine that some of us (probably including some > of the OTP team itself) would enjoy working on a rewrite in Rust that > is an alternative to, but not a replacement for, the current runtime. > > Competing runtimes occasionally expose hidden benefits. But a blanket > replacement would be a massive, end-of-Netscape type upheaval. It is > not as though Erlang is implemented in an obfuscated style of C++ that > suffers from fundamental structure and human incompatability issues. > > Large C projects can potentially be rewritten in Rust gradually. An example of this can be found in Remacs ( http://www.wilfred.me.uk/blog/2017/01/11/announcing-remacs-porting-emacs-to-rust/ ). -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshmbarney@REDACTED Wed Sep 13 00:48:49 2017 From: joshmbarney@REDACTED (Joshua Barney) Date: Tue, 12 Sep 2017 18:48:49 -0400 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: I think the real point of implementing the Erlang VM in Rust is to increase the awesomeness 100 fold But really, the only thing I can think of that affects erlang programs is that it could make is simpler to write nifs that can't crash your whole vm. I don't know how, however. I feel like cargo could have a similar community building effect as npm has had for nodejs--an explosion of creativity that resulted in a lot of good quality tools for everyone to use. It's conceivable to me that development of a rust/erl VM could move very quickly by leveraging cargo crates Sent from my iPhone > On Sep 12, 2017, at 6:27 PM, Lo?c Hoguin wrote: > > Any thoughts on the benefits existing Erlang programs would get from a VM implemented in Rust? > > It may or may not simplify working on the VM and may or may not make it more solid. But what would it improve for the programs written in Erlang themselves? > > Cheers, > >> On 09/12/2017 10:10 PM, austin aigbe wrote: >> Hello, >> Any consideration for implementing the Erlang VM in Rust? >> Regards, >> Austin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From scott_ribe@REDACTED Wed Sep 13 00:50:48 2017 From: scott_ribe@REDACTED (scott ribe) Date: Tue, 12 Sep 2017 16:50:48 -0600 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: > On Sep 12, 2017, at 4:48 PM, Joshua Barney wrote: > > But really, the only thing I can think of that affects erlang programs is that it could make is simpler to write nifs that can't crash your whole vm. I don't know how, however. > There's already a Rust library to support exactly that. -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From t@REDACTED Wed Sep 13 02:05:33 2017 From: t@REDACTED (Tristan Sloughter) Date: Tue, 12 Sep 2017 17:05:33 -0700 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <1505261133.582606.1104113280.714AB626@webmail.messagingengine.com> Maybe Tony 'the tiger' Ramine would come back to working on Erlang if it was in Rust! -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED From dave@REDACTED Wed Sep 13 03:08:05 2017 From: dave@REDACTED (David Goehrig) Date: Tue, 12 Sep 2017 21:08:05 -0400 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: Based on past experience, every rewrite in solution/language du jour merely fragments the community and introduces new classes of incompatible bugs. The broad cost in social capital outweighs any narrow technical benefits. I have seen it in my own projects. Even projects like Python, Ruby, and Perl are suffering the social costs of rewrites. We should remember language is a tool for communication between humans, and they are not as easy to program :) Dave On Sep 12, 2017 6:27 PM, "Lo?c Hoguin" wrote: > Any thoughts on the benefits existing Erlang programs would get from a VM > implemented in Rust? > > It may or may not simplify working on the VM and may or may not make it > more solid. But what would it improve for the programs written in Erlang > themselves? > > Cheers, > > On 09/12/2017 10:10 PM, austin aigbe wrote: > >> Hello, >> >> Any consideration for implementing the Erlang VM in Rust? >> >> Regards, >> Austin >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Sep 13 03:28:18 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 13 Sep 2017 13:28:18 +1200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <9714375.VumhWqLY8o@changa> Message-ID: As I see it, the main problem rewriting (or better still, rebuilding) Erlang in Rust is that I'd rather see that human time spent on improving the system we already have. As an open source project, Erlang is or was eligible for free checking by Coverity. I see on the Coverity web site that the last check was in August 2015, a little over 2 years ago. Is there a plan to have it checked again? From zxq9@REDACTED Wed Sep 13 03:30:28 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 10:30:28 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <1505261133.582606.1104113280.714AB626@webmail.messagingengine.com> References: <1505261133.582606.1104113280.714AB626@webmail.messagingengine.com> Message-ID: <3587013.qFDS1ZT4Xo@changa> On 2017?09?12? ??? 17:05:33 Tristan Sloughter wrote: > Maybe Tony 'the tiger' Ramine would come back to working on Erlang if it > was in Rust! Hey! I really like that guy. Almost a good enough reason on its own. But I'm being selfish. ;-) -Craig From thom.cherryhomes@REDACTED Wed Sep 13 03:47:08 2017 From: thom.cherryhomes@REDACTED (Thom Cherryhomes) Date: Wed, 13 Sep 2017 01:47:08 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: For a language that purports to be a replacement for low-level systems programming, Rust not having bitfields or unions, would be a real pain in the ass to do any intricate hardware driver work... I'm guessing that Rust is designed by 20-somethings who have never done a line of assembler in their lives? sigh. -Thom On Tue, Sep 12, 2017 at 3:25 PM Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > I tried to begin once. > > Here's the catch: > > - Rust had no unions at the time of my experiments (added in 1.19 - > July 2017) > - Rust has no bit fields (some libraries exist trying to alleviate > this) > - Erlang process heap is created and maintained by abusing pointers > and bits in them. Casting integers to pointers and back i believe is > possible but will give you some pain. > - A really fast VM loop is done using C/C++ goto (void*) extension, in > Rust you will probably have to use old good switch/case (slower) > > When you overcome these base problems, it should be possible to proceed > carefully :) > > 2017-09-12 22:10 GMT+02:00 austin aigbe : > >> Hello, >> >> Any consideration for implementing the Erlang VM in Rust? >> >> Regards, >> Austin >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 13 04:26:46 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 11:26:46 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <6197571.1QanT7AGF8@changa> On 2017?09?13? ??? 01:47:08 Thom Cherryhomes wrote: > For a language that purports to be a replacement for low-level systems > programming, Rust not having bitfields or unions, would be a real pain in > the ass to do any intricate hardware driver work... I'm guessing that Rust > is designed by 20-somethings who have never done a line of assembler in > their lives? That is curious. I wonder if these have been deferred for the time being in favor of using C as an extension (as Rust conforms to the C ABI)? -Craig From joshmbarney@REDACTED Wed Sep 13 04:57:18 2017 From: joshmbarney@REDACTED (Joshua Barney) Date: Tue, 12 Sep 2017 22:57:18 -0400 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <9D9DC4A2-64DB-4C47-BDBA-BC6B62980487@icloud.com> Rust supports inline assembly https://doc.rust-lang.org/1.8.0/book/inline-assembly.html Sent from my iPhone > On Sep 12, 2017, at 9:47 PM, Thom Cherryhomes wrote: > > For a language that purports to be a replacement for low-level systems programming, Rust not having bitfields or unions, would be a real pain in the ass to do any intricate hardware driver work... I'm guessing that Rust is designed by 20-somethings who have never done a line of assembler in their lives? > > sigh. > -Thom > >> On Tue, Sep 12, 2017 at 3:25 PM Dmytro Lytovchenko wrote: >> I tried to begin once. >> >> Here's the catch: >> Rust had no unions at the time of my experiments (added in 1.19 - July 2017) >> Rust has no bit fields (some libraries exist trying to alleviate this) >> Erlang process heap is created and maintained by abusing pointers and bits in them. Casting integers to pointers and back i believe is possible but will give you some pain. >> A really fast VM loop is done using C/C++ goto (void*) extension, in Rust you will probably have to use old good switch/case (slower) >> When you overcome these base problems, it should be possible to proceed carefully :) >> >> 2017-09-12 22:10 GMT+02:00 austin aigbe : >>> Hello, >>> >>> Any consideration for implementing the Erlang VM in Rust? >>> >>> Regards, >>> Austin >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From walter.weinmann@REDACTED Wed Sep 13 08:42:27 2017 From: walter.weinmann@REDACTED (Walter Weinmann) Date: Wed, 13 Sep 2017 08:42:27 +0200 Subject: [erlang-questions] Tutorial (Erlang / Python) Message-ID: What is the idea behind the examples in the tutorial? Should these be executable or are they just non-executable program code patterns. I want to use Apache Thrift to connect Erlang and Python, but I am struggling with connection problems from Erlang Client to Python Server (see THRIFT-4312 or Apache Thrift mailing list >). Since the community is very quiet here, I'm afraid this is a much too exotic combination for Apache Thrift. Any comments? Thanks and regards Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 13 09:07:04 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 16:07:04 +0900 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: References: Message-ID: <1906062.A71kjsWVgg@changa> On 2017?09?13? ??? 08:42:27 Walter Weinmann wrote: > What is the idea behind the examples in the tutorial? Should these be > executable or are they just non-executable program code patterns. I want to > use Apache Thrift to connect Erlang and Python, but I am struggling with > connection problems from Erlang Client to Python Server (see THRIFT-4312 > or Apache Thrift > mailing list > >). > Since the community is very quiet here, I'm afraid this is a much too > exotic combination for Apache Thrift. Any comments? I'm not familiar with Apache Thrift. Normally when I want Python and Erlang to talk to one another I use BERT over a socket (usually network socket, but anything works). But that is just old-fashioned socket programming. Looking at the docs for Thrift[1] it looks like there is supposed to be something like a Thrift compiler that builds an Erlang (and Python and whatever else) module for you that you should be invoking. I assume that is the `thrift_client` module you are invoking in your program. The error you are seeing there with {error, econnrefused} is a normal TCP network error, not a specific Thrift/Erlang problem -- so check that the service you're trying to connect to is running and the port you're trying to contact is open. It seems very odd that the return value of `thrift_client:call/2` is being assigned to `Client1,2,3`. Are you sure the return value is supposed to look like this? Once you get the network problem sorted out, give that part a look -- I imagine that's actually supposed to always match the original Client (so that maybe you could open several connections at once, and match on them in a larger `receive` or `case` clause. -Craig [1] It looks like your formatting for Jira got sort of crazy at some point, so its a little hard to tell how things were supposed to be formatted. Pasting your example in a paste bin or here in the mailing list could be helpful. From walter.weinmann@REDACTED Wed Sep 13 09:29:45 2017 From: walter.weinmann@REDACTED (Walter Weinmann) Date: Wed, 13 Sep 2017 09:29:45 +0200 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: <1906062.A71kjsWVgg@changa> References: <1906062.A71kjsWVgg@changa> Message-ID: Hi Craig, thanks for the answer. I have a repository in Github with the complete test environment based on the Erlang and Python examples from the Thrift tutorial. Of course, the corresponding program parts were also generated by the Thrift compiler. The Python server is also ok and can be reached via the Python client. The Erlang server can be accessed via both clients (Erlang and Python). Only the connection from the Erlang Client to the Python server is not established. Of course, it's the kind of connection I need first. Apache Thrift sounds really good, as it theoretically connects a lot of platforms. On 13 September 2017 at 09:07, zxq9 wrote: > On 2017?09?13? ??? 08:42:27 Walter Weinmann wrote: > > What is the idea behind the examples in the tutorial? Should these be > > executable or are they just non-executable program code patterns. I want > to > > use Apache Thrift to connect Erlang and Python, but I am struggling with > > connection problems from Erlang Client to Python Server (see THRIFT-4312 > > or Apache Thrift > > mailing list > > >) > . > > Since the community is very quiet here, I'm afraid this is a much too > > exotic combination for Apache Thrift. Any comments? > > I'm not familiar with Apache Thrift. Normally when I want Python and > Erlang to talk to one another I use BERT over a socket (usually network > socket, but anything works). But that is just old-fashioned socket > programming. > > Looking at the docs for Thrift[1] it looks like there is supposed to be > something like a Thrift compiler that builds an Erlang (and Python and > whatever else) module for you that you should be invoking. I assume that is > the `thrift_client` module you are invoking in your program. > > The error you are seeing there with {error, econnrefused} is a normal TCP > network error, not a specific Thrift/Erlang problem -- so check that the > service you're trying to connect to is running and the port you're trying > to contact is open. > > It seems very odd that the return value of `thrift_client:call/2` is being > assigned to `Client1,2,3`. Are you sure the return value is supposed to > look like this? Once you get the network problem sorted out, give that part > a look -- I imagine that's actually supposed to always match the original > Client (so that maybe you could open several connections at once, and match > on them in a larger `receive` or `case` clause. > > -Craig > > [1] It looks like your formatting for Jira got sort of crazy at some > point, so its a little hard to tell how things were supposed to be > formatted. Pasting your example in a paste bin or here in the mailing list > could be helpful. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 13 09:58:07 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 16:58:07 +0900 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: References: <1906062.A71kjsWVgg@changa> Message-ID: <3604217.GnLJpqhNQz@changa> On 2017?09?13? ??? 09:29:45 you wrote: > Hi Craig, thanks for the answer. > > I have a repository in Github > with the > complete test environment based on the Erlang and Python examples from the > Thrift tutorial. Of course, the corresponding program parts were also > generated by the Thrift compiler. The Python server is also ok and can be > reached via the Python client. The Erlang server can be accessed via both > clients (Erlang and Python). Only the connection from the Erlang Client to > the Python server is not established. Of course, it's the kind of > connection I need first. > > Apache Thrift sounds really good, as it theoretically connects a lot of > platforms. If the Python client can talk to the Python service over the network without any problem, the perhaps the generated Erlang code has a runtime dependency on inets or something similar. Try something like this: t() -> {ok, Client} = connect(9090), % ... whatever else you were going to do. connect(Port) -> ok = application:ensure_started(inets), thrift_client_util:new("localhost", Port, calculator_thrift, []). Also, each time you receive a response from the call/3 function, check if the received `Client1,2,3` values are actually the same as the original `Client` value returned by `thrift_client_util:connect/4`. Also check if there are any additional start or init functions that are supposed to be called in thrift_client or thrift_client_util to get things set up (some init functions may take care of things like starting dependencies). -Craig From walter.weinmann@REDACTED Wed Sep 13 10:49:42 2017 From: walter.weinmann@REDACTED (Walter Weinmann) Date: Wed, 13 Sep 2017 10:49:42 +0200 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: <3604217.GnLJpqhNQz@changa> References: <1906062.A71kjsWVgg@changa> <3604217.GnLJpqhNQz@changa> Message-ID: No problem with: *ok = application:ensure_started(inets),* but still: *** exception error: no match of right hand side value {error,econnrefused}* * in function client:t/0 (d:/SoftDevelopment/Projects/erlang_thrift_tutorial* *_idea/erlang_thrift_tutorial/_build/default/lib/tutorial/src/client.erl, line 43* with: {ok, Client0} = connect(Port), respectively: connect(Port) -> ok = application:ensure_started(inets), io:format("inets~n", []), thrift_client_util:new("localhost", Port, calculator_thrift, []). Walter On 13 September 2017 at 09:58, zxq9 wrote: > On 2017?09?13? ??? 09:29:45 you wrote: > > Hi Craig, thanks for the answer. > > > > I have a repository in Github > > with the > > complete test environment based on the Erlang and Python examples from > the > > Thrift tutorial. Of course, the corresponding program parts were also > > generated by the Thrift compiler. The Python server is also ok and can be > > reached via the Python client. The Erlang server can be accessed via both > > clients (Erlang and Python). Only the connection from the Erlang Client > to > > the Python server is not established. Of course, it's the kind of > > connection I need first. > > > > Apache Thrift sounds really good, as it theoretically connects a lot of > > platforms. > > If the Python client can talk to the Python service over the network > without any problem, the perhaps the generated Erlang code has a runtime > dependency on inets or something similar. > > Try something like this: > > t() -> > {ok, Client} = connect(9090), > % ... whatever else you were going to do. > > > connect(Port) -> > ok = application:ensure_started(inets), > thrift_client_util:new("localhost", Port, calculator_thrift, []). > > Also, each time you receive a response from the call/3 function, check if > the received `Client1,2,3` values are actually the same as the original > `Client` value returned by `thrift_client_util:connect/4`. Also check if > there are any additional start or init functions that are supposed to be > called in thrift_client or thrift_client_util to get things set up (some > init functions may take care of things like starting dependencies). > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 13 10:54:32 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 17:54:32 +0900 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: References: <3604217.GnLJpqhNQz@changa> Message-ID: <2210220.sbZ9P2UiAS@changa> On 2017?09?13? ??? 10:49:42 Walter Weinmann wrote: > {error,econnrefused} What happens if you do (in your shell): telnet localhost 9090 And in an Erlang shell gen_tcp:connect("localhost", 9090). ? From zxq9@REDACTED Wed Sep 13 11:21:17 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 18:21:17 +0900 Subject: [erlang-questions] Tutorial (Erlang / Python) In-Reply-To: <2210220.sbZ9P2UiAS@changa> References: <2210220.sbZ9P2UiAS@changa> Message-ID: <2524477.ffpkqUbc2B@changa> On 2017?09?13? ??? 17:54:32 zxq9 wrote: > > gen_tcp:connect("localhost", 9090). HA HA HA! Sorry, spectators, that was not supposed to be a practical joke... gen_tcp:connect("localhost", 9090, []). Walter was kind and wrote me off-list to let me know that this didn't work, not realizing that my mistakes tend to come right out in public most of the time in any case. Dhoh! -Craig From jesper.louis.andersen@REDACTED Wed Sep 13 13:11:36 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Sep 2017 11:11:36 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: On Tue, Sep 12, 2017 at 10:10 PM austin aigbe wrote: > Hello, > > Any consideration for implementing the Erlang VM in Rust? > > I think the problem is roughly the same as with any other language: A lot of good effort has been sunk into the current VM in C and this makes a switch harder. Rust has ABI compatibility with C in some ways, which makes interoperability simpler, so one could imagine taking some of the more security-oriented parts of the VM and rewriting those in Rust or some other language of the more restrictive kind. If you want to rewrite all of it, a good approach is to start by making changes to the current VM which makes it smaller: * Move more work out into NIFs. Reimplement those in Rust * Coalesce more work on Dirty Schedulers * Implement Native Processes, so you can start running Rust-native-processes * Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot of the BIF optimized functions into pure Erlang. A smaller core is way easier to replace. Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamhedd@REDACTED Wed Sep 13 13:20:05 2017 From: jamhedd@REDACTED (Roman Galeev) Date: Wed, 13 Sep 2017 13:20:05 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: > Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P Do you think concurrent OCaml has advantages over Erlang? And if yes, what are they, in your opinion? On Wed, Sep 13, 2017 at 1:11 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Sep 12, 2017 at 10:10 PM austin aigbe wrote: > >> Hello, >> >> Any consideration for implementing the Erlang VM in Rust? >> >> > I think the problem is roughly the same as with any other language: A lot > of good effort has been sunk into the current VM in C and this makes a > switch harder. Rust has ABI compatibility with C in some ways, which makes > interoperability simpler, so one could imagine taking some of the more > security-oriented parts of the VM and rewriting those in Rust or some other > language of the more restrictive kind. > > If you want to rewrite all of it, a good approach is to start by making > changes to the current VM which makes it smaller: > > * Move more work out into NIFs. Reimplement those in Rust > * Coalesce more work on Dirty Schedulers > * Implement Native Processes, so you can start running > Rust-native-processes > * Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot > of the BIF optimized functions into pure Erlang. > > A smaller core is way easier to replace. > > Personally, I'd just throw some more time after concurrent OCaml, and then > write a translator from Erlang to OCaml :P > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- With best regards, Roman Galeev, +420 702 817 968 -------------- next part -------------- An HTML attachment was scrubbed... URL: From palani@REDACTED Wed Sep 13 13:31:37 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Wed, 13 Sep 2017 17:01:37 +0530 Subject: [erlang-questions] How to insert null value into map Message-ID: Hi Guys, I want to get data from mysql and append them into map. But some row had null values. Hence i cant able to convert the null value into map. Its there any way to convert null value into map. Thanks in Advance -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Sep 13 13:43:16 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Sep 2017 11:43:16 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: On Wed, Sep 13, 2017 at 1:20 PM Roman Galeev wrote: > > Personally, I'd just throw some more time after concurrent OCaml, and > then write a translator from Erlang to OCaml :P > > Do you think concurrent OCaml has advantages over Erlang? And if yes, what > are they, in your opinion? > You are making tradeoffs whenever you choose another language. * Semantically, OCaml is far superior to Erlang in any conceivable way. * OCaml has a powerful type system, which Erlang doesn't. * The native code generator in OCaml easily surpasses the bytecode interpreter in Erlang. On the other hand: * OCaml is currently not parallel in any way. There has been a long effort in fixing this, but even *if* it gets released, we are looking at years of maturing needed before we can hope for robust operation. * OCaml currently uses something like Lwt and/or Async for concurrent work. This is Node.js in disguise. I don't believe in indirect concurrency codings in general, far preferring Erlang or Go's direct approaches (in which you favor preemption over cooperation in multitasking). * OCaml is unlikely to ever support hot code upgrade. * OCaml currently has no dTrace-style production ad-hoc tracing facilities where Erlang has it naturally. * OCaml programs are easy to make robust (cope with unknown input), but harder to make resilient (coping gracefully with internal failure). So proviso OCaml gets a good parallel story, and we can implement the important parts of Erlang on top of that, then I think there are programs which can be written in OCaml with advantage. In particular those which can afford to be restarted once in a while. This is true for many modern systems in which you are deploy clusters of nodes() and provisioning them this way. The prime candidate programs are those which require the efficiency the OCaml native code compiler provides and where you can't just run the OCaml program behind an Erlang Port easily. The *current* state of the art: use OCaml whenever you have a problem requiring the combination of functional abstraction and efficiency[0]. If you don't require the abstraction levels OCaml provide, go for C, C++, Rust, Go, etc. But chances are that the added productivity of OCaml will get you a far better and faster program when you are working with a limited time frame, since you can try more solutions, quicker. If your problem doesn't require efficient use of the CPU core, don't underestimate how productive Erlang is. Since everything are just Erlang terms, you can often write a good program in a fraction of the time it requires to come up with a good type model in something like OCaml. All tooling supports Erlang terms. This is a powerful abstraction which can be put to good use. And you get resilience on top. I should really blog about how unfairly effective Erlang is at being productive. Efficiency is far from the only important factor in software development. [0] Haskell is another excellent candidate here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Wed Sep 13 13:47:54 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Wed, 13 Sep 2017 13:47:54 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: I am a strong proponent for rewriting parts of BEAM VM in C++, and I've even made a list of points to consider and benefits it would bring: http://beam-wisdoms.clau.se/en/latest/otp-cpp-ramblings.html I was personally ready to sink a lot of my time into this both paid (we have our ways to get paid for helping OTP team) and unpaid. But i never started because OTP team wasn't happy with this idea. Main problem with all these rewritings, is that either you fork and take over the fork completely, i.e. maintain version and all bugfixes and all new developments, or you listen what OTP team has got to say. And so far they say no to any major rewrite. 2017-09-13 13:11 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > On Tue, Sep 12, 2017 at 10:10 PM austin aigbe wrote: > >> Hello, >> >> Any consideration for implementing the Erlang VM in Rust? >> >> > I think the problem is roughly the same as with any other language: A lot > of good effort has been sunk into the current VM in C and this makes a > switch harder. Rust has ABI compatibility with C in some ways, which makes > interoperability simpler, so one could imagine taking some of the more > security-oriented parts of the VM and rewriting those in Rust or some other > language of the more restrictive kind. > > If you want to rewrite all of it, a good approach is to start by making > changes to the current VM which makes it smaller: > > * Move more work out into NIFs. Reimplement those in Rust > * Coalesce more work on Dirty Schedulers > * Implement Native Processes, so you can start running > Rust-native-processes > * Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot > of the BIF optimized functions into pure Erlang. > > A smaller core is way easier to replace. > > Personally, I'd just throw some more time after concurrent OCaml, and then > write a translator from Erlang to OCaml :P > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 13 13:49:36 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 13 Sep 2017 20:49:36 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <14204021.BGHsmkh3C1@changa> On 2017?09?13? ??? 13:47:54 Dmytro Lytovchenko wrote: > I am a strong proponent for rewriting parts of BEAM VM in C++ As a serious thought, or as satire? From dmytro.lytovchenko@REDACTED Wed Sep 13 13:53:40 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Wed, 13 Sep 2017 13:53:40 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <14204021.BGHsmkh3C1@changa> References: <14204021.BGHsmkh3C1@changa> Message-ID: Absolutely serious. It is all listed in that article I linked, I wrote it. 2017-09-13 13:49 GMT+02:00 zxq9 : > On 2017?09?13? ??? 13:47:54 Dmytro Lytovchenko wrote: > > I am a strong proponent for rewriting parts of BEAM VM in C++ > > As a serious thought, or as satire? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Wed Sep 13 14:05:13 2017 From: hugo@REDACTED (Hugo Mills) Date: Wed, 13 Sep 2017 12:05:13 +0000 Subject: [erlang-questions] How to insert null value into map In-Reply-To: References: Message-ID: <20170913120513.GA7067@carfax.org.uk> On Wed, Sep 13, 2017 at 05:01:37PM +0530, Palanikumar Gopalakrishnan wrote: > Hi Guys, > I want to get data from mysql and append them into map. But > some row had null values. Hence i cant able to convert the null value into > map. Its there any way to convert null value into map. Why can't you put nulls into a map? This works, for example: 2> #{null => null}. #{null => null} so we can clearly put some null atom in both key and value positions in a map. If you could show an example row without a null, and what that turns into when you put it in a map, and then show how this fails when you do have a null, then it might be easier to answer your question. Hugo. -- Hugo Mills | Klytus! Are your men on the right pills? Maybe you hugo@REDACTED carfax.org.uk | should execute their trainer! http://carfax.org.uk/ | PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From n.oxyde@REDACTED Wed Sep 13 14:10:30 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 13 Sep 2017 14:10:30 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <3587013.qFDS1ZT4Xo@changa> References: <1505261133.582606.1104113280.714AB626@webmail.messagingengine.com> <3587013.qFDS1ZT4Xo@changa> Message-ID: <751D43E0-392B-4A2D-8C69-7FC74685E1EF@gmail.com> C++ would never avoid crappy things from happening, such as silly data races because someone used a static variable in the middle of nowhere[1]. Rust would. Rewriting something in C++ in the year of our lord of 2017 isn't the smartest move when there are better tools available. Well I guess it's good for job security, but there is Erlang itself for that already. As for the emulator loop which requires computed gotos to be implemented, it could instead be generated from a higher-level description of the opcodes directly to an LLVM IR module, which would be faster to compile because the CFG would be less silly written by hand. Using LLVM IR is also something the JIT project I don't remember the name wants to do anyway. As for what would Rust bring us (apart from a massive improvement of the memory safety of the whole VM), Rust allows people to be way more reckless when writing code, and that usually ends up with less runtime safety belts in the system [2]. I started writing a BEAM module loader in Rust (but I work on it only during holidays because I am actually busy at Mozilla working on things 10 years ahead of the rest of the world), and I think I found some issues in BEAM's code already. I will try to cook some BEAM compiled modules that weird out the VM in the near future. [1]: https://github.com/erlang/otp/pull/643 [2]: http://www.randomhacks.net/2014/09/19/rust-lifetimes-reckless-cxx/ > Le 13 sept. 2017 ? 03:30, zxq9 a ?crit : > > On 2017?09?12? ??? 17:05:33 Tristan Sloughter wrote: >> Maybe Tony 'the tiger' Ramine would come back to working on Erlang if it >> was in Rust! > > Hey! > I really like that guy. > > Almost a good enough reason on its own. But I'm being selfish. ;-) > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jamhedd@REDACTED Wed Sep 13 14:24:48 2017 From: jamhedd@REDACTED (Roman Galeev) Date: Wed, 13 Sep 2017 14:24:48 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: > don't underestimate how productive Erlang I don't. In Erlang, I can make a system to work relatively quickly, and with all these features like resilience and concurrency, even with some distribution, and I like it. On the other hand, a powerful type system can ease life a lot too. Kind of a tradeoff you've mentioned I guess. Anyway, thank you for the detailed response. On Wed, Sep 13, 2017 at 1:43 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Wed, Sep 13, 2017 at 1:20 PM Roman Galeev wrote: > >> > Personally, I'd just throw some more time after concurrent OCaml, and >> then write a translator from Erlang to OCaml :P >> >> Do you think concurrent OCaml has advantages over Erlang? And if yes, >> what are they, in your opinion? >> > > You are making tradeoffs whenever you choose another language. > > * Semantically, OCaml is far superior to Erlang in any conceivable way. > * OCaml has a powerful type system, which Erlang doesn't. > * The native code generator in OCaml easily surpasses the bytecode > interpreter in Erlang. > > On the other hand: > > * OCaml is currently not parallel in any way. There has been a long effort > in fixing this, but even *if* it gets released, we are looking at years of > maturing needed before we can hope for robust operation. > * OCaml currently uses something like Lwt and/or Async for concurrent > work. This is Node.js in disguise. I don't believe in indirect concurrency > codings in general, far preferring Erlang or Go's direct approaches (in > which you favor preemption over cooperation in multitasking). > * OCaml is unlikely to ever support hot code upgrade. > * OCaml currently has no dTrace-style production ad-hoc tracing facilities > where Erlang has it naturally. > * OCaml programs are easy to make robust (cope with unknown input), but > harder to make resilient (coping gracefully with internal failure). > > So proviso OCaml gets a good parallel story, and we can implement the > important parts of Erlang on top of that, then I think there are programs > which can be written in OCaml with advantage. In particular those which can > afford to be restarted once in a while. This is true for many modern > systems in which you are deploy clusters of nodes() and provisioning them > this way. The prime candidate programs are those which require the > efficiency the OCaml native code compiler provides and where you can't just > run the OCaml program behind an Erlang Port easily. > > The *current* state of the art: use OCaml whenever you have a problem > requiring the combination of functional abstraction and efficiency[0]. If > you don't require the abstraction levels OCaml provide, go for C, C++, > Rust, Go, etc. But chances are that the added productivity of OCaml will > get you a far better and faster program when you are working with a limited > time frame, since you can try more solutions, quicker. > > If your problem doesn't require efficient use of the CPU core, don't > underestimate how productive Erlang is. Since everything are just Erlang > terms, you can often write a good program in a fraction of the time it > requires to come up with a good type model in something like OCaml. All > tooling supports Erlang terms. This is a powerful abstraction which can be > put to good use. And you get resilience on top. I should really blog about > how unfairly effective Erlang is at being productive. Efficiency is far > from the only important factor in software development. > > > [0] Haskell is another excellent candidate here. > -- With best regards, Roman Galeev, +420 702 817 968 -------------- next part -------------- An HTML attachment was scrubbed... URL: From palani@REDACTED Wed Sep 13 14:50:38 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Wed, 13 Sep 2017 18:20:38 +0530 Subject: [erlang-questions] How to insert null value into map In-Reply-To: <20170913120513.GA7067@carfax.org.uk> References: <20170913120513.GA7067@carfax.org.uk> Message-ID: Hi Hugo, Null value showed in sql +---------------------+ | NULL | | NULL | +---------------------+ When we applied it with map like below, It returns the following error *(cowboy_ws_server@REDACTED)27> maps:get("A", NULL, Some).* 1: variable 'NULL' is unbound(cowboy_ws_server@REDACTED)28> * On 13 September 2017 at 17:35, Hugo Mills wrote: > On Wed, Sep 13, 2017 at 05:01:37PM +0530, Palanikumar Gopalakrishnan wrote: > > Hi Guys, > > I want to get data from mysql and append them into map. > But > > some row had null values. Hence i cant able to convert the null value > into > > map. Its there any way to convert null value into map. > > Why can't you put nulls into a map? This works, for example: > > 2> #{null => null}. > #{null => null} > > so we can clearly put some null atom in both key and value positions > in a map. > > If you could show an example row without a null, and what that > turns into when you put it in a map, and then show how this fails when > you do have a null, then it might be easier to answer your question. > > Hugo. > > -- > Hugo Mills | Klytus! Are your men on the right pills? Maybe you > hugo@REDACTED carfax.org.uk | should execute their trainer! > http://carfax.org.uk/ | > PGP: E2AB1DE4 | Ming the Merciless, Flash > Gordon > -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Wed Sep 13 15:01:35 2017 From: hugo@REDACTED (Hugo Mills) Date: Wed, 13 Sep 2017 13:01:35 +0000 Subject: [erlang-questions] How to insert null value into map In-Reply-To: References: <20170913120513.GA7067@carfax.org.uk> Message-ID: <20170913130135.GB7067@carfax.org.uk> On Wed, Sep 13, 2017 at 06:20:38PM +0530, Palanikumar Gopalakrishnan wrote: > Hi Hugo, > > Null value showed in sql > > +---------------------+ > | NULL | > | NULL | > +---------------------+ This is the printed human-readable representation of the output from some tool which isn't itself erlang or the SQL library you're using. So yes, you have null values in the result, but it doesn't tell you how they are represented in erlang code. > When we applied it with map like below, It returns the following error > > *(cowboy_ws_server@REDACTED)27> maps:get("A", NULL, Some).* 1: variable > 'NULL' is unbound(cowboy_ws_server@REDACTED)28> * That's because NULL is a variable -- in fact, the error message tells you *exactly* that. In erlang code, variables start with a capital letter and are unquoted, so this is clearly not going to work, and the compiler is telling you this. Also, you have the order of the parameters for maps:get/3 wrong. Check the documentation for what order they should go in. Finally, what value should you be using to find the NULL? Well, it depends on how it's represented by the library you're using. It should be in the documentation for the library. If it isn't, then you could just look at the actual data in your map (print the map with io:format("~p~n", [Some])). You should then be able to see how a null is represented, and you can use that representation in your maps:get call. Hugo. > On 13 September 2017 at 17:35, Hugo Mills wrote: > > > On Wed, Sep 13, 2017 at 05:01:37PM +0530, Palanikumar Gopalakrishnan wrote: > > > Hi Guys, > > > I want to get data from mysql and append them into map. > > But > > > some row had null values. Hence i cant able to convert the null value > > into > > > map. Its there any way to convert null value into map. > > > > Why can't you put nulls into a map? This works, for example: > > > > 2> #{null => null}. > > #{null => null} > > > > so we can clearly put some null atom in both key and value positions > > in a map. > > > > If you could show an example row without a null, and what that > > turns into when you put it in a map, and then show how this fails when > > you do have a null, then it might be easier to answer your question. > > > > Hugo. > > -- Hugo Mills | Klytus! Are your men on the right pills? Maybe you hugo@REDACTED carfax.org.uk | should execute their trainer! http://carfax.org.uk/ | PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From ameretat.reith@REDACTED Wed Sep 13 15:26:56 2017 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Wed, 13 Sep 2017 17:56:56 +0430 Subject: [erlang-questions] Improved IO polling in erts In-Reply-To: References: Message-ID: This is amazing! Here is result of simple UDP echo server. It spans 1000 processes for client and servers [1]. I ran it on Xeon E5-2620 with eight physical cores. On baseline 20: 16:22:34.977 [notice] 1505303554: recv_pkts: 1793113 recv_size: 8965565 sent_pkts: 1793113 sent_size: 2639462336 recv: 7.172452 Mbit/s send: 2111.569869 Mbit/s 16:22:44.978 [notice] 1505303564: recv_pkts: 1801476 recv_size: 9007380 sent_pkts: 1801476 sent_size: 2651772672 recv: 7.205904 Mbit/s send: 2121.418138 Mbit/s 16:22:54.979 [notice] 1505303574: recv_pkts: 1799551 recv_size: 8997755 sent_pkts: 1799551 sent_size: 2648939072 recv: 7.198204 Mbit/s send: 2119.151258 Mbit/s On poll-threads branch I get best results by two polling threads: 16:58:14.696 [notice] 1505305694: recv_pkts: 2835781 recv_size: 14178905 sent_pkts: 2835781 sent_size: 4174269632 <%28417%29%20426-9632> recv: 11.343124 Mbit/s send: 3339.415706 Mbit/s 16:58:24.698 [notice] 1505305704: recv_pkts: 2836201 recv_size: 14181005 sent_pkts: 2836201 sent_size: 4174887872 <%28417%29%20488-7872> recv: 11.344804 Mbit/s send: 3339.910298 Mbit/s 16:58:34.699 [notice] 1505305714: recv_pkts: 2833905 recv_size: 14169525 sent_pkts: 2833905 sent_size: 4171508160 recv: 11.335620 Mbit/s send: 3337.206528 Mbit/s A %50 throughput increase for this small system! I expect more enhancements in bigger systems; those involving more message passings or using encryption. 1: https://github.com/reith/udpstress -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Wed Sep 13 15:02:05 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Wed, 13 Sep 2017 10:02:05 -0300 Subject: [erlang-questions] How to insert null value into map In-Reply-To: References: <20170913120513.GA7067@carfax.org.uk> Message-ID: <81E016A2-69ED-429C-B4C0-2ECE308F7DFD@gmail.com> Palanikumar: On one hand, you should show us what?s in your Some variable. On the other hand, you don?t seem to be using maps:get/3 as it?s intended to be used (check the docs ). Even more so, you should know that in Erlang variables start with an uppercase letter, that?s why in your example NULL is a variable name (not a value). See how Erlang VM tells you that the variable ?NULL? is unbound. Hope this helps, Cheers! Brujo Benavides > On Sep 13, 2017, at 09:50, Palanikumar Gopalakrishnan wrote: > > Hi Hugo, > > Null value showed in sql > > +---------------------+ > | NULL | > | NULL | > +---------------------+ > > When we applied it with map like below, It returns the following error > (cowboy_ws_server@REDACTED)27> maps:get("A", NULL, Some). > * 1: variable 'NULL' is unbound > (cowboy_ws_server@REDACTED)28> > > > > > > > > On 13 September 2017 at 17:35, Hugo Mills > wrote: > On Wed, Sep 13, 2017 at 05:01:37PM +0530, Palanikumar Gopalakrishnan wrote: > > Hi Guys, > > I want to get data from mysql and append them into map. But > > some row had null values. Hence i cant able to convert the null value into > > map. Its there any way to convert null value into map. > > Why can't you put nulls into a map? This works, for example: > > 2> #{null => null}. > #{null => null} > > so we can clearly put some null atom in both key and value positions > in a map. > > If you could show an example row without a null, and what that > turns into when you put it in a map, and then show how this fails when > you do have a null, then it might be easier to answer your question. > > Hugo. > > -- > Hugo Mills | Klytus! Are your men on the right pills? Maybe you > hugo@REDACTED carfax.org.uk | should execute their trainer! > http://carfax.org.uk/ | > PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon > > > > -- > > Warm Regards, > > Palanikumar Gopalakrishnan > Developer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Wed Sep 13 17:11:12 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Wed, 13 Sep 2017 17:11:12 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <9714375.VumhWqLY8o@changa> References: <9714375.VumhWqLY8o@changa> Message-ID: There are some really big point to consider for not doing it too: For once Rust is in it?s infancy, the whole language still changes, code that works today might not work tomorrow and libraries often depend on nightly builds and new features. This goes 100% against the erlang philosophy if a long time maintained system. A large project like erlang will amplify this problem even more then small projects do. Rust supports a lot less platforms then C, erlang runs perfectly fine on nearly everything, rust won?t even run on FreeBSD 12 at this point. It would turn erlang on yet another ?We only care about Linux? project which would make me vomit. (Depending on who you ask making me vomit might not be a bad thing ?) That said rust is cool for NIFs and small projects, but the requirements for a platform like erlang are vastly different. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From daniel.goertzen@REDACTED Wed Sep 13 21:46:08 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 13 Sep 2017 19:46:08 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <6197571.1QanT7AGF8@changa> References: <6197571.1QanT7AGF8@changa> Message-ID: - Rust got C-like unions as of 1.19 (20 July 2017). This is primarily for interfacing with C code; in native Rust you almost always opt for enums (aka tagged unions, sum types). - Rust does not have bitfields but it does have the same bit mashing operators as C (& | ^ ! << >>). C bitfields have different memory layouts on different platforms which limits their usefulness. All the microcontroller C code I've ever seen has opted for explicit bit mashing instead of bitfields. - There is an OS written in Rust, https://www.redox-os.org/ . The drivers are all in Rust. On Tue, Sep 12, 2017 at 9:27 PM zxq9 wrote: > On 2017?09?13? ??? 01:47:08 Thom Cherryhomes wrote: > > For a language that purports to be a replacement for low-level systems > > programming, Rust not having bitfields or unions, would be a real pain in > > the ass to do any intricate hardware driver work... I'm guessing that > Rust > > is designed by 20-somethings who have never done a line of assembler in > > their lives? > > That is curious. I wonder if these have been deferred for the time being > in favor of using C as an extension (as Rust conforms to the C ABI)? > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Sep 13 22:42:39 2017 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 13 Sep 2017 13:42:39 -0700 Subject: [erlang-questions] [ANN] CloudI 1.7.2 Released! Message-ID: Download 1.7.2 from http://sourceforge.net/projects/cloudi/files/latest/download The details for this release are below: * Service configuration option request_timeout_adjustment was modified to use Erlang monotonic time so timeout adjustments are not impacted by any OS time changes (now all CloudI service request timeout handling is using monotonic time) * Added CloudI Service API logging_set configuration option log_time_offset to log when the Erlang VM updates its time-offset with the time-offset change logged in nanoseconds (the offset is logged exactly when logging timestamps are changed) * configure script now accepts environment variables to set release configuration values: CLOUDI_ARG_NODE_NAME_SHORT, CLOUDI_ARG_NODE_NAME_LONG, CLOUDI_ARG_NODE_COOKIE, CLOUDI_PID_FILE (can be used to avoid manual vm.args modifications) * Many deployment changes to make CloudI use robust: * The release script is safe for root use (external environment variables are blocked, functionality is locked down, manages a pid file correctly, etc.) * Execution now occurs within the logging directory so any crash dump output is put there automatically * cloudi_service_null is an Erlang service added to provide a simple way of consuming CloudI service requests (during testing and/or to handle invalid URL service names) * cloudi_service_request_rate was added to the CloudI release to provide an easy way of testing max throughput * cloudi_service_http_cowboy and cloudi_service_http_elli now use an update_delay argument to apply a change to timeout_async and/or timeout_sync performed with the CloudI Service API services_update * Bugs were fixed and other improvements were added (see the ChangeLog for more detail) CloudI is a "universal integrator" using an Erlang core to provide fault-tolerance with efficiency and scalability. The CloudI API provides a minimal interface to communicate among services so programming language agnostic and protocol agnostic integration can occur. CloudI currently integrates with the programming languages: C/C++, Elixir, Erlang, Go, Haskell, Java, JavaScript/node.js, OCaml, PHP, Perl, Python, and Ruby, Many reusable services are included that rely on the CloudI service bus. Please mention any problems, issues, or ideas! Thanks, Michael SHA256 CHECKSUMS cloudi-1.7.2.tar.bz2 (12963740 bytes) 987864e6e5a26ff7260a7d47e6a15085cdcef3fc88cc416662723e6c1398b5f0 cloudi-1.7.2.tar.gz (15810122 bytes) 5ffeddb4ddabc9611d02fe356022849d2cbb1d1557311651f3f5b42c37ba3d71 From ok@REDACTED Wed Sep 13 23:37:55 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Sep 2017 09:37:55 +1200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> On 13/09/17 11:43 PM, Jesper Louis Andersen wrote: [OCaml is better than Erlang in many ways] [OCaml is not parallel] What's your opinion of F#? It seems to be the language of choice for people who liked Caml but need to take advantage of multiple cores. I haven't done any benchmarking; I doubt that it could match OCaml in raw speed. From ok@REDACTED Wed Sep 13 23:52:34 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Sep 2017 09:52:34 +1200 Subject: [erlang-questions] How to insert null value into map In-Reply-To: References: <20170913120513.GA7067@carfax.org.uk> Message-ID: <59614be3-d16a-3bfe-c222-942ed3637819@cs.otago.ac.nz> On 14/09/17 12:50 AM, Palanikumar Gopalakrishnan wrote: > Hi Hugo, > > Null value showed in sql > > +---------------------+ > | NULL | > | NULL | > +---------------------+ > > When we applied it with map like below, It returns the following error > *(cowboy_ws_server@REDACTED)27> maps:get("A", NULL, Some). > * 1: variable 'NULL' is unbound > (cowboy_ws_server@REDACTED)28> That is because NULL is indeed an Erlang *variable* name. 'NULL', null, "NULL", <<"NULL">>, [], {} and a whole lot of other alternatives are not variables. Now maps:get(Key, Map, Default) wants a Map as its second argument. Presumably your data base interface returns a row as a map, so you should be looking at something like #{"A" => null, "B" => null} for a row with columns A and B, both having null values. But you need to check the documentation for your data base interface. PS: SQL keywords are not case sensitive: NULL = null = Null = nUlL. My own SQL style puts keywords in lower case and other identifiers in Mixed_Case_Well_Separated (just as in the Ada Quality and Style Guidelines for Ada). Point is, there isn't the slightest reason in any programming language other than SQL why SQL's "null" should be represented as NULL. Also, quoting the MySQL manual, "Column, index, stored routine, and event names are not case sensitive on any platform, nor are column aliases." So are you really sure that "A" should be "A" and not "a"? From ok@REDACTED Thu Sep 14 00:01:48 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Sep 2017 10:01:48 +1200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <6197571.1QanT7AGF8@changa> Message-ID: <640f3bea-b132-5f4a-8366-ad4e6f94a2cd@cs.otago.ac.nz> On 14/09/17 7:46 AM, Daniel Goertzen wrote: > - Rust does not have bitfields but it does have the same bit mashing > operators as C (& | ^ ! << >>). C bitfields have different memory > layouts on different platforms which limits their usefulness. All the > microcontroller C code I've ever seen has opted for explicit bit mashing > instead of bitfields. Back in the 1980s, I learned very quickly that bitfield manipulation in C was (a) more portable and (b) faster if I used masking and shifting than if I used bitfield syntax. Compilers have got a lot better since then, but (a) is still an issue. However, when you want bitfields in C, it's usually to interface with something that is platform-specific anyway. For example, code to get at the fields of an IEEE floating- point number. From n.oxyde@REDACTED Thu Sep 14 00:25:01 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 14 Sep 2017 00:25:01 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <9714375.VumhWqLY8o@changa> Message-ID: <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> Just don't use unstable features. You can do a damn lot without them, like the whole style system of Servo that is landing in Firefox 57. https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-engine-quantum-css-aka-stylo/ That may not be the correct venue to discuss this, of course. :) The argument about embedded platforms in 100% correct, though. Rust isn't ready yet on that front. > Le 13 sept. 2017 ? 17:11, Heinz N. Gies a ?crit : > > For once Rust is in it?s infancy, the whole language still changes, code that works today might not work tomorrow and libraries often depend on nightly builds and new features. From t@REDACTED Thu Sep 14 00:54:04 2017 From: t@REDACTED (Tristan Sloughter) Date: Wed, 13 Sep 2017 15:54:04 -0700 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <1505343244.3160245.1105355912.55BAE207@webmail.messagingengine.com> Alternatively, work on Alpaca (https://github.com/alpaca-lang/alpaca) and improvements to BEAM :) -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Wed, Sep 13, 2017, at 04:43 AM, Jesper Louis Andersen wrote: > On Wed, Sep 13, 2017 at 1:20 PM Roman Galeev > wrote:>> > Personally, I'd just throw some more time after concurrent OCaml, >> > and then write a translator from Erlang to OCaml :P>> >> Do you think concurrent OCaml has advantages over Erlang? And if yes, >> what are they, in your opinion?> > You are making tradeoffs whenever you choose another language. > > * Semantically, OCaml is far superior to Erlang in any > conceivable way.> * OCaml has a powerful type system, which Erlang doesn't. > * The native code generator in OCaml easily surpasses the bytecode > interpreter in Erlang.> > On the other hand: > > * OCaml is currently not parallel in any way. There has been a long > effort in fixing this, but even *if* it gets released, we are > looking at years of maturing needed before we can hope for robust > operation.> * OCaml currently uses something like Lwt and/or Async for concurrent > work. This is Node.js in disguise. I don't believe in indirect > concurrency codings in general, far preferring Erlang or Go's direct > approaches (in which you favor preemption over cooperation in > multitasking).> * OCaml is unlikely to ever support hot code upgrade. > * OCaml currently has no dTrace-style production ad-hoc tracing > facilities where Erlang has it naturally.> * OCaml programs are easy to make robust (cope with unknown input), > but harder to make resilient (coping gracefully with internal > failure).> > So proviso OCaml gets a good parallel story, and we can implement the > important parts of Erlang on top of that, then I think there are > programs which can be written in OCaml with advantage. In particular > those which can afford to be restarted once in a while. This is true > for many modern systems in which you are deploy clusters of nodes() > and provisioning them this way. The prime candidate programs are > those which require the efficiency the OCaml native code compiler > provides and where you can't just run the OCaml program behind an > Erlang Port easily.> > The *current* state of the art: use OCaml whenever you have a problem > requiring the combination of functional abstraction and efficiency[0]. > If you don't require the abstraction levels OCaml provide, go for C, > C++, Rust, Go, etc. But chances are that the added productivity of > OCaml will get you a far better and faster program when you are > working with a limited time frame, since you can try more solutions, > quicker.> > If your problem doesn't require efficient use of the CPU core, don't > underestimate how productive Erlang is. Since everything are just > Erlang terms, you can often write a good program in a fraction of the > time it requires to come up with a good type model in something like > OCaml. All tooling supports Erlang terms. This is a powerful > abstraction which can be put to good use. And you get resilience on > top. I should really blog about how unfairly effective Erlang is at > being productive. Efficiency is far from the only important factor in > software development.> > > [0] Haskell is another excellent candidate here. > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Thu Sep 14 08:15:34 2017 From: zkessin@REDACTED (Zachary Kessin) Date: Thu, 14 Sep 2017 09:15:34 +0300 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <1505343244.3160245.1105355912.55BAE207@webmail.messagingengine.com> References: <1505343244.3160245.1105355912.55BAE207@webmail.messagingengine.com> Message-ID: It seems to me an Idea that is interesting in theory but would involve a *LOT* of work for an unclear benefit. Zach ? On Thu, Sep 14, 2017 at 1:54 AM, Tristan Sloughter wrote: > Alternatively, work on Alpaca (https://github.com/alpaca-lang/alpaca) and > improvements to BEAM :) > > -- > Tristan Sloughter > "I am not a crackpot" - Abe Simpson > t@REDACTED > > > On Wed, Sep 13, 2017, at 04:43 AM, Jesper Louis Andersen wrote: > > On Wed, Sep 13, 2017 at 1:20 PM Roman Galeev wrote: > > > Personally, I'd just throw some more time after concurrent OCaml, and > then write a translator from Erlang to OCaml :P > > Do you think concurrent OCaml has advantages over Erlang? And if yes, what > are they, in your opinion? > > > You are making tradeoffs whenever you choose another language. > > * Semantically, OCaml is far superior to Erlang in any conceivable way. > * OCaml has a powerful type system, which Erlang doesn't. > * The native code generator in OCaml easily surpasses the bytecode > interpreter in Erlang. > > On the other hand: > > * OCaml is currently not parallel in any way. There has been a long effort > in fixing this, but even *if* it gets released, we are looking at years of > maturing needed before we can hope for robust operation. > * OCaml currently uses something like Lwt and/or Async for concurrent > work. This is Node.js in disguise. I don't believe in indirect concurrency > codings in general, far preferring Erlang or Go's direct approaches (in > which you favor preemption over cooperation in multitasking). > * OCaml is unlikely to ever support hot code upgrade. > * OCaml currently has no dTrace-style production ad-hoc tracing facilities > where Erlang has it naturally. > * OCaml programs are easy to make robust (cope with unknown input), but > harder to make resilient (coping gracefully with internal failure). > > So proviso OCaml gets a good parallel story, and we can implement the > important parts of Erlang on top of that, then I think there are programs > which can be written in OCaml with advantage. In particular those which can > afford to be restarted once in a while. This is true for many modern > systems in which you are deploy clusters of nodes() and provisioning them > this way. The prime candidate programs are those which require the > efficiency the OCaml native code compiler provides and where you can't just > run the OCaml program behind an Erlang Port easily. > > The *current* state of the art: use OCaml whenever you have a problem > requiring the combination of functional abstraction and efficiency[0]. If > you don't require the abstraction levels OCaml provide, go for C, C++, > Rust, Go, etc. But chances are that the added productivity of OCaml will > get you a far better and faster program when you are working with a limited > time frame, since you can try more solutions, quicker. > > If your problem doesn't require efficient use of the CPU core, don't > underestimate how productive Erlang is. Since everything are just Erlang > terms, you can often write a good program in a fraction of the time it > requires to come up with a good type model in something like OCaml. All > tooling supports Erlang terms. This is a powerful abstraction which can be > put to good use. And you get resilience on top. I should really blog about > how unfairly effective Erlang is at being productive. Efficiency is far > from the only important factor in software development. > > > [0] Haskell is another excellent candidate here. > *_______________________________________________* > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Zach Kessin Teaching Web Developers to test code to find more bugs in less time Skype: zachkessin +972 54 234 3956 / +44 203 734 9790 / +1 617 778 7213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohammedmazharuddin@REDACTED Thu Sep 14 10:22:14 2017 From: mohammedmazharuddin@REDACTED (Mohammed mazhar uddin) Date: Thu, 14 Sep 2017 13:52:14 +0530 Subject: [erlang-questions] COWBOY loop handler issue Message-ID: <8b021a16-4025-9ff6-5fa0-316587a45079@utl.in> Hello, I am new to cowboy and i am trying to create a web application but here i am having a issue with loop handler or say server sent events, my requirement is that i need to send a message to my loop handler from another module which in terms will send the message to clients and buttons will be rendered on the web page. my questions are 1)How can i send message to my loop handler from another module? 2)Which process exactly spawns my loop handler. 3)Why my loop handler is dying and executing again and again after a specific span of time when a client is in communication with server. The big picture of my requirement. -I have a Terminal and few network elements, the moment those elements boots up they send a registration request to my erlang module from where i have to send a message to my loop handler to render those elements in the form of buttons in my web application. -Here i cant even register my loop handler as every time a new request comes, a new process is being created. how do i overcome this issue? please reply in the earliest. Thanks in advance. Regards, Mazhar. From raimo+erlang-questions@REDACTED Thu Sep 14 12:34:39 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 14 Sep 2017 12:34:39 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <20170914103439.GA16273@erix.ericsson.se> On Wed, Sep 13, 2017 at 01:47:08AM +0000, Thom Cherryhomes wrote: : > the ass to do any intricate hardware driver work... I'm guessing that Rust > is designed by 20-somethings who have never done a line of assembler in > their lives? That was a bit offensive... > > sigh. > -Thom -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Thu Sep 14 13:10:34 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 14 Sep 2017 14:10:34 +0300 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <20170914103439.GA16273@erix.ericsson.se> References: <20170914103439.GA16273@erix.ericsson.se> Message-ID: Good topic =) We are writing right now IP camera firmware in Rust and I can tell you: it is COMPLICATED =) When Rust comes to Future and Stream, it becomes a really cryptic thing for a newcomer. However, one can wait that writing something in Rust will allow to use all power of LLVM. -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Sep 14 13:11:12 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 14 Sep 2017 13:11:12 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: > Le 13 sept. 2017 ? 03:47, Thom Cherryhomes a ?crit : > > For a language that purports to be a replacement for low-level systems programming, Rust not having bitfields or unions, would be a real pain in the ass to do any intricate hardware driver work... I'm guessing that Rust is designed by 20-somethings who have never done a line of assembler in their lives? It must be a real treat to work with you! You seem to be working on "a next generation smart home system", so how do you feel about the inevitable breach of security your product will be hit with for not using memory-safe languages? Or are you one of these people who think "only bad programmers write insecure C/C++ code"? Anyway, you are the one who sounds like an immature 20-somethings here. From roger@REDACTED Thu Sep 14 13:36:10 2017 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 14 Sep 2017 12:36:10 +0100 Subject: [erlang-questions] COWBOY loop handler issue In-Reply-To: <8b021a16-4025-9ff6-5fa0-316587a45079@utl.in> References: <8b021a16-4025-9ff6-5fa0-316587a45079@utl.in> Message-ID: On 14 September 2017 at 09:22, Mohammed mazhar uddin wrote: > Hello, > > I am new to cowboy and i am trying to create a web application but here > i am having a issue with loop handler or say server sent events, > > my requirement is that i need to send a message to my loop handler > from another module which in terms will send the message to clients and > buttons will be rendered on the web page. > > my questions are > > 1)How can i send message to my loop handler from another module? You need to be able to find the loop handler pid from somewhere else. That usually means some sort of process registry. Once you've got the Pid, just Pid ! Event. See http://blog.differentpla.net/blog/2014/11/07/erlang-sup-event#give-me-an-example, for an example. That covers Erlang supervised event handlers as well (which is out of scope for what you're asking), but it does demonstrate using a cowboy loop handler for server-sent events. > 2)Which process exactly spawns my loop handler. Doesn't matter. > 3)Why my loop handler is dying and executing again and again after a > specific span of time when a client is in communication with server. Timeouts? Impossible to say. > please reply in the earliest. This is a mailing list, not a paid support channel. From Dinislam.Salikhov@REDACTED Thu Sep 14 16:55:04 2017 From: Dinislam.Salikhov@REDACTED (Salikhov Dinislam) Date: Thu, 14 Sep 2017 17:55:04 +0300 Subject: [erlang-questions] prim_inet:send/3 hangs with nonexisting port Message-ID: <089d1162-3fba-fa65-c681-ce235bde8da2@kaspersky.com> Hello. I have ejabberd based application. I've paid attention that some of the processes have messages in their message queue and their reductions number doesn't change. After some investigation I've figured out that all such processes waits in prim_inet:send/3 with a port passed there (each process with a separate port). But when I've tried to find the port in erlang:ports(), there is no such port in it. Is it a bug I stumbled upon or such situation may be possible under normal operation? P.S. I use OTP-18.3 Thanks in advance. Salikhov Dinislam From sverker.eriksson@REDACTED Thu Sep 14 17:30:47 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 14 Sep 2017 17:30:47 +0200 Subject: [erlang-questions] prim_inet:send/3 hangs with nonexisting port In-Reply-To: <089d1162-3fba-fa65-c681-ce235bde8da2@kaspersky.com> References: <089d1162-3fba-fa65-c681-ce235bde8da2@kaspersky.com> Message-ID: <6338df51-a3e6-b0e0-66c1-c4ead3699e03@ericsson.com> This release note from erts-7.3.1 (OTP-18.3.1) seems to fit your symptom: A process communicating with a port via one of the erlang:port_* BIFs could potentially end up in an inconsistent state if the port terminated during the communication. When this occurred the process could later block in a receive even though it had messages matching in its message queue. This bug was introduced in erts version 5.10 (OTP R16A). /Sverker On 09/14/2017 04:55 PM, Salikhov Dinislam wrote: > Hello. > > I have ejabberd based application. > I've paid attention that some of the processes have messages in their > message queue and their reductions number doesn't change. > > After some investigation I've figured out that all such processes > waits in prim_inet:send/3 with a port passed there (each process with > a separate port). > But when I've tried to find the port in erlang:ports(), there is no > such port in it. > Is it a bug I stumbled upon or such situation may be possible under > normal operation? > > P.S. I use OTP-18.3 > > Thanks in advance. > Salikhov Dinislam > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From heinz@REDACTED Thu Sep 14 23:49:43 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Thu, 14 Sep 2017 23:49:43 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> Message-ID: <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> Oh yes don?t get me wrong, rust IS great, I use it for some applications myself, but those applications are short running or frequently restarting (like a browse ;) the occasional update there is not a problem at all, but yea the kind of things erlang is often used probably doesn?t share that characteristic. > On 14. Sep 2017, at 00:25, Anthony Ramine wrote: > > Just don't use unstable features. You can do a damn lot without them, like the whole style system of Servo that is landing in Firefox 57. > > https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-engine-quantum-css-aka-stylo/ > > That may not be the correct venue to discuss this, of course. :) > > The argument about embedded platforms in 100% correct, though. Rust isn't ready yet on that front. > >> Le 13 sept. 2017 ? 17:11, Heinz N. Gies a ?crit : >> >> For once Rust is in it?s infancy, the whole language still changes, code that works today might not work tomorrow and libraries often depend on nightly builds and new features. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From gabriel.alberto.rivas@REDACTED Thu Sep 14 23:16:03 2017 From: gabriel.alberto.rivas@REDACTED (Gabriel Rivas) Date: Thu, 14 Sep 2017 17:16:03 -0400 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] Message-ID: Hello all, I am trying to understand BEAM files most specifically the Code section. For that objective I wrote a simple program consisting of just a subtraction function taking 2 parameters: -module(ex3). -export([sub/2]). sub(X,Y) -> X-Y. >From that file I created the .S version, and I can see that the assembly version for the subtraction is: {gc_bif,'-',{f,0},2,[{x,0},{x,1}],{x,0}}. return. In the BEAM file I can see that the bytecode part that correlated with this function is (in hex numbers): 7D 05 20 00 03 13 03 13 After some reading of the BEAM book and some analysis, I could translate these hex numbers to: 7D 05 20 = {gc_bif2, Lbl = 05, Live X registers = 2, Bif = 00, 03 13 03 13 = args: X0,X1 return: X0 13 -> k_return But I still don't see how the subtraction happens. Can anybody shed some light on the internals of calling a BIF in the erlang module? Regards, Gabriel -------------- next part -------------- An HTML attachment was scrubbed... URL: From peterdmv@REDACTED Fri Sep 15 10:06:18 2017 From: peterdmv@REDACTED (Peter Dimitrov) Date: Fri, 15 Sep 2017 10:06:18 +0200 Subject: [erlang-questions] Patch package OTP 20.0.5 released Message-ID: <20170915080618.GA32677@duper.otp.ericsson.se> Patch Package: OTP 20.0.5 Git Tag: OTP-20.0.5 Date: 2017-09-14 Trouble Report Id: OTP-14450, OTP-14573, OTP-14590 Seq num: ERL-474 System: OTP Release: 20 Application: erts-9.0.5, inets-6.4.1 Predecessor: OTP 20.0.4 Check out the git tag OTP-20.0.5, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-9.0.5 ------------------------------------------------------ --------------------------------------------------------------------- The erts-9.0.5 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14590 Application(s): erts Related Id(s): ERL-474 Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0). Full runtime dependencies of erts-9.0.5: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --- inets-6.4.1 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.4.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-14573 Application(s): inets http_uri aligned to follow RFC 3986 and not convert "+" to space when decoding URIs. --- Improvements and New Features --- OTP-14450 Application(s): inets Added new option max_client_body_chunk to httpd server to allow chunked delivery of PUT and POST data to mod_esi callback. Note, new mod_esi callback implementation is required. Also correct value provided by server_name environment variable Full runtime dependencies of inets-6.4.1: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From dmytro.lytovchenko@REDACTED Fri Sep 15 10:24:58 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Fri, 15 Sep 2017 10:24:58 +0200 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] In-Reply-To: References: Message-ID: Similar to bif0...bif3 gc_bifN uses import index from the import table. Import table contains triples {Mod, Fun, Arity} for external functions used in the current module: http://beam-wisdoms.clau.se/en/latest/indepth-beam-instructions.html#bif1-11-bif2-152-bif3 So Bif=00 is the 0th index in the import table. http://beam-wisdoms.clau.se/en/latest/indepth-beam-file.html#impt-imports-table 2017-09-14 23:16 GMT+02:00 Gabriel Rivas : > Hello all, > > I am trying to understand BEAM files most specifically the Code section. > For that objective I wrote a simple program consisting of just a > subtraction function > taking 2 parameters: > > -module(ex3). > -export([sub/2]). > > sub(X,Y) -> > X-Y. > > From that file I created the .S version, and I can see that the assembly > version for the subtraction is: > > {gc_bif,'-',{f,0},2,[{x,0},{x,1}],{x,0}}. > return. > > In the BEAM file I can see that the bytecode part that correlated with this > function is (in hex numbers): > > 7D 05 20 00 03 13 03 13 > > After some reading of the BEAM book and some analysis, I could translate > these hex numbers to: > > 7D 05 20 = {gc_bif2, > Lbl = 05, > Live X registers = 2, > Bif = 00, > > 03 13 03 13 = args: X0,X1 > return: X0 > 13 -> k_return > > But I still don't see how the subtraction happens. Can anybody shed some > light > on the internals of calling a BIF in the erlang module? > > Regards, > Gabriel > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Fri Sep 15 10:46:06 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 15 Sep 2017 10:46:06 +0200 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] In-Reply-To: References: Message-ID: On Thu, Sep 14, 2017 at 11:16 PM, Gabriel Rivas wrote: > > But I still don't see how the subtraction happens. Can anybody shed some > light > on the internals of calling a BIF in the erlang module? Not sure that you are aware of it, but there is a disassembler that is used like this: erts_debug:df(ex3). It creates a file named ex3.dis. In it, you can see that the loaded code looks this: 00007F9C8F953AD0: i_func_info_IaaI 0 ex3 sub 2 00007F9C8F953AF8: i_minus_jIxxd j(0000000000000000) 2 x(0) x(1) x(0) 00007F9C8F953B28: return In this particular case, you can see that the BIF call has been translated to a special instruction. The translations that the loader makes can be found in erts/emulator/beam/ops.tab. /Bjorn From bchesneau@REDACTED Fri Sep 15 11:50:35 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 15 Sep 2017 11:50:35 +0200 Subject: [erlang-questions] valgrind and OTP 20 on osx Message-ID: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> Hi all I followed the installation steps on the INSTALL page to build with valgrind support: http://erlang.org/doc/installation_guide/INSTALL.html but when I?m running cerl, I get the following error. Any idea what could be the issue? - benoit 2017-09-15 11:44:52 ? Benoits-iMac in ~/local/src/otp_src_20.0 ? ? ./bin/cerl -valgrind -pa ~//_build/default/lib/*/ebin ? ==59047== ==59047== Syscall param msg->desc.port.name points to uninitialised byte(s) ==59047== at 0x1008EC34A: mach_msg_trap (in /usr/lib/system/libsystem_kernel.dylib) ==59047== by 0x1008EB796: mach_msg (in /usr/lib/system/libsystem_kernel.dylib) ==59047== by 0x1008E5485: task_set_special_port (in /usr/lib/system/libsystem_kernel.dylib) ==59047== by 0x100A8110E: _os_trace_create_debug_control_port (in /usr/lib/system/libsystem_trace.dylib) ==59047== by 0x100A81458: _libtrace_init (in /usr/lib/system/libsystem_trace.dylib) ==59047== by 0x1005A09DF: libSystem_initializer (in /usr/lib/libSystem.B.dylib) ==59047== by 0x1004F2A1A: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld) ==59047== by 0x1004F2C1D: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld) ==59047== by 0x1004EE4A9: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) ==59047== by 0x1004EE440: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) ==59047== by 0x1004ED523: ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) ==59047== by 0x1004ED5B8: ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) (in /usr/lib/dyld) ==59047== Address 0x104d63abc is on thread 1's stack ==59047== in frame #2, created by task_set_special_port (???:) ==59047== ==59047== Warning: set address range perms: large range [0x10c64b000, 0x14c64b000) (noaccess) ==59047== Thread 2: ==59047== Invalid read of size 4 ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) ==59047== Address 0x18 is not stack'd, malloc'd or (recently) free'd ==59047== ==59047== ==59047== Process terminating with default action of signal 11 (SIGSEGV) ==59047== Access not within mapped region at address 0x18 ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) ==59047== If you believe this happened as a result of a stack ==59047== overflow in your program's main thread (unlikely but ==59047== possible), you can try to increase the size of the ==59047== main thread stack using the --main-stacksize= flag. ==59047== The main thread stack size used in this run was 8388608. --59047:0:schedule VG_(sema_down): read returned -4 ==59047== ==59047== HEAP SUMMARY: ==59047== in use at exit: 1,708,783 bytes in 821 blocks ==59047== total heap usage: 1,747 allocs, 926 frees, 2,874,463 bytes allocated ==59047== ==59047== LEAK SUMMARY: ==59047== definitely lost: 0 bytes in 0 blocks ==59047== indirectly lost: 0 bytes in 0 blocks ==59047== possibly lost: 410,126 bytes in 181 blocks ==59047== still reachable: 74,145 bytes in 325 blocks ==59047== suppressed: 1,224,512 bytes in 315 blocks ==59047== Rerun with --leak-check=full to see details of leaked memory ==59047== ==59047== For counts of detected and suppressed errors, rerun with: -v ==59047== Use --track-origins=yes to see where uninitialised values come from ==59047== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 25 from 4) Segmentation fault: 11 From sverker.eriksson@REDACTED Fri Sep 15 13:37:04 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 15 Sep 2017 13:37:04 +0200 Subject: [erlang-questions] valgrind and OTP 20 on osx In-Reply-To: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> References: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> Message-ID: <248bf710-1c77-7018-f3d9-042dc5035f9a@ericsson.com> To increase your chance of success, I suggest you configure with --disable-hipe. I don't know if I have ever run beam on valgrind on osx, so I don't know that to expect. It looks like it has trouble just spawning a thread. /Sverker, Erlang/OTP On 09/15/2017 11:50 AM, Benoit Chesneau wrote: > Hi all I followed the installation steps on the INSTALL page to build with valgrind support: > http://erlang.org/doc/installation_guide/INSTALL.html > > but when I?m running cerl, I get the following error. Any idea what could be the issue? > > - benoit > > 2017-09-15 11:44:52 ? Benoits-iMac in ~/local/src/otp_src_20.0 > ? ? ./bin/cerl -valgrind -pa ~//_build/default/lib/*/ebin > > ? > > ==59047== > ==59047== Syscall param msg->desc.port.name points to uninitialised byte(s) > ==59047== at 0x1008EC34A: mach_msg_trap (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x1008EB796: mach_msg (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x1008E5485: task_set_special_port (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x100A8110E: _os_trace_create_debug_control_port (in /usr/lib/system/libsystem_trace.dylib) > ==59047== by 0x100A81458: _libtrace_init (in /usr/lib/system/libsystem_trace.dylib) > ==59047== by 0x1005A09DF: libSystem_initializer (in /usr/lib/libSystem.B.dylib) > ==59047== by 0x1004F2A1A: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld) > ==59047== by 0x1004F2C1D: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld) > ==59047== by 0x1004EE4A9: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004EE440: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004ED523: ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004ED5B8: ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) (in /usr/lib/dyld) > ==59047== Address 0x104d63abc is on thread 1's stack > ==59047== in frame #2, created by task_set_special_port (???:) > ==59047== > ==59047== Warning: set address range perms: large range [0x10c64b000, 0x14c64b000) (noaccess) > ==59047== Thread 2: > ==59047== Invalid read of size 4 > ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== Address 0x18 is not stack'd, malloc'd or (recently) free'd > ==59047== > ==59047== > ==59047== Process terminating with default action of signal 11 (SIGSEGV) > ==59047== Access not within mapped region at address 0x18 > ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== If you believe this happened as a result of a stack > ==59047== overflow in your program's main thread (unlikely but > ==59047== possible), you can try to increase the size of the > ==59047== main thread stack using the --main-stacksize= flag. > ==59047== The main thread stack size used in this run was 8388608. > --59047:0:schedule VG_(sema_down): read returned -4 > ==59047== > ==59047== HEAP SUMMARY: > ==59047== in use at exit: 1,708,783 bytes in 821 blocks > ==59047== total heap usage: 1,747 allocs, 926 frees, 2,874,463 bytes allocated > ==59047== > ==59047== LEAK SUMMARY: > ==59047== definitely lost: 0 bytes in 0 blocks > ==59047== indirectly lost: 0 bytes in 0 blocks > ==59047== possibly lost: 410,126 bytes in 181 blocks > ==59047== still reachable: 74,145 bytes in 325 blocks > ==59047== suppressed: 1,224,512 bytes in 315 blocks > ==59047== Rerun with --leak-check=full to see details of leaked memory > ==59047== > ==59047== For counts of detected and suppressed errors, rerun with: -v > ==59047== Use --track-origins=yes to see where uninitialised values come from > ==59047== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 25 from 4) > Segmentation fault: 11 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sergej.jurecko@REDACTED Fri Sep 15 13:37:49 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 15 Sep 2017 13:37:49 +0200 Subject: [erlang-questions] valgrind and OTP 20 on osx In-Reply-To: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> References: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> Message-ID: <0D8F036B-76BD-4CCD-AEE4-3297811D65BE@gmail.com> Hi, not sure if valgrind has improved on macOS in the last year or two but historically it has only barely worked on it. I never could get it to work with Erlang. Much less of a hassle to run it on linux. Regards, Sergej > On 15 Sep 2017, at 11:50, Benoit Chesneau wrote: > > Hi all I followed the installation steps on the INSTALL page to build with valgrind support: > http://erlang.org/doc/installation_guide/INSTALL.html > > but when I?m running cerl, I get the following error. Any idea what could be the issue? > > - benoit > > 2017-09-15 11:44:52 ? Benoits-iMac in ~/local/src/otp_src_20.0 > ? ? ./bin/cerl -valgrind -pa ~//_build/default/lib/*/ebin > > ? > > ==59047== > ==59047== Syscall param msg->desc.port.name points to uninitialised byte(s) > ==59047== at 0x1008EC34A: mach_msg_trap (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x1008EB796: mach_msg (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x1008E5485: task_set_special_port (in /usr/lib/system/libsystem_kernel.dylib) > ==59047== by 0x100A8110E: _os_trace_create_debug_control_port (in /usr/lib/system/libsystem_trace.dylib) > ==59047== by 0x100A81458: _libtrace_init (in /usr/lib/system/libsystem_trace.dylib) > ==59047== by 0x1005A09DF: libSystem_initializer (in /usr/lib/libSystem.B.dylib) > ==59047== by 0x1004F2A1A: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld) > ==59047== by 0x1004F2C1D: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld) > ==59047== by 0x1004EE4A9: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004EE440: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004ED523: ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) > ==59047== by 0x1004ED5B8: ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) (in /usr/lib/dyld) > ==59047== Address 0x104d63abc is on thread 1's stack > ==59047== in frame #2, created by task_set_special_port (???:) > ==59047== > ==59047== Warning: set address range perms: large range [0x10c64b000, 0x14c64b000) (noaccess) > ==59047== Thread 2: > ==59047== Invalid read of size 4 > ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== Address 0x18 is not stack'd, malloc'd or (recently) free'd > ==59047== > ==59047== > ==59047== Process terminating with default action of signal 11 (SIGSEGV) > ==59047== Access not within mapped region at address 0x18 > ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) > ==59047== If you believe this happened as a result of a stack > ==59047== overflow in your program's main thread (unlikely but > ==59047== possible), you can try to increase the size of the > ==59047== main thread stack using the --main-stacksize= flag. > ==59047== The main thread stack size used in this run was 8388608. > --59047:0:schedule VG_(sema_down): read returned -4 > ==59047== > ==59047== HEAP SUMMARY: > ==59047== in use at exit: 1,708,783 bytes in 821 blocks > ==59047== total heap usage: 1,747 allocs, 926 frees, 2,874,463 bytes allocated > ==59047== > ==59047== LEAK SUMMARY: > ==59047== definitely lost: 0 bytes in 0 blocks > ==59047== indirectly lost: 0 bytes in 0 blocks > ==59047== possibly lost: 410,126 bytes in 181 blocks > ==59047== still reachable: 74,145 bytes in 325 blocks > ==59047== suppressed: 1,224,512 bytes in 315 blocks > ==59047== Rerun with --leak-check=full to see details of leaked memory > ==59047== > ==59047== For counts of detected and suppressed errors, rerun with: -v > ==59047== Use --track-origins=yes to see where uninitialised values come from > ==59047== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 25 from 4) > Segmentation fault: 11 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lloyd@REDACTED Fri Sep 15 16:20:15 2017 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Fri, 15 Sep 2017 10:20:15 -0400 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers Message-ID: Hi, Joe Armstrong states in his 2003 Doctor of Technology thesis: "Our system has very little need of an operating system. We make use of very few operating system services, thus it is relatively easy to port our system to specialised environments such as embedded systems." As a one-time Forth developer, software bloat offends me. I sigh and use Ubuntu on my development system out laziness and convenience. But I would like my production servers to be lean and mean. What light weight open-source off-the shelf operating systems would battle-hardened Erlang gurus recommend? All the best, LRP Sent from my iPad -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Sep 15 16:34:44 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Sep 2017 15:34:44 +0100 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: On 15 September 2017 at 15:20, Lloyd R. Prentice wrote: > What light weight open-source off-the shelf operating systems would > battle-hardened Erlang gurus recommend? I can't claim to be battle-hardened, and I can't claim to have any experience of *this* at all, but how about *no* OS? http://erlangonxen.org/ From smcojocaru@REDACTED Fri Sep 15 13:03:45 2017 From: smcojocaru@REDACTED (Sergiu Cojocaru) Date: Fri, 15 Sep 2017 11:03:45 +0000 (UTC) Subject: [erlang-questions] Calling mnesia:first/1 and mnesia:last/1 on fragmented tables References: <1152565176.82809.1505473425387.ref@mail.yahoo.com> Message-ID: <1152565176.82809.1505473425387@mail.yahoo.com> Hi, Is it possible to get the expected first and last records from a fragmented table, using mnesia:first/1 and mnesia:last/1 functions? From my testing, when using these functions with mnesia:activity, the returned records do not correspond to the actually first and last inserted data. If?mnesia:first/1 and mnesia:last/1 do not work, is it possible maybe achieve the desired result using some other functions/technics? Many thanks,Sergiu -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabriel.alberto.rivas@REDACTED Fri Sep 15 15:11:20 2017 From: gabriel.alberto.rivas@REDACTED (Gabriel Rivas) Date: Fri, 15 Sep 2017 09:11:20 -0400 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] In-Reply-To: References: Message-ID: Thanks Dmytro, I understand it now, you are doing a great job with BEAM Wisdoms. On Fri, Sep 15, 2017 at 4:24 AM, Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > Similar to bif0...bif3 gc_bifN uses import index from the import table. > Import table contains triples {Mod, Fun, Arity} for external functions used > in the current module: > > http://beam-wisdoms.clau.se/en/latest/indepth-beam- > instructions.html#bif1-11-bif2-152-bif3 > > So Bif=00 is the 0th index in the import table. > > http://beam-wisdoms.clau.se/en/latest/indepth-beam-file. > html#impt-imports-table > > 2017-09-14 23:16 GMT+02:00 Gabriel Rivas > : > >> Hello all, >> >> I am trying to understand BEAM files most specifically the Code section. >> For that objective I wrote a simple program consisting of just a >> subtraction function >> taking 2 parameters: >> >> -module(ex3). >> -export([sub/2]). >> >> sub(X,Y) -> >> X-Y. >> >> From that file I created the .S version, and I can see that the assembly >> version for the subtraction is: >> >> {gc_bif,'-',{f,0},2,[{x,0},{x,1}],{x,0}}. >> return. >> >> In the BEAM file I can see that the bytecode part that correlated with >> this >> function is (in hex numbers): >> >> 7D 05 20 00 03 13 03 13 >> >> After some reading of the BEAM book and some analysis, I could translate >> these hex numbers to: >> >> 7D 05 20 = {gc_bif2, >> Lbl = 05, >> Live X registers = 2, >> Bif = 00, >> >> 03 13 03 13 = args: X0,X1 >> return: X0 >> 13 -> k_return >> >> But I still don't see how the subtraction happens. Can anybody shed some >> light >> on the internals of calling a BIF in the erlang module? >> >> Regards, >> Gabriel >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabriel.alberto.rivas@REDACTED Fri Sep 15 15:15:32 2017 From: gabriel.alberto.rivas@REDACTED (Gabriel Rivas) Date: Fri, 15 Sep 2017 09:15:32 -0400 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] In-Reply-To: References: Message-ID: Hello Bjorn, I was not aware of it, I am new to Erlang. I started doing development on Elixir but I would like to learn and understand what Erlang does under the hood. Thanks for the tip, it is very useful. Gabriel On Fri, Sep 15, 2017 at 4:46 AM, Bj?rn Gustavsson wrote: > On Thu, Sep 14, 2017 at 11:16 PM, Gabriel Rivas > wrote: > > > > > But I still don't see how the subtraction happens. Can anybody shed some > > light > > on the internals of calling a BIF in the erlang module? > > Not sure that you are aware of it, but there is a disassembler that is > used like this: > > erts_debug:df(ex3). > > It creates a file named ex3.dis. In it, you can see that the loaded > code looks this: > > 00007F9C8F953AD0: i_func_info_IaaI 0 ex3 sub 2 > 00007F9C8F953AF8: i_minus_jIxxd j(0000000000000000) 2 x(0) x(1) x(0) > 00007F9C8F953B28: return > > In this particular case, you can see that the BIF call has been > translated to a special instruction. > > The translations that the loader makes can be found in > erts/emulator/beam/ops.tab. > > /Bjorn > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Fri Sep 15 16:45:58 2017 From: felixgallo@REDACTED (felixgallo@REDACTED) Date: Fri, 15 Sep 2017 07:45:58 -0700 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: FreeBSD and alpine Linux have both served me well. F. > On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice wrote: > > Hi, > > Joe Armstrong states in his 2003 Doctor of Technology thesis: > > "Our system has very little need of an operating system. We make use of very few operating system services, thus it is relatively easy to port our system to specialised environments such as embedded systems." > > As a one-time Forth developer, software bloat offends me. I sigh and use Ubuntu on my development system out laziness and convenience. > > But I would like my production servers to be lean and mean. > > What light weight open-source off-the shelf operating systems would battle-hardened Erlang gurus recommend? > > All the best, > > LRP > > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Sep 15 17:28:16 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 15 Sep 2017 17:28:16 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> Message-ID: <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> While we are on it , does rust run on solaris? - benoit > On 14 Sep 2017, at 23:49, Heinz N. Gies wrote: > > Oh yes don?t get me wrong, rust IS great, I use it for some applications myself, but those applications are short running or frequently restarting (like a browse ;) the occasional update there is not a problem at all, but yea the kind of things erlang is often used probably doesn?t share that characteristic. >> On 14. Sep 2017, at 00:25, Anthony Ramine wrote: >> >> Just don't use unstable features. You can do a damn lot without them, like the whole style system of Servo that is landing in Firefox 57. >> >> https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-engine-quantum-css-aka-stylo/ >> >> That may not be the correct venue to discuss this, of course. :) >> >> The argument about embedded platforms in 100% correct, though. Rust isn't ready yet on that front. >> >>> Le 13 sept. 2017 ? 17:11, Heinz N. Gies a ?crit : >>> >>> For once Rust is in it?s infancy, the whole language still changes, code that works today might not work tomorrow and libraries often depend on nightly builds and new features. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From heinz@REDACTED Fri Sep 15 17:36:09 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 15 Sep 2017 17:36:09 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: <9966C83C-A2C4-4346-97B9-F86A03C7D3C2@licenser.net> SmartOS (or other Illumos?s) they run Erlang very well, Illumos is one of the most stable systems in existence, and SmartOS it a pure in memory OS that doesn?t even take any disk space Boots either from USB or PXE > On 15. Sep 2017, at 16:20, Lloyd R. Prentice wrote: > > Hi, > > Joe Armstrong states in his 2003 Doctor of Technology thesis: > > "Our system has very little need of an operating system. We make use of very few operating system services, thus it is relatively easy to port our system to specialised environments such as embedded systems." > > As a one-time Forth developer, software bloat offends me. I sigh and use Ubuntu on my development system out laziness and convenience. > > But I would like my production servers to be lean and mean. > > What light weight open-source off-the shelf operating systems would battle-hardened Erlang gurus recommend? > > All the best, > > LRP > > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From jozsef.berces@REDACTED Fri Sep 15 17:51:31 2017 From: jozsef.berces@REDACTED (=?utf-8?B?SsOzenNlZiBCw6lyY2Vz?=) Date: Fri, 15 Sep 2017 15:51:31 +0000 Subject: [erlang-questions] xilinx petalinux Message-ID: Hi, Has anyone tried to compile Erlang/OTP for Xilinx Petalinux? Well, I did and failed, so my last hope is in this community ?? I created an xcomp config file based on the existing erl-xcomp-arm-linux.conf and replaced the compiler varables to point to XSDK. For example: CC="/opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/arm-linux-gnueabihf-gcc --sysroot=$ARM_SYSROOT" I also added the following at the beginning of the config file: ARM_SYSROOT=/opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/arm-linux-gnueabihf/libc It went well until linking time when I got these errors: /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -ldlpi /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -linet /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -ltinfo /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lkstat Any ideas how to fix this? Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Sep 15 18:05:45 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 15 Sep 2017 18:05:45 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <20170914103439.GA16273@erix.ericsson.se> Message-ID: <1F19263E-F4DF-4B84-9730-B4785497F7C8@gmail.com> > On 14 Sep 2017, at 13:10, Max Lapshin wrote: > > Good topic =) > > We are writing right now IP camera firmware in Rust and I can tell you: it is COMPLICATED =) > > When Rust comes to Future and Stream, it becomes a really cryptic thing for a newcomer. Rust Futures are way overhyped, overused and I think the last thing a newcomer should be using. Regards, Sergej From joel@REDACTED Fri Sep 15 18:33:13 2017 From: joel@REDACTED (Joel Reymont) Date: Fri, 15 Sep 2017 16:33:13 +0000 Subject: [erlang-questions] Aeternity is hiring Erlang developers with blockchain experience Message-ID: Aeternity is looking for fully autonomous software hackers proficient in Erlang. We are a distributed team, we work remotely and our software is completely open source. You should be able to explain how Bitcoin works and how it's different from Ethereum. Ideally, you should have further experience with decentralized crypto technologies (IPFS, torrents) or other cryptocurrencies. You should also have a Github account with a significant body of public source code. We are in a rush to deliver cutting-edge blockchain software so you should be able to hit the ground running and start contributing. No on-the-job training is provided! Please contact Joel Reymont at joel@REDACTED for more information. Principals only. No recruiters, agencies or third parties. https://blog.aeternity.com/join-the-t%C3%A6m-rust-or-erlang-devs-wanted-31908daba788 -------------- next part -------------- An HTML attachment was scrubbed... URL: From codewiget95@REDACTED Fri Sep 15 18:43:40 2017 From: codewiget95@REDACTED (code wiget) Date: Fri, 15 Sep 2017 12:43:40 -0400 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store Message-ID: Hello everyone, I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. I have looked at as many options as I can -> Voldemort : http://project-voldemort.com/ - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. Cassandra: http://cassandra.apache.org/ - looks good too, but apparently there is a small community and apparently isn?t updated often Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. So from my research, which consisted heavily of this blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? Thanks for your time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Fri Sep 15 19:33:40 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Fri, 15 Sep 2017 19:33:40 +0200 Subject: [erlang-questions] xilinx petalinux In-Reply-To: References: Message-ID: <22972.3828.348353.512183@gargle.gargle.HOWL> J?zsef B?rces writes: > Hi, > > Has anyone tried to compile Erlang/OTP for Xilinx Petalinux? Well, I did and failed, so my last hope is in this community ?? > > I created an xcomp config file based on the existing erl-xcomp-arm-linux.conf and replaced the compiler varables to point to XSDK. For example: > > CC="/opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/arm-linux-gnueabihf-gcc --sysroot=$ARM_SYSROOT" > > I also added the following at the beginning of the config file: > > ARM_SYSROOT=/opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/arm-linux-gnueabihf/libc > > It went well until linking time when I got these errors: > > /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -ldlpi > /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -linet > /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -ltinfo > /opt/Xilinx/SDK/2017.2/gnu/aarch32/lin/gcc-arm-linux-gnueabi/bin/../lib/gcc/arm-linux-gnueabihf/6.2.1/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lkstat > > Any ideas how to fix this? There are two documented ways to cross-compile Erlang/OTP. I've never used the pre-cooked "xcomp" route myself, but a normal configure/make (following HOWTO/INSTALL-CROSS.md) has worked well for me in the past. Three of the missing libraries listed above seem unexpected to me for a GNU/Linux system. ./configure _should_ detect their absence so I can only assume that the "xcomp" route makes too many hard-coded assumptions, or has mis-identified your target system. The -ltinfo issue can be worked around by passing --without-termcap to ./configure, but you then get an Erlang shell that's less featureful when it comes to command-line editing and such. If you want these features, cross-build ncurses and install that in your sysroot first. From bchesneau@REDACTED Fri Sep 15 19:38:48 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 15 Sep 2017 19:38:48 +0200 Subject: [erlang-questions] valgrind and OTP 20 on osx In-Reply-To: <0D8F036B-76BD-4CCD-AEE4-3297811D65BE@gmail.com> References: <849BB4AD-A26E-4774-AF2D-9AB5A313B9F2@gmail.com> <0D8F036B-76BD-4CCD-AEE4-3297811D65BE@gmail.com> Message-ID: <70C82F9D-C504-4C06-969E-D4A13DA9C979@gmail.com> I confirm. It works like a charm on Linux. Thanks to both :) - benoit > On 15 Sep 2017, at 13:37, Sergej Jure?ko wrote: > > Hi, > > not sure if valgrind has improved on macOS in the last year or two but historically it has only barely worked on it. I never could get it to work with Erlang. Much less of a hassle to run it on linux. > > > Regards, > Sergej > >> On 15 Sep 2017, at 11:50, Benoit Chesneau wrote: >> >> Hi all I followed the installation steps on the INSTALL page to build with valgrind support: >> http://erlang.org/doc/installation_guide/INSTALL.html >> >> but when I?m running cerl, I get the following error. Any idea what could be the issue? >> >> - benoit >> >> 2017-09-15 11:44:52 ? Benoits-iMac in ~/local/src/otp_src_20.0 >> ? ? ./bin/cerl -valgrind -pa ~//_build/default/lib/*/ebin >> >> ? >> >> ==59047== >> ==59047== Syscall param msg->desc.port.name points to uninitialised byte(s) >> ==59047== at 0x1008EC34A: mach_msg_trap (in /usr/lib/system/libsystem_kernel.dylib) >> ==59047== by 0x1008EB796: mach_msg (in /usr/lib/system/libsystem_kernel.dylib) >> ==59047== by 0x1008E5485: task_set_special_port (in /usr/lib/system/libsystem_kernel.dylib) >> ==59047== by 0x100A8110E: _os_trace_create_debug_control_port (in /usr/lib/system/libsystem_trace.dylib) >> ==59047== by 0x100A81458: _libtrace_init (in /usr/lib/system/libsystem_trace.dylib) >> ==59047== by 0x1005A09DF: libSystem_initializer (in /usr/lib/libSystem.B.dylib) >> ==59047== by 0x1004F2A1A: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld) >> ==59047== by 0x1004F2C1D: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld) >> ==59047== by 0x1004EE4A9: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) >> ==59047== by 0x1004EE440: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) >> ==59047== by 0x1004ED523: ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld) >> ==59047== by 0x1004ED5B8: ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) (in /usr/lib/dyld) >> ==59047== Address 0x104d63abc is on thread 1's stack >> ==59047== in frame #2, created by task_set_special_port (???:) >> ==59047== >> ==59047== Warning: set address range perms: large range [0x10c64b000, 0x14c64b000) (noaccess) >> ==59047== Thread 2: >> ==59047== Invalid read of size 4 >> ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== Address 0x18 is not stack'd, malloc'd or (recently) free'd >> ==59047== >> ==59047== >> ==59047== Process terminating with default action of signal 11 (SIGSEGV) >> ==59047== Access not within mapped region at address 0x18 >> ==59047== at 0x100A49899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== by 0x100A49886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== by 0x100A4908C: thread_start (in /usr/lib/system/libsystem_pthread.dylib) >> ==59047== If you believe this happened as a result of a stack >> ==59047== overflow in your program's main thread (unlikely but >> ==59047== possible), you can try to increase the size of the >> ==59047== main thread stack using the --main-stacksize= flag. >> ==59047== The main thread stack size used in this run was 8388608. >> --59047:0:schedule VG_(sema_down): read returned -4 >> ==59047== >> ==59047== HEAP SUMMARY: >> ==59047== in use at exit: 1,708,783 bytes in 821 blocks >> ==59047== total heap usage: 1,747 allocs, 926 frees, 2,874,463 bytes allocated >> ==59047== >> ==59047== LEAK SUMMARY: >> ==59047== definitely lost: 0 bytes in 0 blocks >> ==59047== indirectly lost: 0 bytes in 0 blocks >> ==59047== possibly lost: 410,126 bytes in 181 blocks >> ==59047== still reachable: 74,145 bytes in 325 blocks >> ==59047== suppressed: 1,224,512 bytes in 315 blocks >> ==59047== Rerun with --leak-check=full to see details of leaked memory >> ==59047== >> ==59047== For counts of detected and suppressed errors, rerun with: -v >> ==59047== Use --track-origins=yes to see where uninitialised values come from >> ==59047== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 25 from 4) >> Segmentation fault: 11 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From heinz@REDACTED Fri Sep 15 19:47:29 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 15 Sep 2017 19:47:29 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> Message-ID: The answer would have to be ?a bit? it?s on and off depending on packages available and versions, same goes for BSD sadly. While rust does make an real effort to run outside of linux it?s very much ?linux or good luck to you my frinbed? these days ... > On 15. Sep 2017, at 17:28, Benoit Chesneau wrote: > > While we are on it , does rust run on solaris? > > - benoit >> On 14 Sep 2017, at 23:49, Heinz N. Gies wrote: >> >> Oh yes don?t get me wrong, rust IS great, I use it for some applications myself, but those applications are short running or frequently restarting (like a browse ;) the occasional update there is not a problem at all, but yea the kind of things erlang is often used probably doesn?t share that characteristic. >>> On 14. Sep 2017, at 00:25, Anthony Ramine wrote: >>> >>> Just don't use unstable features. You can do a damn lot without them, like the whole style system of Servo that is landing in Firefox 57. >>> >>> https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-engine-quantum-css-aka-stylo/ >>> >>> That may not be the correct venue to discuss this, of course. :) >>> >>> The argument about embedded platforms in 100% correct, though. Rust isn't ready yet on that front. >>> >>>> Le 13 sept. 2017 ? 17:11, Heinz N. Gies a ?crit : >>>> >>>> For once Rust is in it?s infancy, the whole language still changes, code that works today might not work tomorrow and libraries often depend on nightly builds and new features. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From heinz@REDACTED Fri Sep 15 19:48:41 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 15 Sep 2017 19:48:41 +0200 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: Message-ID: <9752DBE8-9168-41F9-A760-A9488506F3E0@licenser.net> Have you considered Risk? It?s a distributed erlang nosql/k/v store. > On 15. Sep 2017, at 18:43, code wiget wrote: > > Hello everyone, > > I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. > > The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. > > Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. > > A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. > > I have looked at as many options as I can -> > > Voldemort : http://project-voldemort.com/ > - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. > > Cassandra: http://cassandra.apache.org/ > - looks good too, but apparently there is a small community and apparently isn?t updated often > > Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf > - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. > > > So from my research, which consisted heavily of this blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. > > BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? > > Thanks for your time. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From sergej.jurecko@REDACTED Fri Sep 15 20:01:28 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 15 Sep 2017 20:01:28 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> Message-ID: <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> > On 15 Sep 2017, at 19:47, Heinz N. Gies wrote: > > The answer would have to be ?a bit? it?s on and off depending on packages available and versions, same goes for BSD sadly. > > While rust does make an real effort to run outside of linux it?s very much ?linux or good luck to you my frinbed? these days ? Linux or good luck is far from true. Rust works just fine and is well supported on Windows, macOS, iOS and Android. All giant platforms, though user not server oriented. Obviously a consequence of the largest Rust user being Mozilla. Regards, Sergej From nathaniel@REDACTED Fri Sep 15 20:02:10 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Fri, 15 Sep 2017 14:02:10 -0400 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: Message-ID: Scatter-shot reply: Since you're using Redis right now, have you considered Redis Cluster (https://redis.io/topics/cluster-tutorial )? I'm using Cassandra and don't feel that it's got a small community or slow pace of updates. There are a lot of NoSQL databases and they all have quite different tradeoffs which tends to fragment the community, so your expectations may be too high. Riak, ElasticSearch, EtcD, MongoDB, etc. You have many (too many!) options. When you say "read speed and consistency" what sort of consistency are you looking for? Is eventual consistency good, or do you require that every read that takes place after a write gets the new data? > On Sep 15, 2017, at 12:43 PM, code wiget wrote: > > Hello everyone, > > I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. > > The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. > > Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. > > A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. > > I have looked at as many options as I can -> > > Voldemort : http://project-voldemort.com/ > - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. > > Cassandra: http://cassandra.apache.org/ > - looks good too, but apparently there is a small community and apparently isn?t updated often > > Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf > - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. > > > So from my research, which consisted heavily of this blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. > > BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? > > Thanks for your time. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Fri Sep 15 20:50:12 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 15 Sep 2017 18:50:12 +0000 (UTC) Subject: [erlang-questions] Is there a way to prevent a crashing process from sending a message to error_logger? References: <2059779354.2996009.1505501412900.ref@mail.yahoo.com> Message-ID: <2059779354.2996009.1505501412900@mail.yahoo.com> I am dealing with crashing processes that tend to spam the error logger. The crashing processes error reason is being caught and everything is taken care of and logged via the trapping_exits, spawn_linker. Still the crashes spam the error_logger. I am wondering if there is a way to set specific processes to not send a message upon crash to the error_logger? What I do not want is to disable the error logger all together, redirect it, disable stdout, or write a custom error logger. The reason I still want the error logger to function as it does is because other processes? can crash, visualization of code recompiles/failures during development is important and general logging.? One solution I was helped with on IRC included wrapping the entire code with a try/catch and exiting with a :normal exit reason and passing the original error as a tuple member. But I would like to avoid wrapping everything in a giant try/catch, and this would still produce a error_report on a brutal_kill. -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Fri Sep 15 21:20:28 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 15 Sep 2017 21:20:28 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> Message-ID: <562F1C02-3C64-4576-A2A1-498F2456CCF6@licenser.net> True macOS and Windows are both tier 1 platforms for rust (along with linux as 3 rd option). I don?t think it?s very relevant however, as you said yourself neither are server oriented systems (windows might a bit be). But for the majority of users it?d effectively be ?linux or good luck? unless they plan to deploy on a macbook. > On 15. Sep 2017, at 20:01, Sergej Jure?ko wrote: > > >> On 15 Sep 2017, at 19:47, Heinz N. Gies wrote: >> >> The answer would have to be ?a bit? it?s on and off depending on packages available and versions, same goes for BSD sadly. >> >> While rust does make an real effort to run outside of linux it?s very much ?linux or good luck to you my frinbed? these days ? > > Linux or good luck is far from true. Rust works just fine and is well supported on Windows, macOS, iOS and Android. All giant platforms, though user not server oriented. Obviously a consequence of the largest Rust user being Mozilla. > > Regards, > Sergej -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From freza@REDACTED Fri Sep 15 21:36:08 2017 From: freza@REDACTED (Jachym Holecek) Date: Fri, 15 Sep 2017 15:36:08 -0400 Subject: [erlang-questions] Is there a way to prevent a crashing process from sending a message to error_logger? In-Reply-To: <2059779354.2996009.1505501412900@mail.yahoo.com> References: <2059779354.2996009.1505501412900.ref@mail.yahoo.com> <2059779354.2996009.1505501412900@mail.yahoo.com> Message-ID: <20170915193608.GA8190@circlewave.net> # Vans S 2017-09-15: > I am dealing with crashing processes that tend to spam the error logger. > > The crashing processes error reason is being caught and everything is taken care of and > logged via the trapping_exits, spawn_linker. > > Still the crashes spam the error_logger. > > I am wondering if there is a way to set specific processes to not send a message upon > crash to the error_logger? Sure, just avoid the proc_lib wrapper (and its direct consumers) and use plain erlang:spawn_link/X and friends directly -- remember, they're part of the language for a reason and its perfectly alright to be using them, just be prepared to ocassionaly dig a bit deeper and learn one obscure corner of OTP or another every once in a while. If you wish to use one of the standard OTP behaviours, like gen_server and such (they're built on top of proc_lib and gen), have a look whether they provide an 'enter_loop/X' exported function -- that will allow you to still use the generic logic, but arrange process startup your way, ie. no toplevel try/catch sending stuff to error_logger. Note that this will not play well with gen_server's hibernation feature, as return from hibernation will reinstate proc_lib's toplevel try/catch. Of course, you could always write your own custom behaviours doing whatever it takes to integrate well with the rest of your system -- rarely worth the trouble, but otherwise a perfectly fine thing to do as well. So much for addressing your specific question. Wider point to make is that various sources of logging in your system operate at different frequencies and have varying relevance, blindly routing them all into a single sink is asking for an operational disaster of one sort or another -- and this is a mistake users unfortunately seem to run into all too easily. Addresing it properly takes some effort across all components comprising your system, no silver bullets there. HTH, -- Jachym From phil.toland@REDACTED Fri Sep 15 19:14:15 2017 From: phil.toland@REDACTED (Phil Toland) Date: Fri, 15 Sep 2017 12:14:15 -0500 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: I second the vote for Alpine Linux. My application is a mix of Erlang and Elixir and runs in Docker containers. I use Alpine as the base and I get reasonably sized containers that don't have anything I don't need. As a side note, it isn't just about providing the necessary support to run the BEAM VM. I also want enough of a familiar environment (ls, ps, top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without the other stuff that you don't need. ~phil On September 15, 2017 at 9:47:00 AM, felixgallo@REDACTED ( felixgallo@REDACTED) wrote: FreeBSD and alpine Linux have both served me well. F. On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice wrote: Hi, Joe Armstrong states in his 2003 Doctor of Technology thesis: "Our system has very little need of an operating system. We make use of very few operating system services, thus it is relatively easy to port our system to specialised environments such as embedded systems." As a one-time Forth developer, software bloat offends me. I sigh and use Ubuntu on my development system out laziness and convenience. But I would like my production servers to be lean and mean. What light weight open-source off-the shelf operating systems would battle-hardened Erlang gurus recommend? All the best, LRP Sent from my iPad _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From phil.toland@REDACTED Fri Sep 15 19:37:36 2017 From: phil.toland@REDACTED (Phil Toland) Date: Fri, 15 Sep 2017 12:37:36 -0500 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: Message-ID: I have some experience with Cassandra, and based on your description of your needs it sounds like Cassandra would be overkill. I think you could make it work, but it may not be worth the effort. Cassandra uses a ring architecture similar to Riak. Reads and writes happen on multiple nodes and the entire transaction doesn't succeed until it has succeeded on a certain number of nodes. For your read-heavy workload you would probably want to set that value higher for writes and lower for reads. For example, you might say that a write happens on 3 nodes and succeeds when it is successful on all of those nodes. You might feel safe then saying that a read only needs to succeed on one node to be successful. This is something you would definitely want to test thoroughly so that you understand the performance tradeoffs of changing those values. Also, recent versions of Cassandra have adopted a table model and query language (CQL) that are superficially similar to RDBMS tables and SQL, but are actually completely different. This led to a lot of cognitive dissonance for my team as we would do things that made sense for an RDBMS, and that we could express in CQL, but were totally the wrong thing to do for Cassandra's architecture. Personally, I would look at Riak before Cassandra if you think that the ring architecture makes sense for you. Because it doesn't have the trappings of tables and a SQL-like language, we found it much more straightforward to reason about the strengths and limitations of the system. It is very much a straightforward key/value data store. However, you mentioned read consistency being important, and Cassandra and Riak both trade off read consistency for availability. They are "eventually consistent" systems (https://en.wikipedia.org/wiki/Eventual_consistency). I suggest you read up on the CAP theorem ( https://en.wikipedia.org/wiki/CAP_theorem) and decide what tradeoffs you are willing to make before choosing a database. Good luck! ~phil On September 15, 2017 at 11:43:52 AM, code wiget (codewiget95@REDACTED) wrote: Hello everyone, I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. I have looked at as many options as I can -> Voldemort : http://project-voldemort.com/ - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. Cassandra: http://cassandra.apache.org/ - looks good too, but apparently there is a small community and apparently isn?t updated often Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. So from my research, which consisted heavily of this blog: https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? Thanks for your time. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From b@REDACTED Fri Sep 15 19:56:33 2017 From: b@REDACTED (Bence Golda) Date: Fri, 15 Sep 2017 19:56:33 +0200 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: <9752DBE8-9168-41F9-A760-A9488506F3E0@licenser.net> References: <9752DBE8-9168-41F9-A760-A9488506F3E0@licenser.net> Message-ID: <5d0e0773-a18b-e08b-5c45-c959b18819fb@blorb.io> ...or Barrel-DB: https://barrel-db.org/ ? BR, Bence On 09/15/2017 19:48, Heinz N. Gies wrote: > Have you considered Risk? It?s a distributed erlang nosql/k/v store. >> On 15. Sep 2017, at 18:43, code wiget > > wrote: >> >> Hello everyone, >> >> I am at the point where I have many Erlang nodes, and I am going to >> have to move to a distributed database. Right now, I am using a basic >> setup: each Erlang node has a copy of the same Redis DB, and all of >> those DBs are slaves(non-writable copies) of a master. A big problem >> with this is obvious - If the db goes down, the node goes down. If the >> master goes down, the slaves won?t get updated, so I would like to >> move to a distributed db that all of my nodes can read/write to that >> can not/does not go down. >> >> The nodes do ~50 reads per write, and are constantly reading, so read >> speed and consistency is my real concern. I believe this will be the >> node?s main speed factor. >> >> Another thing is that all of my data is key/key/value , so it would >> mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t >> need a SQL DB. >> >> A big thing also is that I don?t need disc copies, as a I have a large >> backup store where the values are generated from. >> >> I have looked at as many options as I can -> >> >> Voldemort : http://project-voldemort.com/ >> - looks perfect, but there are 0 resources on learning how to use it >> outside of their docs and no Erlang driver, which is huge because I >> would both have to learn how to write a c driver and everything about >> this just to get it to work. >> >> Cassandra: http://cassandra.apache.org/ >> - looks good too, but apparently there is a small community and >> apparently isn?t updated often >> >> Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf >> - Looks very very cool, seems great, but there is 0 active community >> and their GitHub isn?t updated often. This is a distributed all >> in-memory database, written in Erlang. >> >> >> So from my research, which consisted heavily of this >> blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , >> I have narrowed it down to these three. >> >> BUT you are all the real experts and have built huge applications in >> Erlang, what do you use? What do you have experience in that performs >> well with Erlang nodes spread across multiple machines and possibly >> multiple data centers? >> >> Thanks for your time. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From t@REDACTED Fri Sep 15 22:48:01 2017 From: t@REDACTED (Tristan Sloughter) Date: Fri, 15 Sep 2017 13:48:01 -0700 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> Same, and I have an example multi-stage docker file for building the image on Alpine for an example project: https://github.com/SpaceTime-IoT/presence-sample/blob/master/Dockerfile It also uses all the options relx supports to shrink the size of the tarball https://github.com/SpaceTime-IoT/presence-sample/blob/master/rebar.config#L26-L34 One feature relx doesn't yet support that may help a little with size of the unpacked release would be compressed applications (.ez). Pretty sure that is what reltool would do by default? Not sure how much it saves. Also, it is possible to use docker scratch* to have no real base image, but with Alpine being so small I don't think it saves enough to be worth it -- esp considering everything you then don't have, as Phil mentioned. * And some hacks, copying over libs and a linker. Or a statically linked Erlang? Never done it with a statically linked Erlang, would that work with SSL? -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Fri, Sep 15, 2017, at 10:14 AM, Phil Toland wrote: > I second the vote for Alpine Linux. My application is a mix of Erlang > and Elixir and runs in Docker containers. I use Alpine as the base > and I get reasonably sized containers that don't have anything I > don't need.> > As a side note, it isn't just about providing the necessary support to > run the BEAM VM. I also want enough of a familiar environment (ls, ps, > top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides > that without the other stuff that you don't need.> > ~phil > > > > On September 15, 2017 at 9:47:00 AM, felixgallo@REDACTED > (felixgallo@REDACTED) wrote:>> >> >> >> FreeBSD and alpine Linux have both served me well. >> >> F. >> >> On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice >> wrote:>>> Hi, >>> Joe Armstrong states in his 2003 Doctor of Technology thesis: >>> "Our system has very little need of an operating system. We make use >>> of very few operating system services, thus it is relatively easy to >>> port our system to specialised environments such as embedded >>> systems.">>> As a one-time Forth developer, software bloat offends me. I sigh and >>> use Ubuntu on my development system out laziness and convenience.>>> But I would like my production servers to be lean and mean. >>> What light weight open-source off-the shelf operating systems would >>> battle-hardened Erlang gurus recommend?>>> All the best, >>> LRP >>> >>> Sent from my iPad >>> _______________________________________________ >>> erlang-questions mailing list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions>> _______________________________________________ >> erlang-questions mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions>> > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Fri Sep 15 23:04:30 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Fri, 15 Sep 2017 17:04:30 -0400 (EDT) Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> Message-ID: <1505509470.223524977@apps.rackspace.com> Thanks, all. Good ideas to pursue. All the best, Lloyd -----Original Message----- From: "Tristan Sloughter" Sent: Friday, September 15, 2017 4:48pm To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Same, and I have an example multi-stage docker file for building the image on Alpine for an example project: https://github.com/SpaceTime-IoT/presence-sample/blob/master/Dockerfile It also uses all the options relx supports to shrink the size of the tarball https://github.com/SpaceTime-IoT/presence-sample/blob/master/rebar.config#L26-L34 One feature relx doesn't yet support that may help a little with size of the unpacked release would be compressed applications (.ez). Pretty sure that is what reltool would do by default? Not sure how much it saves. Also, it is possible to use docker scratch* to have no real base image, but with Alpine being so small I don't think it saves enough to be worth it -- esp considering everything you then don't have, as Phil mentioned. * And some hacks, copying over libs and a linker. Or a statically linked Erlang? Never done it with a statically linked Erlang, would that work with SSL? -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Fri, Sep 15, 2017, at 10:14 AM, Phil Toland wrote: > I second the vote for Alpine Linux. My application is a mix of Erlang > and Elixir and runs in Docker containers. I use Alpine as the base > and I get reasonably sized containers that don't have anything I > don't need.> > As a side note, it isn't just about providing the necessary support to > run the BEAM VM. I also want enough of a familiar environment (ls, ps, > top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides > that without the other stuff that you don't need.> > ~phil > > > > On September 15, 2017 at 9:47:00 AM, felixgallo@REDACTED > (felixgallo@REDACTED) wrote:>> >> >> >> FreeBSD and alpine Linux have both served me well. >> >> F. >> >> On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice >> wrote:>>> Hi, >>> Joe Armstrong states in his 2003 Doctor of Technology thesis: >>> "Our system has very little need of an operating system. We make use >>> of very few operating system services, thus it is relatively easy to >>> port our system to specialised environments such as embedded >>> systems.">>> As a one-time Forth developer, software bloat offends me. I sigh and >>> use Ubuntu on my development system out laziness and convenience.>>> But I would like my production servers to be lean and mean. >>> What light weight open-source off-the shelf operating systems would >>> battle-hardened Erlang gurus recommend?>>> All the best, >>> LRP >>> >>> Sent from my iPad >>> _______________________________________________ >>> erlang-questions mailing list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions>> _______________________________________________ >> erlang-questions mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions>> > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vans_163@REDACTED Fri Sep 15 23:25:00 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 15 Sep 2017 21:25:00 +0000 (UTC) Subject: [erlang-questions] Is there a way to prevent a crashing process from sending a message to error_logger? In-Reply-To: <20170915193608.GA8190@circlewave.net> References: <2059779354.2996009.1505501412900.ref@mail.yahoo.com> <2059779354.2996009.1505501412900@mail.yahoo.com> <20170915193608.GA8190@circlewave.net> Message-ID: <64824957.3087401.1505510700621@mail.yahoo.com> I tried this, infact the throwaway processes I am spawning are not OTP behaviors but just spawn_linked processes. 1> spawn(fun()-> 1/0 end).<0.63.0>2>?=ERROR REPORT==== 15-Sep-2017::17:24:08 ===Error in process <0.63.0> with exit value:{badarith,[{erlang,'/',[1,0],[]}]} Regardless I get an error report. On Friday, September 15, 2017 3:34 PM, Jachym Holecek wrote: # Vans S 2017-09-15: > I am dealing with crashing processes that tend to spam the error logger. > > The crashing processes error reason is being caught and everything is taken care of and > logged via the trapping_exits, spawn_linker. > > Still the crashes spam the error_logger. > > I am wondering if there is a way to set specific processes to not send a message upon > crash to the error_logger? Sure, just avoid the proc_lib wrapper (and its direct consumers) and use plain erlang:spawn_link/X and friends directly -- remember, they're part of the language for a reason and its perfectly alright to be using them, just be prepared to ocassionaly dig a bit deeper and learn one obscure corner of OTP or another every once in a while. If you wish to use one of the standard OTP behaviours, like gen_server and such (they're built on top of proc_lib and gen), have a look whether they provide an 'enter_loop/X' exported function -- that will allow you to still use the generic logic, but arrange process startup your way, ie. no toplevel try/catch sending stuff to error_logger. Note that this will not play well with gen_server's hibernation feature, as return from hibernation will reinstate proc_lib's toplevel try/catch. Of course, you could always write your own custom behaviours doing whatever it takes to integrate well with the rest of your system -- rarely worth the trouble, but otherwise a perfectly fine thing to do as well. So much for addressing your specific question. Wider point to make is that various sources of logging in your system operate at different frequencies and have varying relevance, blindly routing them all into a single sink is asking for an operational disaster of one sort or another -- and this is a mistake users unfortunately seem to run into all too easily. Addresing it properly takes some effort across all components comprising your system, no silver bullets there. HTH, ??? -- Jachym -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Fri Sep 15 23:25:10 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 15 Sep 2017 21:25:10 +0000 (UTC) Subject: [erlang-questions] Is there a way to prevent a crashing process from sending a message to error_logger? In-Reply-To: <20170915193608.GA8190@circlewave.net> References: <2059779354.2996009.1505501412900.ref@mail.yahoo.com> <2059779354.2996009.1505501412900@mail.yahoo.com> <20170915193608.GA8190@circlewave.net> Message-ID: <285516687.2403250.1505510710378@mail.yahoo.com> I tried this, infact the throwaway processes I am spawning are not OTP behaviors but just spawn_linked processes. 1> spawn(fun()-> 1/0 end).<0.63.0>2>?=ERROR REPORT==== 15-Sep-2017::17:24:08 ===Error in process <0.63.0> with exit value:{badarith,[{erlang,'/',[1,0],[]}]} Regardless I get an error report. On Friday, September 15, 2017 3:34 PM, Jachym Holecek wrote: # Vans S 2017-09-15: > I am dealing with crashing processes that tend to spam the error logger. > > The crashing processes error reason is being caught and everything is taken care of and > logged via the trapping_exits, spawn_linker. > > Still the crashes spam the error_logger. > > I am wondering if there is a way to set specific processes to not send a message upon > crash to the error_logger? Sure, just avoid the proc_lib wrapper (and its direct consumers) and use plain erlang:spawn_link/X and friends directly -- remember, they're part of the language for a reason and its perfectly alright to be using them, just be prepared to ocassionaly dig a bit deeper and learn one obscure corner of OTP or another every once in a while. If you wish to use one of the standard OTP behaviours, like gen_server and such (they're built on top of proc_lib and gen), have a look whether they provide an 'enter_loop/X' exported function -- that will allow you to still use the generic logic, but arrange process startup your way, ie. no toplevel try/catch sending stuff to error_logger. Note that this will not play well with gen_server's hibernation feature, as return from hibernation will reinstate proc_lib's toplevel try/catch. Of course, you could always write your own custom behaviours doing whatever it takes to integrate well with the rest of your system -- rarely worth the trouble, but otherwise a perfectly fine thing to do as well. So much for addressing your specific question. Wider point to make is that various sources of logging in your system operate at different frequencies and have varying relevance, blindly routing them all into a single sink is asking for an operational disaster of one sort or another -- and this is a mistake users unfortunately seem to run into all too easily. Addresing it properly takes some effort across all components comprising your system, no silver bullets there. HTH, ??? -- Jachym -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Fri Sep 15 23:27:23 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 15 Sep 2017 21:27:23 +0000 (UTC) Subject: [erlang-questions] Is there a way to prevent a crashing process from sending a message to error_logger? In-Reply-To: <64824957.3087401.1505510700621@mail.yahoo.com> References: <2059779354.2996009.1505501412900.ref@mail.yahoo.com> <2059779354.2996009.1505501412900@mail.yahoo.com> <20170915193608.GA8190@circlewave.net> <64824957.3087401.1505510700621@mail.yahoo.com> Message-ID: <739926671.3069884.1505510843624@mail.yahoo.com> Heh sorry for spam. proc_lib:spawn/1|2|3|4 Seems to do what I need, I will investigate it more! Thanks! On Friday, September 15, 2017 5:25 PM, Vans S wrote: I tried this, infact the throwaway processes I am spawning are not OTP behaviors but just spawn_linked processes. 1> spawn(fun()-> 1/0 end).<0.63.0>2>?=ERROR REPORT==== 15-Sep-2017::17:24:08 ===Error in process <0.63.0> with exit value:{badarith,[{erlang,'/',[1,0],[]}]} Regardless I get an error report. On Friday, September 15, 2017 3:34 PM, Jachym Holecek wrote: # Vans S 2017-09-15: > I am dealing with crashing processes that tend to spam the error logger. > > The crashing processes error reason is being caught and everything is taken care of and > logged via the trapping_exits, spawn_linker. > > Still the crashes spam the error_logger. > > I am wondering if there is a way to set specific processes to not send a message upon > crash to the error_logger? Sure, just avoid the proc_lib wrapper (and its direct consumers) and use plain erlang:spawn_link/X and friends directly -- remember, they're part of the language for a reason and its perfectly alright to be using them, just be prepared to ocassionaly dig a bit deeper and learn one obscure corner of OTP or another every once in a while. If you wish to use one of the standard OTP behaviours, like gen_server and such (they're built on top of proc_lib and gen), have a look whether they provide an 'enter_loop/X' exported function -- that will allow you to still use the generic logic, but arrange process startup your way, ie. no toplevel try/catch sending stuff to error_logger. Note that this will not play well with gen_server's hibernation feature, as return from hibernation will reinstate proc_lib's toplevel try/catch. Of course, you could always write your own custom behaviours doing whatever it takes to integrate well with the rest of your system -- rarely worth the trouble, but otherwise a perfectly fine thing to do as well. So much for addressing your specific question. Wider point to make is that various sources of logging in your system operate at different frequencies and have varying relevance, blindly routing them all into a single sink is asking for an operational disaster of one sort or another -- and this is a mistake users unfortunately seem to run into all too easily. Addresing it properly takes some effort across all components comprising your system, no silver bullets there. HTH, ??? -- Jachym -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sat Sep 16 02:38:13 2017 From: zxq9@REDACTED (zxq9) Date: Sat, 16 Sep 2017 09:38:13 +0900 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: <1622499.SgdCHc1Xva@changa> On 2017?09?15? ??? 10:20:15 Lloyd R. Prentice wrote: > As a one-time Forth developer, software bloat offends me. I sigh and use Ubuntu on my development system out laziness and convenience. > > But I would like my production servers to be lean and mean. > > What light weight open-source off-the shelf operating systems would battle-hardened Erlang gurus recommend? I've found success getting very lean, highly efficient systems using Gentoo. It is about halfway between being "off-the-shelf" and a prepackaged system like the others. Gentoo is a meta distribution, not really a distribution, so you choose exactly what you want your custom distro to look like and have it built that way. For systems where I've chosen the hardware (or virtual host environment) and I'm going to run Erlang only that means: - The kernel modules necessary to drive the hardware - The basics: coreutils, bash, vim - Erlang dependencies: enough of gcc, openssl, etc. to build Erlang properly - Extra utilities the system might depend on: Postgres, Mercury, a graph db, etc. Beyond that, as long as I write everything else in Erlang (independent services like Postgres aside) then my from source system retrieves and runs the latest version of everything. As long as I stick to the same hardware I have the Gentoo profile continue to build up to date binaries whenever something changes and distribute those to the service hosts. I've found this to be a really good fit for situations where you have 1-5 systems or >100 systems. With just a few systems you can have the systems rebuild themselves, maintaining it occasionally is not a hassle, and setting up a binary repo isn't worth it. With hundreds or thousands of hosts you will have a sysop who can keep an eye on things, setting up a binary repo is totally worth it, and everything just sort of works in a way that operations and development don't have to be intimate (host maintenance takes very little time for the systems guys in this case once things are set up). Where Gentoo really doesn't fit is that fuzzy space where you have just enough systems to maintain that it is a painful and distracting task but not enough to make having dedicated sysop support necessary. Unfortunately, most projects fit into that uncomfortable middle size, and so we see a proliferation of wonky partial solutions to handle that semi-scaled space. In my experience there is no reasonable way to discuss how lean the system is going to be without addressing how lean the support structure for that system is going to be. In the end we have to write our systems to handle human problems, and that means also addressing human concerns on the back end. Sometimes being lazy and just tossing Debian or Ubuntu out there is really the right solution -- to the point that I favor VMs over containers because nobody has to learn or keep up with any quirks and that's usually the biggest time sink, not execution efficiency. Take care to write your system in a way that you don't wind up marrying a particular flavor of hose unless you have to (and sometimes you have to, at least on the client side). I've found so far that if I write the core system in pure Erlang (avoiding native code) I have zero problems changing the host later -- and that gives me quite a lot of freedom. It has been very rare that a performance issue was so crushingly problematic that I needed NIFs or port drivers -- usually I need a system written in something totally different that I can talk to over the wire. Using the network socket as your universal abstraction is immensely powerful in this regard, allowing you to experiment with whatever platform whims you come up with in the future. My 2 bucks. -Craig From erlang@REDACTED Sat Sep 16 11:29:03 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 16 Sep 2017 11:29:03 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> Message-ID: Old timer here :-) Just out of interest how large is Apline? (in MB?) Just out of interest the last time I made a minimal erlang is ended up at 4.7 MB - with the following files (and sizes) > /Dropbox/experiments/erlang_embedded/tiny_erl_19.1 > tree -s ??? [ 11859] COPYRIGHT ??? [ 238] bin ? ??? [ 2827272] beam.smp ? ??? [ 22580] erl_child_setup ? ??? [ 53548] erlexec ? ??? [ 5479] start.boot ? ??? [ 6459] start.script ??? [ 896] erl ??? [ 204] lib ??? [ 429496] compiler-7.0.2.ez ??? [ 387430] kernel-5.1.ez ??? [ 109322] sasl-3.0.1.ez ??? [ 806013] stdlib-3.1.ez The biggest file is beam.smp -- I guess with a little static analysis many of the files in stdlib and kernel could be removed. In the 'old days' the size to beat was 1.44 MB (ie a floppy disk) - we did actually have an entire OS + Erlang that fitted onto a single floppy disk. I also recall a time when linux kernels were so small that a knowledgeable person knew what every file in the system did. I also believe that "small is fast and secure" loading/storing/cacheing MBs of *anything* is a thousand times faster than loading/storing/caching GBs. Security is possible (though difficult) with MBs of code - and I guess impossible with GB Cheers /Joe On Fri, Sep 15, 2017 at 10:48 PM, Tristan Sloughter wrote: > Same, and I have an example multi-stage docker file for building the image > on Alpine for an example project: > https://github.com/SpaceTime-IoT/presence-sample/blob/master/Dockerfile It > also uses all the options relx supports to shrink the size of the tarball > https://github.com/SpaceTime-IoT/presence-sample/blob/master/rebar.config#L26-L34 > > One feature relx doesn't yet support that may help a little with size of the > unpacked release would be compressed applications (.ez). Pretty sure that is > what reltool would do by default? Not sure how much it saves. > > Also, it is possible to use docker scratch* to have no real base image, but > with Alpine being so small I don't think it saves enough to be worth it -- > esp considering everything you then don't have, as Phil mentioned. > > * And some hacks, copying over libs and a linker. Or a statically linked > Erlang? Never done it with a statically linked Erlang, would that work with > SSL? > > -- > Tristan Sloughter > "I am not a crackpot" - Abe Simpson > t@REDACTED > > > On Fri, Sep 15, 2017, at 10:14 AM, Phil Toland wrote: > > I second the vote for Alpine Linux. My application is a mix of Erlang and > Elixir and runs in Docker containers. I use Alpine as the base and I get > reasonably sized containers that don't have anything I don't need. > > As a side note, it isn't just about providing the necessary support to run > the BEAM VM. I also want enough of a familiar environment (ls, ps, top, > nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without > the other stuff that you don't need. > > ~phil > > > > On September 15, 2017 at 9:47:00 AM, felixgallo@REDACTED > (felixgallo@REDACTED) wrote: > > > > > FreeBSD and alpine Linux have both served me well. > > F. > > On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice > wrote: > > Hi, > > Joe Armstrong states in his 2003 Doctor of Technology thesis: > > "Our system has very little need of an operating system. We make use of very > few operating system services, thus it is relatively easy to port our system > to specialised environments such as embedded systems." > > As a one-time Forth developer, software bloat offends me. I sigh and use > Ubuntu on my development system out laziness and convenience. > > But I would like my production servers to be lean and mean. > > What light weight open-source off-the shelf operating systems would > battle-hardened Erlang gurus recommend? > > All the best, > > LRP > > > Sent from my iPad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Sat Sep 16 12:03:40 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 16 Sep 2017 12:03:40 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <562F1C02-3C64-4576-A2A1-498F2456CCF6@licenser.net> References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> <562F1C02-3C64-4576-A2A1-498F2456CCF6@licenser.net> Message-ID: Reimplementing Erlang in Rust would be a huge task if you want 100% compatibility Erlang is several things 1) A programming language 2) A huge set of libraries 3) A large set of tools The language can be implemented as a byte code emulator (or threaded word emulator) + a load of BIFs (or NIFs) The emulator bit is easy (apart from the GC and process management) - the problem is all the BIFs and NIFs that are not written in Erlang but written in C. If I was going to re-implement Erlang I'd try to define a subset of Erlang with which the entire emulator GC and process management code could be written. This subset would be such that it could be cross-compiled to C (or Rust, or anything) The goal would be to write MORE in Erlang not less - unfortunately the trend of the last few years has been to write LESS in Erlang and more in C (ie with NIFS) - this improves performance but makes reimplementation far more difficult. The path we took in the early 1990's was to make erlang more performant at the expense of portability. So we moved from a byte coded machine (JAM) to a threaded word interpretor (BEAM) - the machine got bigger not smaller. What we never tried was moving in the opposite direction of making the machine smaller and more portable - I'm thinking an ANSI C interpreter and huffman encoded instructions. In the 1990's this would have been far slower than the JAM - but today we're talking GHz whereas it was MHz in the 1990's. It would be great tor re-engineer Erlang, and to take the opportunity to make it more portable and smaller. If we had in mind the goals "very small", "very portable", "as much as possible in erlang, as little as is necessary in C/rust" we might end up with a nice machine. I think the gains would come from a careful re-design of the VM not changing the machine in which the VM is implemented. I believe the Lua VM has 37 instructions and a small emulator. The trick is having a VM with correct instructions for implementing Erlang . The JVM and .NET have the wrong instructions - the JAM and BEAM have (or had) single instructions for spawning processes and sending messages - I guess these could be augmented with Opcodes for micro GCs and process management. I'm not convinced that a better programming language to implement the VM helps - the tricky bit is getting the VM machine instructions correct - at this level of abstration the VM is just moving memory around comparing things and jumping around - which is pretty easy inn *any* programming language. In my opinion the great progenitor of VMs was the Pascal P-code machine (from which .NET and JVM derive) - then came the WAM (for Prolog) then we added opcodes for process spawning and message passing (JAM) and turned this into threaded code (BEAM) . What would be an interesting experiment would be re-engineer the VM turning all the things essential to Erlang and Elixir into opcodes and make an emulator for this. The emulator should (or course) we written in a dialect of Erlang and cross-compiled to C (or Rust() Note: This is how we didn't implement Erlang. I designed and implemented the original VM and opcodes in Prolog - then Mike William re-implemented the Prolog emulator in C. The mistake here was that Prolog has built-in GC and C does not - so there were no instructions in the JAM for GC (just an instruction so say when the stack and heap were in a safe state for GC to occur) By all means re-implement Erlang - but make it smaller and more portable, and forget the NIFs Cheers /Joe On Fri, Sep 15, 2017 at 9:20 PM, Heinz N. Gies wrote: > True macOS and Windows are both tier 1 platforms for rust (along with linux as 3 rd option). I don?t think it?s very relevant however, as you said yourself neither are server oriented systems (windows might a bit be). But for the majority of users it?d effectively be ?linux or good luck? unless they plan to deploy on a macbook. > >> On 15. Sep 2017, at 20:01, Sergej Jure?ko wrote: >> >> >>> On 15 Sep 2017, at 19:47, Heinz N. Gies wrote: >>> >>> The answer would have to be ?a bit? it?s on and off depending on packages available and versions, same goes for BSD sadly. >>> >>> While rust does make an real effort to run outside of linux it?s very much ?linux or good luck to you my frinbed? these days ? >> >> Linux or good luck is far from true. Rust works just fine and is well supported on Windows, macOS, iOS and Android. All giant platforms, though user not server oriented. Obviously a consequence of the largest Rust user being Mozilla. >> >> Regards, >> Sergej > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From n.oxyde@REDACTED Sat Sep 16 13:18:42 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 16 Sep 2017 13:18:42 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> <562F1C02-3C64-4576-A2A1-498F2456CCF6@licenser.net> Message-ID: This is absolutely 100% wrong. This kind of code *is* hard to get correct. What about out of bound accesses or uses after free or data races like the bug I mentioned? -- Anthony Ramine > Le 16 sept. 2017 ? 12:03, Joe Armstrong a ?crit : > > I'm not convinced that a better programming language to implement the VM helps - > the tricky bit is getting the VM machine instructions correct - at > this level of abstration > the VM is just moving memory around comparing things and jumping > around - which is pretty > easy inn *any* programming language. From torres.anderson.85@REDACTED Sat Sep 16 14:42:41 2017 From: torres.anderson.85@REDACTED (Anderson Torres) Date: Sat, 16 Sep 2017 09:42:41 -0300 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <9714375.VumhWqLY8o@changa> <5FA6A0CB-B2C4-46CB-82EF-ED36A2FE8070@gmail.com> <43B52049-B656-40A9-91CB-8F7B50EC2E75@licenser.net> <597650A7-ED93-4D91-B61F-A9B0DF466506@gmail.com> <3DCF52E3-465C-4C15-946B-173B5C83CCEF@gmail.com> <562F1C02-3C64-4576-A2A1-498F2456CCF6@licenser.net> Message-ID: 2017-09-16 8:18 GMT-03:00 Anthony Ramine : > This is absolutely 100% wrong. This kind of code *is* hard to get correct. What about out of bound accesses or uses after free or data races like the bug I mentioned? > > -- > Anthony Ramine > >> Le 16 sept. 2017 ? 12:03, Joe Armstrong a ?crit : >> >> I'm not convinced that a better programming language to implement the VM helps - >> the tricky bit is getting the VM machine instructions correct - at >> this level of abstration >> the VM is just moving memory around comparing things and jumping >> around - which is pretty >> easy inn *any* programming language. It can be easy to implement, but ensure correctness is another thing. As A. Ramine said, data races can surge almost from everywhere, and even silly bugs as use-after-free. If Rust can eliminate these things, it can be worth a looking, and if not a full, from-scratch replacement, something more incremental: the interpreters first, the "operating system" part after, and the libraries can be changed on demand. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t@REDACTED Sat Sep 16 17:45:30 2017 From: t@REDACTED (Tristan Sloughter) Date: Sat, 16 Sep 2017 08:45:30 -0700 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> Message-ID: <1505576730.3740175.1108243776.25BE65B8@webmail.messagingengine.com> Alpine is 4-5MB (no kernel). And yea, in a target system much of the space is taken up by beam.smp. On a default install that is like 19M. I've never tried building a smaller beam.smp :) -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Sat, Sep 16, 2017, at 02:29 AM, Joe Armstrong wrote: > Old timer here :-) > > Just out of interest how large is Apline? (in MB?) > > Just out of interest the last time I made a minimal erlang is ended up > at 4.7 MB - > with the following files (and sizes) > > > /Dropbox/experiments/erlang_embedded/tiny_erl_19.1 > tree -s > > ??? [ 11859] COPYRIGHT > ??? [ 238] bin > ? ??? [ 2827272] beam.smp > ? ??? [ 22580] erl_child_setup > ? ??? [ 53548] erlexec > ? ??? [ 5479] start.boot > ? ??? [ 6459] start.script > ??? [ 896] erl > ??? [ 204] lib > ??? [ 429496] compiler-7.0.2.ez > ??? [ 387430] kernel-5.1.ez > ??? [ 109322] sasl-3.0.1.ez > ??? [ 806013] stdlib-3.1.ez > > The biggest file is beam.smp -- I guess with a little static analysis > many of the files > in stdlib and kernel could be removed. > > In the 'old days' the size to beat was 1.44 MB (ie a floppy disk) - we > did actually have > an entire OS + Erlang that fitted onto a single floppy disk. > > I also recall a time when linux kernels were so small that a > knowledgeable person knew what every file in the system did. > > I also believe that "small is fast and secure" > loading/storing/cacheing MBs of *anything* is > a thousand times faster than loading/storing/caching GBs. > > Security is possible (though difficult) with MBs of code - and I guess > impossible with GB > > Cheers > > /Joe > > > On Fri, Sep 15, 2017 at 10:48 PM, Tristan Sloughter > wrote: > > Same, and I have an example multi-stage docker file for building the image > > on Alpine for an example project: > > https://github.com/SpaceTime-IoT/presence-sample/blob/master/Dockerfile It > > also uses all the options relx supports to shrink the size of the tarball > > https://github.com/SpaceTime-IoT/presence-sample/blob/master/rebar.config#L26-L34 > > > > One feature relx doesn't yet support that may help a little with size of the > > unpacked release would be compressed applications (.ez). Pretty sure that is > > what reltool would do by default? Not sure how much it saves. > > > > Also, it is possible to use docker scratch* to have no real base image, but > > with Alpine being so small I don't think it saves enough to be worth it -- > > esp considering everything you then don't have, as Phil mentioned. > > > > * And some hacks, copying over libs and a linker. Or a statically linked > > Erlang? Never done it with a statically linked Erlang, would that work with > > SSL? > > > > -- > > Tristan Sloughter > > "I am not a crackpot" - Abe Simpson > > t@REDACTED > > > > > > On Fri, Sep 15, 2017, at 10:14 AM, Phil Toland wrote: > > > > I second the vote for Alpine Linux. My application is a mix of Erlang and > > Elixir and runs in Docker containers. I use Alpine as the base and I get > > reasonably sized containers that don't have anything I don't need. > > > > As a side note, it isn't just about providing the necessary support to run > > the BEAM VM. I also want enough of a familiar environment (ls, ps, top, > > nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without > > the other stuff that you don't need. > > > > ~phil > > > > > > > > On September 15, 2017 at 9:47:00 AM, felixgallo@REDACTED > > (felixgallo@REDACTED) wrote: > > > > > > > > > > FreeBSD and alpine Linux have both served me well. > > > > F. > > > > On Sep 15, 2017, at 7:20 AM, Lloyd R. Prentice > > wrote: > > > > Hi, > > > > Joe Armstrong states in his 2003 Doctor of Technology thesis: > > > > "Our system has very little need of an operating system. We make use of very > > few operating system services, thus it is relatively easy to port our system > > to specialised environments such as embedded systems." > > > > As a one-time Forth developer, software bloat offends me. I sigh and use > > Ubuntu on my development system out laziness and convenience. > > > > But I would like my production servers to be lean and mean. > > > > What light weight open-source off-the shelf operating systems would > > battle-hardened Erlang gurus recommend? > > > > All the best, > > > > LRP > > > > > > Sent from my iPad > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From ebegumisa@REDACTED Sat Sep 16 19:39:50 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 17 Sep 2017 03:39:50 +1000 Subject: [erlang-questions] Frequency of module reloading and the constant pool Message-ID: Hello all, (Not new here--just haven't posted in a while!) I'm using merl to dynamically generate and regenerate modules at runtime that make use of the ERTS constant pool in order to efficiently avail data to all processes. I'm aware that this technique is known amongst more experienced Erlangers for situations where the data is updated infrequently (e.g. sharing global configuration parameters). The question I'm investigating is how frequent is "infrequent" w.r.t reloading of modules[1]? I'm not entirely sure about the costs involved when (re)loading a module so I've got no idea where to start in order to measure the impact on the emulator. Could anybody with sufficient knowledge of emulator internals point me in the right direction? Thanks in advance. - Edmond - [1] The more exotic use-cases for use/abuse of the constant pool and module reloading I'm looking into are to do with caching and incremental computation. -- Using Opera's mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sat Sep 16 22:28:37 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 17 Sep 2017 06:28:37 +1000 Subject: [erlang-questions] dialyzer -- persisting an initial plt combined from many disjoint plts Message-ID: Hello All, With dialyzer, it's possible to incrementally join persisted plts. That is: to persist an initial plt created from an existing plt using the --add_to_plt and --plt options, then use that persisted initial plt for analysis like so... [stdlib-kernel-erts.plt already exists] $ dialyzer --add_to_plt --apps ssl public_key --plt stdlib-kernel-erts.plt --output_plt ssl-public_key-stdlib-kernel-erts.plt $ dialyzer --apps my_app --plt ssl-public_key-stdlib-kernel-erts.plt Usefully, it's also possible to use disjoint plts directly. That is: to analyze a program using a non-persisted initial plt combined from many disjoint persisted plts using the --plts option like so... [(ssl|public_key|stdlib|kernel|erts).disjoint.plt already exist] $ dialyzer --apps my_app --plts ssl.disjoint.plt public_key.disjoint.plt stdlib.disjoint.plt kernel.disjoint.plt erts.disjoint.plt Is it possible to persist an initial plt combined from many disjoint persisted plts? That is: to persist the initial plt in that last command? - Edmond - -- Using Opera's mail client: http://www.opera.com/mail/ From scott_ribe@REDACTED Sat Sep 16 23:25:46 2017 From: scott_ribe@REDACTED (scott ribe) Date: Sat, 16 Sep 2017 15:25:46 -0600 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> On Sep 15, 2017, at 11:14 AM, Phil Toland wrote: > > I second the vote for Alpine Linux. My application is a mix of Erlang and Elixir and runs in Docker containers. I use Alpine as the base and I get reasonably sized containers that don't have anything I don't need. > > As a side note, it isn't just about providing the necessary support to run the BEAM VM. I also want enough of a familiar environment (ls, ps, top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without the other stuff that you don't need. > > ~phil When I looked at Alpine, the Erlang in their repo was too old for me. Do you guys build the releases yourselves, or live with older versions? -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From t@REDACTED Sat Sep 16 23:34:07 2017 From: t@REDACTED (Tristan Sloughter) Date: Sat, 16 Sep 2017 14:34:07 -0700 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> References: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> Message-ID: <1505597647.359845.1108426704.36C173D1@webmail.messagingengine.com> The docker images keep up to date for the latest alpine and Erlang: https://store.docker.com/images/erlang The dockerfile I linked to https://github.com/SpaceTime-IoT/presence-sample/blob/master/Dockerfile currently uses my own base image, but I'll be moving to the official image, and then creates a second image based only on Alpine 3.3.6 (no Erlang) to copy the release to. -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Sat, Sep 16, 2017, at 02:25 PM, scott ribe wrote: > On Sep 15, 2017, at 11:14 AM, Phil Toland wrote: > > > > I second the vote for Alpine Linux. My application is a mix of Erlang and Elixir and runs in Docker containers. I use Alpine as the base and I get reasonably sized containers that don't have anything I don't need. > > > > As a side note, it isn't just about providing the necessary support to run the BEAM VM. I also want enough of a familiar environment (ls, ps, top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without the other stuff that you don't need. > > > > ~phil > > When I looked at Alpine, the Erlang in their repo was too old for me. Do > you guys build the releases yourselves, or live with older versions? > > -- > Scott Ribe > scott_ribe@REDACTED > (303) 722-0567 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From scott_ribe@REDACTED Sat Sep 16 23:36:07 2017 From: scott_ribe@REDACTED (scott ribe) Date: Sat, 16 Sep 2017 15:36:07 -0600 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <1505597647.359845.1108426704.36C173D1@webmail.messagingengine.com> References: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> <1505597647.359845.1108426704.36C173D1@webmail.messagingengine.com> Message-ID: Ah thanks, I never thought to look for Erlang-created images. I started with Alpine's base image, and looked at what Alpine keeps in their repo. -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From ebegumisa@REDACTED Sat Sep 16 23:57:36 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 17 Sep 2017 07:57:36 +1000 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: References: Message-ID: On Sun, 17 Sep 2017 03:39:50 +1000, Edmond Begumisa wrote: > Hello all, > > (Not new here--just haven't posted in a while!) > > I'm using merl to dynamically generate and regenerate modules at runtime > that make use of the ERTS constant pool in order to efficiently avail > data to all processes. I'm aware that this technique is known amongst > more experienced Erlangers for situations where the data is updated > infrequently (e.g. sharing global configuration parameters). > > The question I'm investigating is how frequent is "infrequent" w.r.t > reloading of modules[1]? I'm not entirely sure about the costs involved > when (re)loading a module so I've got no idea where to start in order to > measure the impact on the emulator. Could anybody with sufficient > knowledge of emulator internals point me in the right direction? > To clarify: When measuring whether or not I'm hot-code swapping too frequently, what are the negative signs to look for? What does the emulator actually do when swapping out code w.r.t potentially disruptive/costly operations like locking, GC, etc which could negatively impact scheduling, memory and responsiveness of the system? I'm trying to figure out when the best time to reload a dynamically generated module/group of modules is and how frequently I should allow reloading to occur. - Edmond - > Thanks in advance. > > - Edmond - > > [1] The more exotic use-cases for use/abuse of the constant pool and > module reloading I'm looking into are to do with caching and incremental > computation. > -- Using Opera's mail client: http://www.opera.com/mail/ From list1@REDACTED Sun Sep 17 09:27:54 2017 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 17 Sep 2017 07:27:54 +0000 Subject: [erlang-questions] Tuples referencing each other problem Message-ID: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> Is it possible to create two tuples that reference each other in Erlang? Something like: T1 = {T2} T2 = {T1} My understanding is that when using pointers (i.e. in C++) one needs to mutate the first tuple in order to store the reference to the second tuple once it has been created. This is of course a generalization of a more specific problem of creating liked structures like a double linked list or a tree with leafs containing links to each other. Are there any techniques that would allow creating algorithms operating on such structures in Erlang? GrzegorzJ From krzysztof.jurewicz@REDACTED Sun Sep 17 10:17:16 2017 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Sun, 17 Sep 2017 10:17:16 +0200 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> Message-ID: <87shflsprn.fsf@gmail.com> Grzegorz Junka writes: > Is it possible to create two tuples that reference each other in Erlang? > Something like: > > T1 = {T2} > T2 = {T1} Data structures in Erlang are generally immutable, hence it is not possible to do it in a simple way. To achieve this, you would need to introduce mutability, for example store tuples in a database and reference them by database ids (you may want to look at ETS for this purpose). From mikpelinux@REDACTED Sun Sep 17 10:32:09 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 17 Sep 2017 10:32:09 +0200 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> Message-ID: <22974.13065.472532.95069@gargle.gargle.HOWL> Grzegorz Junka writes: > Is it possible to create two tuples that reference each other in Erlang? > Something like: > > T1 = {T2} > T2 = {T1} > > My understanding is that when using pointers (i.e. in C++) one needs to > mutate the first tuple in order to store the reference to the second > tuple once it has been created. > > This is of course a generalization of a more specific problem of > creating liked structures like a double linked list or a tree with leafs > containing links to each other. Are there any techniques that would > allow creating algorithms operating on such structures in Erlang? Short answer: No, you cannot create circular data structures in Erlang. Erlang "terms" are made up of leafs (pids, ports, numbers, atoms, the empty list [], binaries, refs) and aggregates (list cells, tuples, function closures, maps). Leafs do not refer to other terms, and aggregates are write-once: they're fully initialized when created and then remain immutable. When you see an "update" operation like setelement/2, it actually creates a copy where the target element has been changed, leaving the old term unchanged. There are various ways to simulate assignment: 1. Put labels in your data where you'd otherwise have direct references, and have a mapping from labels to terms. Update that mapping when needed -- but you also have to pass it around so the update is seen by subsequent code. Erlang now has a built-in "map" type that provides such mappings, otherwise there are tree and list based mapping ADTs in the standard library, or you implement your own. 2. Use an ETS table. This is an external (to your process) bag of Erlang terms, where you can insert/lookup/delete terms. This is a stateful object, so inserts/deletes update the existing table rather than creating a copy with that change. You still have to "indirect" via labels, but you avoid the need for passing around an updated mapping. 3. Use a helper process that keeps the state and allows you to query or update it through messages. Semantically this is very similar to an ETS table. 4. You can use the process dictionary, but it's not recommended as it's a shared resource with both the standard libraries and any other code that might have the same idea. From seriy.pr@REDACTED Sun Sep 17 15:05:38 2017 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Sun, 17 Sep 2017 15:05:38 +0200 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] Message-ID: You might be interested in this online tool to inspect BEAM assembler and bytecode: http://tryerl.seriyps.ru/#id=f6f3 Try 'Compile to: Erlang assembler (.S)' or 'Compile to: beam VM assembler (.dis)'. Regards, Sergey -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabriel.alberto.rivas@REDACTED Sun Sep 17 16:56:21 2017 From: gabriel.alberto.rivas@REDACTED (Gabriel Rivas) Date: Sun, 17 Sep 2017 10:56:21 -0400 Subject: [erlang-questions] Understanding BEAM bytecode [Code section] In-Reply-To: References: Message-ID: Thank you, I will check it out. On Sep 17, 2017 9:05 AM, "?????? ????????" wrote: > You might be interested in this online tool to inspect BEAM assembler and > bytecode: > > http://tryerl.seriyps.ru/#id=f6f3 > > Try 'Compile to: Erlang assembler (.S)' or 'Compile to: beam VM assembler > (.dis)'. > > Regards, > Sergey > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Sep 17 21:27:54 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 17 Sep 2017 22:27:54 +0300 Subject: [erlang-questions] Logging from nif (and especially from rust nif) Message-ID: Hi. I'm playing now with rust nif and can advice nice library: https://github.com/hansihe/rustler For those of us, who do not understand what is there about elixir: don't worry, no need to use it, there should be a simple makefile for compiling rust. I've met stange situation while using println! mechanism in rust nif (I'm speaking about enterprise real world debugging: as lot of prints as possible). Nif crashes from println. if you are lucky to catch debug message, you will see something like: stack backtrace: 0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace 1: std::sys_common::backtrace::_print 2: std::panicking::default_hook::{{closure}} 3: std::panicking::default_hook 4: std::panicking::rust_panic_with_hook 5: std::panicking::begin_panic or even info about error code 35. Problem is that stdout is in non-blocking mode and when rust tries to write output, it gets error that it is not waiting for. I've decided not to change anything in rust stdlib, but use another mechanism: 1) take pid of output writing process 2) send messages to it from rust. Here is the way to get whom to send IO request messages: logger() -> UserDrv = whereis(user_drv), CurGroup = case (is_pid(UserDrv) andalso process_info(UserDrv, dictionary)) of {dictionary, UserDrvDict} -> proplists:get_value(current_group, UserDrvDict); _ -> undefined end, case is_pid(CurGroup) of true -> CurGroup; false -> {_,GL} = process_info(self(),group_leader), GL end. and here is how do I log there from rust: if let Some(ref logger) = mpegts.logger { env.pid(); env.send(logger, ( atoms::io_request().encode(env), env.pid().encode(env), atoms::ok().encode(env), ( atoms::put_chars().encode(env), atoms::unicode().encode(env), (string+"\n").encode(env) ).encode(env) ).encode(env)) } One simple hack here: I do not send Mref as 3'rd argument, but I send atom ok. Seems to be working. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Sep 17 21:40:48 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sun, 17 Sep 2017 15:40:48 -0400 (EDT) Subject: [erlang-questions] mnesia question Message-ID: <1505677248.656126192@apps.rackspace.com> Hello, Life gets tricky in the literary world. Some authors use a pen name on their books rather than legal name. But it's quite possible for two different authors to use the same pen name. So pen name is not a good key to associate an author and a book. But ISBN does nicely. Email provides a convenient user name for access to a website. It minimizes the problem of two different people having the same user name. So in my user record I use email as the primary key. But, at creation of the record, I also generate a unique user_id in slot five in the user record. And I create an index to locate the user by user_id. This is handy, because I can use user_id to locate all of the user's books. So one would think that this query would work: get_member_by_member_id(MemberID) when is_list(MemberID)-> Query = fun() -> mnesia:read({?TABLE, MemberID}) end, {atomic, Results} = mnesia:transaction(Query), case length(Results) < 1 of true -> []; false -> hd(Results) end. But it doesn't. It returns an empty list. Not doubt I've forgotten something about mnesia or never learned it. Any help? Many thanks, LRP From chandrashekhar.mullaparthi@REDACTED Sun Sep 17 21:51:52 2017 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 17 Sep 2017 20:51:52 +0100 Subject: [erlang-questions] mnesia question In-Reply-To: <1505677248.656126192@apps.rackspace.com> References: <1505677248.656126192@apps.rackspace.com> Message-ID: Hi Lloyd, If you are not querying by the primary key, you should use mnesia:index_read/3 rather than mnesia:read/1 hth, Chandru On 17 September 2017 at 20:40, wrote: > Hello, > > Life gets tricky in the literary world. Some authors use a pen name on > their books rather than legal name. But it's quite possible for two > different authors to use the same pen name. So pen name is not a good key > to associate an author and a book. But ISBN does nicely. > > Email provides a convenient user name for access to a website. It > minimizes the problem of two different people having the same user name. > > So in my user record I use email as the primary key. But, at creation of > the record, I also generate a unique user_id in slot five in the user > record. And I create an index to locate the user by user_id. This is handy, > because I can use user_id to locate all of the user's books. > > So one would think that this query would work: > > get_member_by_member_id(MemberID) when is_list(MemberID)-> > Query = > fun() -> > mnesia:read({?TABLE, MemberID}) > end, > {atomic, Results} = mnesia:transaction(Query), > case length(Results) < 1 of > true -> []; > false -> hd(Results) > end. > > But it doesn't. It returns an empty list. > > Not doubt I've forgotten something about mnesia or never learned it. > > Any help? > > Many thanks, > > LRP > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Sep 18 02:34:43 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 18 Sep 2017 12:34:43 +1200 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> Message-ID: <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> On 17/09/17 7:27 PM, Grzegorz Junka wrote: > Is it possible to create two tuples that reference each other in Erlang? > Something like: > > T1 = {T2} > T2 = {T1} No. (1) When you create a new term in Erlang it must be fully specified. (2) You cannot change it once it has been created. > My understanding is that when using pointers (i.e. in C++) one needs to > mutate the first tuple in order to store the reference to the second > tuple once it has been created. In Lisp also. Oddly enough, not in Prolog: | ?- T1 = {T2}, T2 = {T1}. T1 = {{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{ Erlang is closer to Prolog than it is to C++, but logical variables are one of the things it did not copy. > > This is of course a generalization of a more specific problem of > creating liked structures like a double linked list or a tree with leafs > containing links to each other. Are there any techniques that would > allow creating algorithms operating on such structures in Erlang? No, and thank goodness! You wouldn't *believe* how much trouble such structures are in Lisp. Or the grief it caused in Prolog. Just as an example, what do you expect term comparison to do? Doubly linked lists are almost always a very bad idea. The back links in the Document Object Model (trees) are a *stunningly* bad idea, because they prevent sharing. (When processing SGML and XML I use a "Document *Value* Model" library I developed years ago. It is so much easier and so much more efficient that it really is not funny.) From ok@REDACTED Mon Sep 18 02:53:27 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 18 Sep 2017 12:53:27 +1200 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> Message-ID: <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> On further reflection, I suppose I should ask: Why is it that you think you need cyclic structures? What do you actually want to *do* with them? As a matter of curiosity, have you read Okasaki's book on functional data structures? (Didn't someone convert Okasaki's Edison library to Erlang at some point? What happened to that?) From ok@REDACTED Mon Sep 18 03:55:45 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 18 Sep 2017 13:55:45 +1200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> Message-ID: <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> On 16/09/17 9:29 PM, Joe Armstrong wrote: > Old timer here :-) > > Just out of interest how large is Apline? (in MB?) For what it's worth, TinyCore-8.0.iso is just under 17 MB. Installed in VirtualBox and running, complete with GUI, it's 21.7MB. That doesn't include development tools. I note that just the "OTP 20.0 HTML Documentation File (33,948,386)" is bigger than this and "OTP 20.0 Windows 32-bit Binary File (96,337,114)" is a lot bigger. http://distro.ibiblio.org/tinycorelinux/intro.html http://distro.ibiblio.org/tinycorelinux/downloads.html The current Core (TinyCore minus GUI &c) is 11 MB. Bringing in clang and its dependencies blows the system out from 21.7 to 145.8 MB (+ 124.1 MB). Adding gettext brought it to 148.0 MB (+ 2.2 MB). Adding the manual pages and related tools brought it up to 152.4 MB (+ 4.4 MB). Add a couple of text editors (NOT including emacs, which I actually like) and it's 161.4 MB (+ 9.0 MB). So the actual TinyCore distribution is about 1/8th of a fairly minimal development environment. I used to run a full SunOS 3 environment on a 100 MB disc, and 40 MB of that was left over for my files. How times change. I suspect that TinyCore plus enough to *run* Erlang could be quite small by today's standards. Certainly you could carry around TinyCore Linux + C development environment + full Erlang distribution and run it comfortably from an elderly memory stick. From lloyd@REDACTED Mon Sep 18 06:28:56 2017 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Mon, 18 Sep 2017 00:28:56 -0400 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: Hi Richard, So, on a server, it sounds like we're talking about: 11 MB for the OS 5 MB for stripped down Erlang based on Joe's work, plus our application code. Thanks to all for these numbers. My question came out of a conversation with a friend re: how far we could push Erlang on a Raspberry Pi or Odroid C2 SBC. It's interesting to look at the numbers for TinyCore on ARM: http://mirror.epn.edu.ec/tinycorelinux/arm.html Compressed, they seem to run in the 25MB range. Raspian zipped is some 4GB. So, on an RP server TinyCore seems to offer much. This makes me think that a cluster of $5.00 RP Zeros would be viable and a great platform for exploring and teaching distributed Erlang. All the best, Lloyd Sent from my iPad > On Sep 17, 2017, at 9:55 PM, Richard A. O'Keefe wrote: > > >> On 16/09/17 9:29 PM, Joe Armstrong wrote: >> Old timer here :-) >> >> Just out of interest how large is Apline? (in MB?) > > For what it's worth, TinyCore-8.0.iso is just under 17 MB. > Installed in VirtualBox and running, complete with GUI, > it's 21.7MB. That doesn't include development tools. > I note that just the > "OTP 20.0 HTML Documentation File (33,948,386)" > is bigger than this and > "OTP 20.0 Windows 32-bit Binary File (96,337,114)" > is a lot bigger. > > http://distro.ibiblio.org/tinycorelinux/intro.html > http://distro.ibiblio.org/tinycorelinux/downloads.html > The current Core (TinyCore minus GUI &c) is 11 MB. > > Bringing in clang and its dependencies blows the system > out from 21.7 to 145.8 MB (+ 124.1 MB). > Adding gettext brought it to 148.0 MB (+ 2.2 MB). > Adding the manual pages and related tools brought > it up to 152.4 MB (+ 4.4 MB). Add a couple of text > editors (NOT including emacs, which I actually like) > and it's 161.4 MB (+ 9.0 MB). > > So the actual TinyCore distribution is about 1/8th of > a fairly minimal development environment. I used to > run a full SunOS 3 environment on a 100 MB disc, and > 40 MB of that was left over for my files. How times change. > > I suspect that TinyCore plus enough to *run* Erlang could > be quite small by today's standards. Certainly you could > carry around TinyCore Linux + C development environment + > full Erlang distribution and run it comfortably from an > elderly memory stick. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott_ribe@REDACTED Mon Sep 18 13:13:30 2017 From: scott_ribe@REDACTED (scott ribe) Date: Mon, 18 Sep 2017 05:13:30 -0600 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: Oh, well, in that case there's one way to get it onto a Raspberry in 12MB: http://nerves-project.org ;-) > On Sep 17, 2017, at 10:28 PM, Lloyd R. Prentice wrote: > > Hi Richard, > > So, on a server, it sounds like we're talking about: > > 11 MB for the OS > 5 MB for stripped down Erlang based on Joe's work, > plus our application code. > > Thanks to all for these numbers. > > My question came out of a conversation with a friend re: how far we could push Erlang on a Raspberry Pi or Odroid C2 SBC. > > It's interesting to look at the numbers for TinyCore on ARM: > > http://mirror.epn.edu.ec/tinycorelinux/arm.html > > Compressed, they seem to run in the 25MB range. Raspian zipped is some 4GB. So, on an RP server TinyCore seems to offer much. > > This makes me think that a cluster of $5.00 RP Zeros would be viable and a great platform for exploring and teaching distributed Erlang. > > All the best, > > Lloyd > > > Sent from my iPad > > On Sep 17, 2017, at 9:55 PM, Richard A. O'Keefe wrote: > >> >> On 16/09/17 9:29 PM, Joe Armstrong wrote: >>> Old timer here :-) >>> >>> Just out of interest how large is Apline? (in MB?) >> >> For what it's worth, TinyCore-8.0.iso is just under 17 MB. >> Installed in VirtualBox and running, complete with GUI, >> it's 21.7MB. That doesn't include development tools. >> I note that just the >> "OTP 20.0 HTML Documentation File (33,948,386)" >> is bigger than this and >> "OTP 20.0 Windows 32-bit Binary File (96,337,114)" >> is a lot bigger. >> >> http://distro.ibiblio.org/tinycorelinux/intro.html >> http://distro.ibiblio.org/tinycorelinux/downloads.html >> The current Core (TinyCore minus GUI &c) is 11 MB. >> >> Bringing in clang and its dependencies blows the system >> out from 21.7 to 145.8 MB (+ 124.1 MB). >> Adding gettext brought it to 148.0 MB (+ 2.2 MB). >> Adding the manual pages and related tools brought >> it up to 152.4 MB (+ 4.4 MB). Add a couple of text >> editors (NOT including emacs, which I actually like) >> and it's 161.4 MB (+ 9.0 MB). >> >> So the actual TinyCore distribution is about 1/8th of >> a fairly minimal development environment. I used to >> run a full SunOS 3 environment on a 100 MB disc, and >> 40 MB of that was left over for my files. How times change. >> >> I suspect that TinyCore plus enough to *run* Erlang could >> be quite small by today's standards. Certainly you could >> carry around TinyCore Linux + C development environment + >> full Erlang distribution and run it comfortably from an >> elderly memory stick. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From phil.toland@REDACTED Mon Sep 18 15:10:34 2017 From: phil.toland@REDACTED (Phil Toland) Date: Mon, 18 Sep 2017 09:10:34 -0400 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> References: <8E77617E-00DA-4052-9DCE-C6CB4AEAE52E@elevated-dev.com> Message-ID: We build the releases. The Erlang and Elixir releases are built in separate base images so that I don't have to rebuild the whole thing when a new Elixir release comes out. However, I may switch to the official images once we get our application onto Erlang 20 and Elixir 1.5. ~p On September 16, 2017 at 4:26:00 PM, scott ribe (scott_ribe@REDACTED) wrote: On Sep 15, 2017, at 11:14 AM, Phil Toland wrote: > > I second the vote for Alpine Linux. My application is a mix of Erlang and Elixir and runs in Docker containers. I use Alpine as the base and I get reasonably sized containers that don't have anything I don't need. > > As a side note, it isn't just about providing the necessary support to run the BEAM VM. I also want enough of a familiar environment (ls, ps, top, nslookup/dig, netstat) to troubleshoot issues. Alpine provides that without the other stuff that you don't need. > > ~phil When I looked at Alpine, the Erlang in their repo was too old for me. Do you guys build the releases yourselves, or live with older versions? -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From codewiget95@REDACTED Mon Sep 18 17:34:39 2017 From: codewiget95@REDACTED (code wiget) Date: Mon, 18 Sep 2017 11:34:39 -0400 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: Message-ID: HI, Thank you all for your replies. Nathaniel: The reads must be 'eventually' consistent, at least within a second. The problem is that it updates user connection information, and they will be unable to connect if our read does not get information from the write. So if we update, the connection before the write is fully committed will fail. I suppose it is ok if they cannot connect and just have to reconnect, but ideally they should be able to connect every time. So Riak seems like a great solution, but speed wise really worries me. We are trying to connect as many clients as possible per server, this is very important as it saves us money. If the reads take 2-3x as long, this could be very slow and bad. According to this article: https://github.com/citrusbyte/redis-comparison , Riak is up to 10x slower than Redis. This would really hurt our operations. To those who commented redis-cluster, my problem with a cluster solution is that redis-cluster seemed to be in an experimental stage. It also has the problem where if all copies of a node die, then the cluster will lose all that data and it is up to the user to not lose that data. All of this has to be handled by the user, and this seems like it will get tedious when there are multiple nodes and all it would take is for one admin to mess it up. So this is where Aerospike comes in. Reading about them on the web they come off as the perfect tool for a version of redis that is distributed: https://stackoverflow.com/questions/24482337/how-is-aerospike-different-from-other-key-value-nosql-databases . But for some reason, they don?t get as much attention as redis Does anyone have experience with Aerospike? For my application, it seems like a no brainer. Thank you all again, > On Sep 15, 2017, at 2:02 PM, Nathaniel Waisbrot wrote: > > Scatter-shot reply: > > Since you're using Redis right now, have you considered Redis Cluster (https://redis.io/topics/cluster-tutorial )? > > I'm using Cassandra and don't feel that it's got a small community or slow pace of updates. There are a lot of NoSQL databases and they all have quite different tradeoffs which tends to fragment the community, so your expectations may be too high. > > Riak, ElasticSearch, EtcD, MongoDB, etc. You have many (too many!) options. When you say "read speed and consistency" what sort of consistency are you looking for? Is eventual consistency good, or do you require that every read that takes place after a write gets the new data? > > > > >> On Sep 15, 2017, at 12:43 PM, code wiget > wrote: >> >> Hello everyone, >> >> I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. >> >> The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. >> >> Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. >> >> A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. >> >> I have looked at as many options as I can -> >> >> Voldemort : http://project-voldemort.com/ >> - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. >> >> Cassandra: http://cassandra.apache.org/ >> - looks good too, but apparently there is a small community and apparently isn?t updated often >> >> Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf >> - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. >> >> >> So from my research, which consisted heavily of this blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. >> >> BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? >> >> Thanks for your time. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Mon Sep 18 18:08:10 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 18 Sep 2017 16:08:10 +0000 Subject: [erlang-questions] Logging from nif (and especially from rust nif) In-Reply-To: References: Message-ID: The suggestion from https://github.com/rust-lang/rust/issues/24821 is to use writeln! instead of println! so you can handle the WOULDBLOCK situation yourself instead of panicing. But, shooting all the messages over to the Erlang side seems to solve your problem. On Sun, Sep 17, 2017 at 2:28 PM Max Lapshin wrote: > > Hi. > > I'm playing now with rust nif and can advice nice library: > https://github.com/hansihe/rustler > > For those of us, who do not understand what is there about elixir: don't > worry, no need to use it, there should be a simple makefile for compiling > rust. > > > I've met stange situation while using println! mechanism in rust nif > (I'm speaking about enterprise real world debugging: as lot of prints as > possible). Nif crashes from println. > > if you are lucky to catch debug message, you will see something like: > > stack backtrace: > > 0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace > > 1: std::sys_common::backtrace::_print > > 2: std::panicking::default_hook::{{closure}} > > 3: std::panicking::default_hook > > 4: std::panicking::rust_panic_with_hook > > 5: std::panicking::begin_panic > > > or even info about error code 35. > > > Problem is that stdout is in non-blocking mode and when rust tries to > write output, it gets error that it is not waiting for. > > I've decided not to change anything in rust stdlib, but use another > mechanism: > > 1) take pid of output writing process > > 2) send messages to it from rust. > > > Here is the way to get whom to send IO request messages: > > logger() -> > > UserDrv = whereis(user_drv), > > CurGroup = case (is_pid(UserDrv) andalso process_info(UserDrv, > dictionary)) of > > {dictionary, UserDrvDict} -> > > proplists:get_value(current_group, UserDrvDict); > > _ -> > > undefined > > end, > > case is_pid(CurGroup) of > > true -> > > CurGroup; > > false -> > > {_,GL} = process_info(self(),group_leader), > > GL > > end. > > > and here is how do I log there from rust: > > > if let Some(ref logger) = mpegts.logger { > env.pid(); > env.send(logger, ( > atoms::io_request().encode(env), > env.pid().encode(env), > atoms::ok().encode(env), > ( > atoms::put_chars().encode(env), > atoms::unicode().encode(env), > (string+"\n").encode(env) > ).encode(env) > ).encode(env)) > } > > > One simple hack here: I do not send Mref as 3'rd argument, but I send atom > ok. Seems to be working. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Mon Sep 18 18:54:30 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Mon, 18 Sep 2017 18:54:30 +0200 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: Message-ID: <08DBAC06-6022-40F8-82CC-652DD30BEABC@licenser.net> I would not give too much on those ?benchmarks?, they?re highly bogus and that?s if you?re treating them kindly. For a starter it uses default settings and they are not even provided. Redis is a in memory store by default, is it even saving the data? How are risk or Cassandra set up, unlike mongo or redis the others those are build to be clustered, are the default configs used for them disabling unless overhead? Does it mean risk, that is storing every write on disks, perhaps 3 times, is only 10x slower compared to a database that never writes to disk and only keeps one copy? For you own sanity, print that benchmark, find a burn proof area (safety matters!) and set it on fire then move on and benchmark for yourself with a real use case and sensible data. > On 18. Sep 2017, at 17:34, code wiget wrote: > > HI, > > Thank you all for your replies. > > Nathaniel: The reads must be 'eventually' consistent, at least within a second. The problem is that it updates user connection information, and they will be unable to connect if our read does not get information from the write. So if we update, the connection before the write is fully committed will fail. I suppose it is ok if they cannot connect and just have to reconnect, but ideally they should be able to connect every time. > > So Riak seems like a great solution, but speed wise really worries me. We are trying to connect as many clients as possible per server, this is very important as it saves us money. If the reads take 2-3x as long, this could be very slow and bad. According to this article: https://github.com/citrusbyte/redis-comparison , Riak is up to 10x slower than Redis. This would really hurt our operations. > > To those who commented redis-cluster, my problem with a cluster solution is that redis-cluster seemed to be in an experimental stage. It also has the problem where if all copies of a node die, then the cluster will lose all that data and it is up to the user to not lose that data. All of this has to be handled by the user, and this seems like it will get tedious when there are multiple nodes and all it would take is for one admin to mess it up. > > So this is where Aerospike comes in. Reading about them on the web they come off as the perfect tool for a version of redis that is distributed: https://stackoverflow.com/questions/24482337/how-is-aerospike-different-from-other-key-value-nosql-databases . But for some reason, they don?t get as much attention as redis > > Does anyone have experience with Aerospike? For my application, it seems like a no brainer. > > Thank you all again, >> On Sep 15, 2017, at 2:02 PM, Nathaniel Waisbrot > wrote: >> >> Scatter-shot reply: >> >> Since you're using Redis right now, have you considered Redis Cluster (https://redis.io/topics/cluster-tutorial )? >> >> I'm using Cassandra and don't feel that it's got a small community or slow pace of updates. There are a lot of NoSQL databases and they all have quite different tradeoffs which tends to fragment the community, so your expectations may be too high. >> >> Riak, ElasticSearch, EtcD, MongoDB, etc. You have many (too many!) options. When you say "read speed and consistency" what sort of consistency are you looking for? Is eventual consistency good, or do you require that every read that takes place after a write gets the new data? >> >> >> >> >>> On Sep 15, 2017, at 12:43 PM, code wiget > wrote: >>> >>> Hello everyone, >>> >>> I am at the point where I have many Erlang nodes, and I am going to have to move to a distributed database. Right now, I am using a basic setup: each Erlang node has a copy of the same Redis DB, and all of those DBs are slaves(non-writable copies) of a master. A big problem with this is obvious - If the db goes down, the node goes down. If the master goes down, the slaves won?t get updated, so I would like to move to a distributed db that all of my nodes can read/write to that can not/does not go down. >>> >>> The nodes do ~50 reads per write, and are constantly reading, so read speed and consistency is my real concern. I believe this will be the node?s main speed factor. >>> >>> Another thing is that all of my data is key/key/value , so it would mimic the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL DB. >>> >>> A big thing also is that I don?t need disc copies, as a I have a large backup store where the values are generated from. >>> >>> I have looked at as many options as I can -> >>> >>> Voldemort : http://project-voldemort.com/ >>> - looks perfect, but there are 0 resources on learning how to use it outside of their docs and no Erlang driver, which is huge because I would both have to learn how to write a c driver and everything about this just to get it to work. >>> >>> Cassandra: http://cassandra.apache.org/ >>> - looks good too, but apparently there is a small community and apparently isn?t updated often >>> >>> Scalaris: https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf >>> - Looks very very cool, seems great, but there is 0 active community and their GitHub isn?t updated often. This is a distributed all in-memory database, written in Erlang. >>> >>> >>> So from my research, which consisted heavily of this blog:https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , I have narrowed it down to these three. >>> >>> BUT you are all the real experts and have built huge applications in Erlang, what do you use? What do you have experience in that performs well with Erlang nodes spread across multiple machines and possibly multiple data centers? >>> >>> Thanks for your time. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP URL: From joaohf@REDACTED Tue Sep 19 00:52:50 2017 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Mon, 18 Sep 2017 19:52:50 -0300 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: Hi, If you know a bit about Yocto Project you could try this https://github.com/joaohf/meta-erlang I am planning to do a tutorial about it soon. Thanks On Mon, Sep 18, 2017 at 1:28 AM, Lloyd R. Prentice wrote: > Hi Richard, > > So, on a server, it sounds like we're talking about: > > 11 MB for the OS > 5 MB for stripped down Erlang based on Joe's work, > plus our application code. > > Thanks to all for these numbers. > > My question came out of a conversation with a friend re: how far we could > push Erlang on a Raspberry Pi or Odroid C2 SBC. > > It's interesting to look at the numbers for TinyCore on ARM: > > http://mirror.epn.edu.ec/tinycorelinux/arm.html > > Compressed, they seem to run in the 25MB range. Raspian zipped is some > 4GB. So, on an RP server TinyCore seems to offer much. > > This makes me think that a cluster of $5.00 RP Zeros would be viable and a > great platform for exploring and teaching distributed Erlang. > > All the best, > > Lloyd > > > Sent from my iPad > > On Sep 17, 2017, at 9:55 PM, Richard A. O'Keefe wrote: > > > On 16/09/17 9:29 PM, Joe Armstrong wrote: > > Old timer here :-) > > > Just out of interest how large is Apline? (in MB?) > > > For what it's worth, TinyCore-8.0.iso is just under 17 MB. > Installed in VirtualBox and running, complete with GUI, > it's 21.7MB. That doesn't include development tools. > I note that just the > "OTP 20.0 HTML Documentation File (33,948,386)" > is bigger than this and > "OTP 20.0 Windows 32-bit Binary File (96,337,114)" > is a lot bigger. > > http://distro.ibiblio.org/tinycorelinux/intro.html > http://distro.ibiblio.org/tinycorelinux/downloads.html > The current Core (TinyCore minus GUI &c) is 11 MB. > > Bringing in clang and its dependencies blows the system > out from 21.7 to 145.8 MB (+ 124.1 MB). > Adding gettext brought it to 148.0 MB (+ 2.2 MB). > Adding the manual pages and related tools brought > it up to 152.4 MB (+ 4.4 MB). Add a couple of text > editors (NOT including emacs, which I actually like) > and it's 161.4 MB (+ 9.0 MB). > > So the actual TinyCore distribution is about 1/8th of > a fairly minimal development environment. I used to > run a full SunOS 3 environment on a 100 MB disc, and > 40 MB of that was left over for my files. How times change. > > I suspect that TinyCore plus enough to *run* Erlang could > be quite small by today's standards. Certainly you could > carry around TinyCore Linux + C development environment + > full Erlang distribution and run it comfortably from an > elderly memory stick. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Jo?o Henrique Ferreira de Freitas - joaohf_at_gmail.com Campinas-SP-Brasil -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Tue Sep 19 01:00:08 2017 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 18 Sep 2017 23:00:08 +0000 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: <08DBAC06-6022-40F8-82CC-652DD30BEABC@licenser.net> References: <08DBAC06-6022-40F8-82CC-652DD30BEABC@licenser.net> Message-ID: I'd recommend checking jepsen.io for testing of distributed systems. There's a very thorough review of Aerospike there with some results that may give you pause. https://aphyr.com/posts/324-jepsen-aerospike On Tue, Sep 19, 2017 at 4:54 AM Heinz N. Gies wrote: > I would not give too much on those ?benchmarks?, they?re highly bogus and > that?s if you?re treating them kindly. > > For a starter it uses default settings and they are not even provided. > Redis is a in memory store by default, is it even saving the data? How are > risk or Cassandra set up, unlike mongo or redis the others those are build > to be clustered, are the default configs used for them disabling unless > overhead? Does it mean risk, that is storing every write on disks, perhaps > 3 times, is only 10x slower compared to a database that never writes to > disk and only keeps one copy? > > For you own sanity, print that benchmark, find a burn proof area (safety > matters!) and set it on fire then move on and benchmark for yourself with a > real use case and sensible data. > > > On 18. Sep 2017, at 17:34, code wiget wrote: > > HI, > > Thank you all for your replies. > > Nathaniel: The reads must be 'eventually' consistent, at least within a > second. The problem is that it updates user connection information, and > they will be unable to connect if our read does not get information from > the write. So if we update, the connection before the write is fully > committed will fail. I suppose it is ok if they cannot connect and just > have to reconnect, but ideally they should be able to connect every time. > > So Riak seems like a great solution, but speed wise really worries me. We > are trying to connect as many clients as possible per server, this is very > important as it saves us money. If the reads take 2-3x as long, this could > be very slow and bad. According to this article: > https://github.com/citrusbyte/redis-comparison, Riak is up to 10x slower > than Redis. This would really hurt our operations. > > To those who commented redis-cluster, my problem with a cluster solution > is that redis-cluster seemed to be in an experimental stage. It also has > the problem where if all copies of a node die, then the cluster will lose > all that data and it is up to the user to not lose that data. All of this > has to be handled by the user, and this seems like it will get tedious when > there are multiple nodes and all it would take is for one admin to mess it > up. > > So this is where Aerospike comes in. Reading about them on the web they > come off as the perfect tool for a version of redis that is distributed: > https://stackoverflow.com/questions/24482337/how-is-aerospike-different-from-other-key-value-nosql-databases . > But for some reason, they don?t get as much attention as redis > > Does anyone have experience with Aerospike? For my application, it seems > like a no brainer. > > Thank you all again, > > On Sep 15, 2017, at 2:02 PM, Nathaniel Waisbrot > wrote: > > Scatter-shot reply: > > Since you're using Redis right now, have you considered Redis Cluster ( > https://redis.io/topics/cluster-tutorial)? > > I'm using Cassandra and don't feel that it's got a small community or slow > pace of updates. There are a lot of NoSQL databases and they all have quite > different tradeoffs which tends to fragment the community, so your > expectations may be too high. > > Riak, ElasticSearch, EtcD, MongoDB, etc. You have many (too many!) > options. When you say "read speed and consistency" what sort of consistency > are you looking for? Is eventual consistency good, or do you require that > every read that takes place after a write gets the new data? > > > > > On Sep 15, 2017, at 12:43 PM, code wiget wrote: > > Hello everyone, > > I am at the point where I have many Erlang nodes, and I am going to have > to move to a distributed database. Right now, I am using a basic setup: > each Erlang node has a copy of the same Redis DB, and all of those DBs are > slaves(non-writable copies) of a master. A big problem with this is obvious > - If the db goes down, the node goes down. If the master goes down, the > slaves won?t get updated, so I would like to move to a distributed db that > all of my nodes can read/write to that can not/does not go down. > > The nodes do ~50 reads per write, and are constantly reading, so read > speed and consistency is my real concern. I believe this will be the node?s > main speed factor. > > Another thing is that all of my data is key/key/value , so it would mimic > the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL > DB. > > A big thing also is that I don?t need disc copies, as a I have a large > backup store where the values are generated from. > > I have looked at as many options as I can -> > > Voldemort : http://project-voldemort.com/ > - looks perfect, but there are 0 resources on learning how to use it > outside of their docs and no Erlang driver, which is huge because I would > both have to learn how to write a c driver and everything about this just > to get it to work. > > Cassandra: http://cassandra.apache.org/ > - looks good too, but apparently there is a small community and apparently > isn?t updated often > > Scalaris: > https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf > - Looks very very cool, seems great, but there is 0 active community and > their GitHub isn?t updated often. This is a distributed all in-memory > database, written in Erlang. > > > So from my research, which consisted heavily of this blog: > https://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores , > I have narrowed it down to these three. > > BUT you are all the real experts and have built huge applications in > Erlang, what do you use? What do you have experience in that performs well > with Erlang nodes spread across multiple machines and possibly multiple > data centers? > > Thanks for your time. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Sep 19 01:12:21 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 18 Sep 2017 19:12:21 -0400 (EDT) Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: <1505776341.870916502@apps.rackspace.com> Hi Jo?o, I eagerly look forward to your tutorial. My focus has been on web applications. But a dear friend, who has made his living for many years developing embedded systems using 1980's technology, has been asking me about Erlang as a way of opening his systems up to the wider world. I may be whistling in the dark, but I think there's great potential for the RPs and Odroids in the right applications. The new Odroid HC1 looks particularly interesting. And I do wonder what could be squeezed out of the $5.00 RP Zero. https://ameridroid.com/ Best wishes, Lloyd -----Original Message----- From: "Jo?o Henrique Freitas" Sent: Monday, September 18, 2017 6:52pm To: "Lloyd R. Prentice" Cc: "Richard A. O'Keefe" , "Erlang" Subject: Re: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers Hi, If you know a bit about Yocto Project you could try this https://github.com/joaohf/meta-erlang I am planning to do a tutorial about it soon. Thanks On Mon, Sep 18, 2017 at 1:28 AM, Lloyd R. Prentice wrote: > Hi Richard, > > So, on a server, it sounds like we're talking about: > > 11 MB for the OS > 5 MB for stripped down Erlang based on Joe's work, > plus our application code. > > Thanks to all for these numbers. > > My question came out of a conversation with a friend re: how far we could > push Erlang on a Raspberry Pi or Odroid C2 SBC. > > It's interesting to look at the numbers for TinyCore on ARM: > > http://mirror.epn.edu.ec/tinycorelinux/arm.html > > Compressed, they seem to run in the 25MB range. Raspian zipped is some > 4GB. So, on an RP server TinyCore seems to offer much. > > This makes me think that a cluster of $5.00 RP Zeros would be viable and a > great platform for exploring and teaching distributed Erlang. > > All the best, > > Lloyd > > > Sent from my iPad > > On Sep 17, 2017, at 9:55 PM, Richard A. O'Keefe wrote: > > > On 16/09/17 9:29 PM, Joe Armstrong wrote: > > Old timer here :-) > > > Just out of interest how large is Apline? (in MB?) > > > For what it's worth, TinyCore-8.0.iso is just under 17 MB. > Installed in VirtualBox and running, complete with GUI, > it's 21.7MB. That doesn't include development tools. > I note that just the > "OTP 20.0 HTML Documentation File (33,948,386)" > is bigger than this and > "OTP 20.0 Windows 32-bit Binary File (96,337,114)" > is a lot bigger. > > http://distro.ibiblio.org/tinycorelinux/intro.html > http://distro.ibiblio.org/tinycorelinux/downloads.html > The current Core (TinyCore minus GUI &c) is 11 MB. > > Bringing in clang and its dependencies blows the system > out from 21.7 to 145.8 MB (+ 124.1 MB). > Adding gettext brought it to 148.0 MB (+ 2.2 MB). > Adding the manual pages and related tools brought > it up to 152.4 MB (+ 4.4 MB). Add a couple of text > editors (NOT including emacs, which I actually like) > and it's 161.4 MB (+ 9.0 MB). > > So the actual TinyCore distribution is about 1/8th of > a fairly minimal development environment. I used to > run a full SunOS 3 environment on a 100 MB disc, and > 40 MB of that was left over for my files. How times change. > > I suspect that TinyCore plus enough to *run* Erlang could > be quite small by today's standards. Certainly you could > carry around TinyCore Linux + C development environment + > full Erlang distribution and run it comfortably from an > elderly memory stick. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Jo?o Henrique Ferreira de Freitas - joaohf_at_gmail.com Campinas-SP-Brasil From ulf@REDACTED Tue Sep 19 10:13:39 2017 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 19 Sep 2017 10:13:39 +0200 Subject: [erlang-questions] Choice in Distributed Databases for a Key/Value Store In-Reply-To: References: <08DBAC06-6022-40F8-82CC-652DD30BEABC@licenser.net> Message-ID: O.T. Hehe, that Aerospike review mentions that they claim 100% uptime (with seemingly undisclosed precision). ?This makes Aerospike?s uptime infinitely better than the Ericsson AXD301 switch, which delivered somewhere between five and nine nines of availability in a system comprised of 1.1 million lines of Erlang.? (Never mind that the claim that 100% would be "infinitely better" than e.g. 99.999% is ludicrous in itself. I assume it was made tongue-in-cheek.) Of course, this is ridiculous, as also the in-depth review demonstrates. Uptime figures are only useful in context, and comparing uptime claims of systems with different purposes is not particularly meaningful. For example, for the AXD 301, disturbance would be registered as downtime if one network interface (of potentially hundreds) was unable to process calls for 15 seconds (which is, BTW, what happened in the "9 nines" case: a restart of a single device board.) Is a database cluster still "up" if it stays responsive but has lost your data? When claiming availability, you have to be very specific. The article also mentions cluster sizes of "between 1 and 100 nodes". Guaranteeing 100% uptime in a 1-node 'cluster' is simply not possible. Anyway, that article is two years old. Today, Aerospike claims "demonstrated uptime of five 9s". Still very good, but I guess "infinitely worse" than it used to be. ;-) ( http://www.aerospike.com/benefits/high-availability/) BR, Ulf W PS This is not to defend the "9 nines" claim, which was never officially made by Ericsson. It was made in a press release by British Telecom. Ericsson doesn't divulge the actual uptime figures of its systems, but at least at one time it was ok to claim publicly that the average recorded field uptime of AXD 301 systems was "better than 5 nines". 2017-09-19 1:00 GMT+02:00 Paul Oliver : > I'd recommend checking jepsen.io for testing of distributed systems. > There's a very thorough review of Aerospike there with some results that > may give you pause. https://aphyr.com/posts/324-jepsen-aerospike > > On Tue, Sep 19, 2017 at 4:54 AM Heinz N. Gies wrote: > >> I would not give too much on those ?benchmarks?, they?re highly bogus and >> that?s if you?re treating them kindly. >> >> For a starter it uses default settings and they are not even provided. >> Redis is a in memory store by default, is it even saving the data? How are >> risk or Cassandra set up, unlike mongo or redis the others those are build >> to be clustered, are the default configs used for them disabling unless >> overhead? Does it mean risk, that is storing every write on disks, perhaps >> 3 times, is only 10x slower compared to a database that never writes to >> disk and only keeps one copy? >> >> For you own sanity, print that benchmark, find a burn proof area (safety >> matters!) and set it on fire then move on and benchmark for yourself with a >> real use case and sensible data. >> >> >> On 18. Sep 2017, at 17:34, code wiget wrote: >> >> HI, >> >> Thank you all for your replies. >> >> Nathaniel: The reads must be 'eventually' consistent, at least within a >> second. The problem is that it updates user connection information, and >> they will be unable to connect if our read does not get information from >> the write. So if we update, the connection before the write is fully >> committed will fail. I suppose it is ok if they cannot connect and just >> have to reconnect, but ideally they should be able to connect every time. >> >> So Riak seems like a great solution, but speed wise really worries me. We >> are trying to connect as many clients as possible per server, this is very >> important as it saves us money. If the reads take 2-3x as long, this could >> be very slow and bad. According to this article: https://github.com/ >> citrusbyte/redis-comparison, Riak is up to 10x slower than Redis. This >> would really hurt our operations. >> >> To those who commented redis-cluster, my problem with a cluster solution >> is that redis-cluster seemed to be in an experimental stage. It also has >> the problem where if all copies of a node die, then the cluster will lose >> all that data and it is up to the user to not lose that data. All of this >> has to be handled by the user, and this seems like it will get tedious when >> there are multiple nodes and all it would take is for one admin to mess it >> up. >> >> So this is where Aerospike comes in. Reading about them on the web they >> come off as the perfect tool for a version of redis that is distributed: >> https://stackoverflow.com/questions/24482337/how-is-aerospike- >> different-from-other-key-value-nosql-databases . But for some reason, >> they don?t get as much attention as redis >> >> Does anyone have experience with Aerospike? For my application, it seems >> like a no brainer. >> >> Thank you all again, >> >> On Sep 15, 2017, at 2:02 PM, Nathaniel Waisbrot >> wrote: >> >> Scatter-shot reply: >> >> Since you're using Redis right now, have you considered Redis Cluster ( >> https://redis.io/topics/cluster-tutorial)? >> >> I'm using Cassandra and don't feel that it's got a small community or >> slow pace of updates. There are a lot of NoSQL databases and they all have >> quite different tradeoffs which tends to fragment the community, so your >> expectations may be too high. >> >> Riak, ElasticSearch, EtcD, MongoDB, etc. You have many (too many!) >> options. When you say "read speed and consistency" what sort of consistency >> are you looking for? Is eventual consistency good, or do you require that >> every read that takes place after a write gets the new data? >> >> >> >> >> On Sep 15, 2017, at 12:43 PM, code wiget wrote: >> >> Hello everyone, >> >> I am at the point where I have many Erlang nodes, and I am going to have >> to move to a distributed database. Right now, I am using a basic setup: >> each Erlang node has a copy of the same Redis DB, and all of those DBs are >> slaves(non-writable copies) of a master. A big problem with this is obvious >> - If the db goes down, the node goes down. If the master goes down, the >> slaves won?t get updated, so I would like to move to a distributed db that >> all of my nodes can read/write to that can not/does not go down. >> >> The nodes do ~50 reads per write, and are constantly reading, so read >> speed and consistency is my real concern. I believe this will be the node?s >> main speed factor. >> >> Another thing is that all of my data is key/key/value , so it would mimic >> the structure of ID -> name -> ?Fred?, ID->age->20, so I don?t need a SQL >> DB. >> >> A big thing also is that I don?t need disc copies, as a I have a large >> backup store where the values are generated from. >> >> I have looked at as many options as I can -> >> >> Voldemort : http://project-voldemort.com/ >> - looks perfect, but there are 0 resources on learning how to use it >> outside of their docs and no Erlang driver, which is huge because I would >> both have to learn how to write a c driver and everything about this just >> to get it to work. >> >> Cassandra: http://cassandra.apache.org/ >> - looks good too, but apparently there is a small community and >> apparently isn?t updated often >> >> Scalaris: https://github.com/scalaris-team/scalaris/blob/ >> master/user-dev-guide/main.pdf >> - Looks very very cool, seems great, but there is 0 active community and >> their GitHub isn?t updated often. This is a distributed all in-memory >> database, written in Erlang. >> >> >> So from my research, which consisted heavily of this blog: >> https://www.metabrew.com/article/anti-rdbms-a-list-of- >> distributed-key-value-stores , I have narrowed it down to these three. >> >> BUT you are all the real experts and have built huge applications in >> Erlang, what do you use? What do you have experience in that performs well >> with Erlang nodes spread across multiple machines and possibly multiple >> data centers? >> >> Thanks for your time. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Tue Sep 19 11:08:28 2017 From: hello@REDACTED (Adam Lindberg) Date: Tue, 19 Sep 2017 11:08:28 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: Perhaps it's not relevant for you right now, but for an interesting use-case of what is possible: https://www.grisp.org We're successfully running Erlang bare metal on an ARM Cortex M7 without any kernel (the VM is the kernel!). All our software is open source and available at https://github.com/grisp Let me know if you have any questions. Cheers, Adam > On 18 Sep 2017, at 06:28, Lloyd R. Prentice wrote: > > Hi Richard, > > So, on a server, it sounds like we're talking about: > > 11 MB for the OS > 5 MB for stripped down Erlang based on Joe's work, > plus our application code. > > Thanks to all for these numbers. > > My question came out of a conversation with a friend re: how far we could push Erlang on a Raspberry Pi or Odroid C2 SBC. > > It's interesting to look at the numbers for TinyCore on ARM: > > http://mirror.epn.edu.ec/tinycorelinux/arm.html > > Compressed, they seem to run in the 25MB range. Raspian zipped is some 4GB. So, on an RP server TinyCore seems to offer much. > > This makes me think that a cluster of $5.00 RP Zeros would be viable and a great platform for exploring and teaching distributed Erlang. > > All the best, > > Lloyd > > > Sent from my iPad > > On Sep 17, 2017, at 9:55 PM, Richard A. O'Keefe wrote: > >> >> On 16/09/17 9:29 PM, Joe Armstrong wrote: >>> Old timer here :-) >>> >>> Just out of interest how large is Apline? (in MB?) >> >> For what it's worth, TinyCore-8.0.iso is just under 17 MB. >> Installed in VirtualBox and running, complete with GUI, >> it's 21.7MB. That doesn't include development tools. >> I note that just the >> "OTP 20.0 HTML Documentation File (33,948,386)" >> is bigger than this and >> "OTP 20.0 Windows 32-bit Binary File (96,337,114)" >> is a lot bigger. >> >> http://distro.ibiblio.org/tinycorelinux/intro.html >> http://distro.ibiblio.org/tinycorelinux/downloads.html >> The current Core (TinyCore minus GUI &c) is 11 MB. >> >> Bringing in clang and its dependencies blows the system >> out from 21.7 to 145.8 MB (+ 124.1 MB). >> Adding gettext brought it to 148.0 MB (+ 2.2 MB). >> Adding the manual pages and related tools brought >> it up to 152.4 MB (+ 4.4 MB). Add a couple of text >> editors (NOT including emacs, which I actually like) >> and it's 161.4 MB (+ 9.0 MB). >> >> So the actual TinyCore distribution is about 1/8th of >> a fairly minimal development environment. I used to >> run a full SunOS 3 environment on a 100 MB disc, and >> 40 MB of that was left over for my files. How times change. >> >> I suspect that TinyCore plus enough to *run* Erlang could >> be quite small by today's standards. Certainly you could >> carry around TinyCore Linux + C development environment + >> full Erlang distribution and run it comfortably from an >> elderly memory stick. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hello@REDACTED Tue Sep 19 11:13:14 2017 From: hello@REDACTED (Adam Lindberg) Date: Tue, 19 Sep 2017 11:13:14 +0200 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: References: Message-ID: Cost wise I'm not sure, but I would advice you to benchmark for your use case. Operations wise the most common problem you'll see with module reloads is crashed processes. Erlang only keeps two versions of a module in memory at the same time. If you have processes lingering in the old code when reloading yet another version, those processes will get killed. Cheers, Adam > On 16 Sep 2017, at 23:57, Edmond Begumisa wrote: > > On Sun, 17 Sep 2017 03:39:50 +1000, Edmond Begumisa wrote: > >> Hello all, >> >> (Not new here--just haven't posted in a while!) >> >> I'm using merl to dynamically generate and regenerate modules at runtime that make use of the ERTS constant pool in order to efficiently avail data to all processes. I'm aware that this technique is known amongst more experienced Erlangers for situations where the data is updated infrequently (e.g. sharing global configuration parameters). >> >> The question I'm investigating is how frequent is "infrequent" w.r.t reloading of modules[1]? I'm not entirely sure about the costs involved when (re)loading a module so I've got no idea where to start in order to measure the impact on the emulator. Could anybody with sufficient knowledge of emulator internals point me in the right direction? >> > > To clarify: When measuring whether or not I'm hot-code swapping too frequently, what are the negative signs to look for? What does the emulator actually do when swapping out code w.r.t potentially disruptive/costly operations like locking, GC, etc which could negatively impact scheduling, memory and responsiveness of the system? > > I'm trying to figure out when the best time to reload a dynamically generated module/group of modules is and how frequently I should allow reloading to occur. > > - Edmond - > >> Thanks in advance. >> >> - Edmond - >> >> [1] The more exotic use-cases for use/abuse of the constant pool and module reloading I'm looking into are to do with caching and incremental computation. >> > > > -- > Using Opera's mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lukas@REDACTED Tue Sep 19 11:40:14 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 19 Sep 2017 11:40:14 +0200 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: References: Message-ID: On Tue, Sep 19, 2017 at 11:13 AM, Adam Lindberg wrote: > Cost wise I'm not sure, but I would advice you to benchmark for your use > case. > Every time you purge a module, each process heap is scanned for any sign of the old literals (aka constants) and if a literal is found a GC is done of that process which copies the literal onto its heap. So the cost depends on the total heap size of all running processes and the number of processes that have a reference to those literals on their heaps at the moment of the purge. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Tue Sep 19 13:24:58 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Tue, 19 Sep 2017 16:54:58 +0530 (IST) Subject: [erlang-questions] Test suite for checking concurrency in erlang Message-ID: <731976319.49967.1505820298212@webmail.blacklightsw.com> Hello, I wanted to write a test case which can show me that when one single code base is used and accessed by two or more individuals at the same time then how does it behaves(I basically want to test for concurrancy). I have chosen the code base as a tic-tac-toe game present on this link https://github.com/bekkopen/erlang-tictactoe/tree/master/src . I have changed the code a bit though and have attached the files in this mail. Now I tried common_test and wrote the following module testing2 -module(testing2). -include_lib("common_test/include/ct.hrl"). %% ==================================================================== %% API functions %% ==================================================================== -export([all/0, groups/0, init_per_group/2, end_per_group/2]). -export([test1/1,test2/1]). %% ==================================================================== %% Internal functions %% ==================================================================== all() -> [{group, session}]. groups() -> [{session, [], [{group, clients}]}, {clients, [parallel, {repeat, 1}], [test1, test2]}]. init_per_group(session, Config) -> gameserver:start(), Config; init_per_group(_, Config) -> Config. end_per_group(session, _Config) -> gameclient:stop(); end_per_group(_, _Config) -> ok. test1(_Config)-> Abhishek=gameclient:login("Abhishek"), Dharun=gameclient:login("Dharun"), gameclient:new_game(Abhishek), timer:sleep(5), gameclient:make_move(Abhishek, a1), timer:sleep(5), gameclient:make_move(Dharun, b2), timer:sleep(5), gameclient:make_move(Abhishek, a2), timer:sleep(5), gameclient:make_move(Dharun, c3), timer:sleep(5), gameclient:make_move(Abhishek, a3). test2(_Config)-> Abhijeet=gameclient:login("Abhijeet"), Ranjan=gameclient:login("Ranjan"), gameclient:new_game(Abhijeet), timer:sleep(5), gameclient:make_move(Abhijeet, a1), timer:sleep(5), gameclient:make_move(Ranjan, b2), timer:sleep(5), gameclient:make_move(Abhijeet, a2), timer:sleep(5), gameclient:make_move(Ranjan, c3), timer:sleep(5), gameclient:make_move(Abhijeet, a3). Now when I see the logs of ct I find the following output: for test1()-> === Started at 2017-09-19 16:37:16 ?Welcome to tictactoe server!? ?Welcome to tictactoe server!? ?Ready to rumble against Abhishek!? ?Ready to rumble against Dharun!? === Ended at 2017-09-19 16:37:16 === successfully completed test case === Returned value: {make_move,a3} for test2()-> ? === Started at 2017-09-19 16:37:16 ?Welcome to tictactoe server!? ?Welcome to tictactoe server!? ?Ready to rumble against Ranjan!? ?Ready to rumble against Abhijeet!? ?It is not your turn yet!? ?It is not your turn yet!? === Ended at 2017-09-19 16:37:16 === successfully completed test case === Returned value: {make_move,a3} But I expected the output of both the games in the logs as I am running them in parallel. Can anyone suggest me what am I missing and how to go about it? Thank you. Best Regards, Abhishek -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -module(gameclient). -compile(export_all). t() -> gameserver:start(), Player1 = login("player1"), Player2 = login("player2"), new_game(Player1), make_move(Player1, a1), timer:sleep(5), make_move(Player1, b1), timer:sleep(5), make_move(Player2, b2), timer:sleep(5), make_move(Player1, a2), timer:sleep(5), make_move(Player2, a2), timer:sleep(5), make_move(Player2, c3), timer:sleep(5), make_move(Player1, a3). login(Name) -> Pid = spawn(gameclient, loop, [ Name ]), gameserver:connect(Name, Pid), Pid. new_game(Pid) -> Pid ! {new_game}. make_move(Pid, Move) -> Pid ! { make_move, Move }. stop()-> self() ! {stop}. exit(Pid) -> Pid ! { exit}. game_tie(Pid, OpponentName, GameState) -> Pid ! { tie, OpponentName, GameState }. send_message(Pid, Message) -> Pid ! { msg, Message}. loop(Name) -> receive { msg, Message } -> io:format("~p~n", [Message]), loop(Name); { new_game} -> gameserver:new_game(Name), loop(Name); { make_move, Move } -> gameserver:make_move(Name, Move), loop(Name); {stop}-> gameserver:stop(), loop(Name); {exit}-> gameserver:exit(Name), loop(Name); { tie, OpponentName, GameState } -> io:format("~p~n", [get_tie_message(OpponentName)]), GameState, loop(Name) end. get_win_message(Over) -> "You won over " ++ Over ++ " :)". get_loose_message(For) -> "You lost over " ++ For ++ "... :(". get_tie_message(Opponent) -> "It is a tie between you and " ++ Opponent. -------------- next part -------------- -module(gameserver). -compile(export_all). -define(SERVER, gameserver). start() -> server_util:start(?SERVER, { gameserver, game_loop, [ dict:new(), dict:new() ]}). stop()-> server_util:stop(?SERVER). connect(Name, Pid) -> global:send(?SERVER, { connect, Name, Pid }). new_game(Name) -> global:send(?SERVER, { new_game, Name}). make_move(Name, Move) -> global:send(?SERVER, { make_move, Name, Move }). exit(Name)-> global:send(?SERVER, { exit,Name}). game_loop(Players, Games) -> process_flag(trap_exit, true), receive { connect, Name, Pid } -> Pid ! { msg, "Welcome to tictactoe server!" }, game_loop(dict:store(Name, Pid, Players), Games); { new_game, Name} -> PlayerList=dict:fetch_keys(Players), PlayerAndOpponent=lists:partition(fun(A)->Name==A end, PlayerList), {_,OpponentNameList}=PlayerAndOpponent, OpponentName=list_to_tuple(OpponentNameList), case dict:find(Name, Players) of { ok, Pid } -> case dict:find(element(1,OpponentName), Players) of { ok, OpponentPid} -> Pid ! { msg, "Ready to rumble against " ++ element(1,OpponentName) ++ "!" }, OpponentPid ! { msg, "Ready to rumble against " ++ Name ++ "!" }, GamePid = tictactoe:start({Pid, Name}, {OpponentPid, element(1,OpponentName)}), GameKey = create_game_key(Name, element(1,OpponentName)), link(GamePid), game_loop(Players, dict:store(GameKey, GamePid, Games)); error -> Pid ! { msg, "Did not find opponent " ++ OpponentName ++ "!" }, game_loop(Players, Games) end; error -> io:format("Could not find player ~p~n", [ Name ]), game_loop(Players, Games) end; { make_move, Name, Move } -> PlayerList=dict:fetch_keys(Players), PlayerAndOpponent=lists:partition(fun(A)->Name==A end, PlayerList), {_,OpponentNameList}=PlayerAndOpponent, OpponentName=list_to_tuple(OpponentNameList), case dict:find(create_game_key(Name, element(1,OpponentName)), Games) of { ok, GamePid } -> tictactoe:make_move(GamePid, Name, Move), game_loop(Players, Games); error -> game_loop(Players, Games) end; {'EXIT', GamePid, _} -> io:format("Removing game ~p~n", [GamePid]), game_loop(Players, remove_game(GamePid, Games)); {exit, Name} -> PlayerList=dict:fetch_keys(Players), PlayerAndOpponent=lists:partition(fun(A)->Name==A end, PlayerList), {_,OpponentNameList}=PlayerAndOpponent, OpponentName=list_to_tuple(OpponentNameList), case dict:find(create_game_key(Name, element(1,OpponentName)), Games) of { ok, GamePid } -> io:format("Exit from game ~p~n", [GamePid]), game_loop(Players, remove_game(GamePid, Games)); error -> io:format("Iside exit but error"), game_loop(Players, Games) end; Oops -> io:format("I don't get ~p~n", [ Oops ]), game_loop(Players, Games) end. remove_game(GamePid, Games) -> dict:filter(fun(_, Value) -> Value =/= GamePid end, Games). create_game_key(PlayerOne, PlayerTwo) -> io:format("Value of Playerone in createGameKey is ~p~n",[PlayerOne]), io:format("Value of PlayerTwo in createGameKey is ~p~n",[PlayerTwo]), string:join(lists:sort([PlayerOne, PlayerTwo]), ""). -------------- next part -------------- -module(tictactoe). -compile(export_all). start(PlayerOne, PlayerTwo) -> InitialGame = [a1, a2, a3, b1, b2, b3, c1, c2, c3], spawn(tictactoe, loop, [PlayerOne, PlayerTwo, PlayerOne, InitialGame]). make_move(GamePid, Player, Move) -> GamePid ! { make_move, Player, Move }. loop(PlayerOne, PlayerTwo, CurrentPlayer, GameState) -> {CurrentPid, CurrentName} = CurrentPlayer, io:format("Inside tictactoe The value of CurrentPid is ~p~n",[CurrentPid]), io:format("Inside tictactoe The value of CurrentName is ~p~n",[CurrentName]), {PlayerOnePid, PlayerOneName} = PlayerOne, {PlayerTwoPid, PlayerTwoName} = PlayerTwo, receive { make_move, CurrentName, Move } -> ValidMove = lists:any(fun(X) -> X =:= Move end, GameState), %% io:format("Inside tictactoe The value of is ~p~n",[), io:format("Inside tictactoe The value of Move is ~p~n",[Move]), io:format("Inside tictactoe The value of Gamestate is ~p~n",[GameState]), io:format("Inside tictactoe The value of Currentname is ~p~n",[CurrentName]), io:format("Inside tictactoe The value of Validmove is ~p~n",[ValidMove]), if ValidMove -> CurrentPlayerAtom = player_atom(PlayerOne, PlayerTwo, CurrentPlayer), UpdateFun = fun(Pos) -> value_if_match(Pos, Move, CurrentPlayerAtom) end, NewGameState = lists:map(UpdateFun, GameState), NewCurrentPlayer = change_player(PlayerOne, PlayerTwo, CurrentPlayer), io:format("State: ~p -> ~p~n", [GameState, NewGameState]), case get_game_result(NewGameState) of tie -> gameclient:game_tie(PlayerOnePid, PlayerTwoName, NewGameState), gameclient:game_tie(PlayerTwoPid, PlayerOneName, NewGameState); x -> gameclient:send_message(PlayerOnePid, get_win_message(PlayerTwoName)), gameclient:send_message(PlayerTwoPid, get_loose_message(PlayerOneName)); %% gameserver:stop(); o -> gameclient:send_message(PlayerTwoPid, get_win_message(PlayerOneName)), gameclient:send_message(PlayerOnePid, get_loose_message(PlayerTwoName)); %% gameserver:stop(); continue -> loop(PlayerOne, PlayerTwo, NewCurrentPlayer, NewGameState) end; true -> gameclient:send_message(CurrentPid, "Position " ++ erlang:atom_to_list(Move) ++ " is not available."), loop(PlayerOne, PlayerTwo, CurrentPlayer, GameState) end; { make_move, WrongPlayerName, _ } -> WrongPlayerPid = get_pid_for_player_name(PlayerOne, PlayerTwo, WrongPlayerName), gameclient:send_message(WrongPlayerPid, "It is not your turn yet!"), loop(PlayerOne, PlayerTwo, CurrentPlayer, GameState); Message -> io:format("PlayerOne: ~p~n PlayerTwo: ~p~n CurrentPlayer: ~p~n Message: ~p~n", [PlayerOne, PlayerTwo, CurrentPlayer, Message]), loop(PlayerOne, PlayerTwo, CurrentPlayer, GameState) end. get_win_message(Over) -> "You won over " ++ Over ++ " :)". get_loose_message(For) -> "You lost over " ++ For ++ "... :(". get_game_result(GameState) -> case check_for_winner(GameState) of undecided -> GameOver = lists:all(fun(X) -> (X =:= x) or (X =:= o) end, GameState), if GameOver -> tie; true -> continue end; X -> X end. check_for_winner([A1, A2, A3, B1, B2, B3, C1, C2, C3]) -> Rows = [[A1, A2, A3], [B1, B2, B3], [C1, C2, C3]], Columns = [[A1, B1, C1], [A2, B2, C2], [A3, B3, C3]], Diagonals = [[A1, B2, C3], [A3, B2, C1]], get_winner(Rows ++ Columns ++ Diagonals). get_winner([]) -> undecided; get_winner([[X, X, X] | _]) -> X; get_winner([_ | T]) -> get_winner(T). value_if_match(X, X, NewValue) -> NewValue; value_if_match(X, _, _) -> X. player_atom(X, _, X) -> x; player_atom(_, Y, Y) -> o. change_player(PlayerOne, PlayerTwo, PlayerOne) -> PlayerTwo; change_player(PlayerOne, PlayerTwo, PlayerTwo) -> PlayerOne. get_pid_for_player_name({PlayerOnePid, WrongPlayerName}, _, WrongPlayerName) -> PlayerOnePid; get_pid_for_player_name(_, {PlayerTwoPid, WrongPlayerName}, WrongPlayerName) -> PlayerTwoPid. -------------- next part -------------- -module(server_util). -compile([export_all]). start(ServerName, {Module, Function, Args}) -> Pid = spawn(Module, Function, Args), global:register_name(ServerName, Pid). stop(ServerName) -> global:unregister_name(ServerName), global:send(ServerName, shutdown). From jesper.louis.andersen@REDACTED Tue Sep 19 13:37:47 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 19 Sep 2017 11:37:47 +0000 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: Message-ID: On Fri, Sep 15, 2017 at 4:20 PM Lloyd R. Prentice wrote: > What light weight open-source off-the shelf operating systems would > battle-hardened Erlang gurus recommend? > > I'm partial to either stripped down Illumos or FreeBSD systems. Build a minimal Zone/Jail and use that for deploying. You can do green/blue deployments easily by spinning zones/jails up and down for instance. At work (Shopgun), we are running stock Ubuntu LTS with AWS CodeDeploy of Erlang releases. That seems to work pretty well, if you can stand the enterpriseyness of the AWS stack. In the future, I'd like to explore a container-based standard. But I think that solution is still maturing, so I/we haven't made that jump yet. Currently, we are learning toward Kubernetes, but it might change, depending on circumstances. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Sep 19 13:41:19 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 19 Sep 2017 11:41:19 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe wrote: > What's your opinion of F#? It seems to be the language > of choice for people who liked Caml but need to take advantage > of multiple cores. I haven't done any benchmarking; I doubt > that it could match OCaml in raw speed. > > I have not used it enough to have an opinion (yet). Were I to communicate a lot with the .NET platform, I'd probably pick it because it has a null value and this is a necessity when talking to C# I'm told. Given that it runs under a pretty powerful JIT, it could perform really well for a lot of tasks I think. My "dream" would be an industry-supported parallel MLton :P -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Tue Sep 19 15:57:08 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Tue, 19 Sep 2017 13:57:08 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe > wrote: > >> What's your opinion of F#? It seems to be the language >> of choice for people who liked Caml but need to take advantage >> of multiple cores. I haven't done any benchmarking; I doubt >> that it could match OCaml in raw speed. >> >> > I have not used it enough to have an opinion (yet). Were I to communicate > a lot with the .NET platform, I'd probably pick it because it has a null > value and this is a necessity when talking to C# I'm told. > > Given that it runs under a pretty powerful JIT, it could perform really > well for a lot of tasks I think. > > F# doesn't typically have any speed advantages of any other .NET languages and in every comparison I've seen to OCaml it has performed worse. If anything the allocation costs induced by a functional first programming style means it is typically a bit slower than the equivalent C# code (also there is no goto). As a language F# is the nicest I've ever used substantially. I find it easy (and fun) to write reasonably correct code in. Also I hardly ever fear refactoring (compared to erlang where I break out in cold sweats even for code bases that pass dialyzer). I even like it so much I've started hacking on an fsharp to core erlang compiler. (https://github.com/kjnilsson/fez). > My "dream" would be an industry-supported parallel MLton :P > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Sep 19 16:32:06 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 19 Sep 2017 16:32:06 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: 2017-09-19 15:57 GMT+02:00 Karl Nilsson : > > I even like it so much I've started hacking on an fsharp to core erlang > compiler. (https://github.com/kjnilsson/fez). > You win the compiler naming contest. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Tue Sep 19 16:57:40 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 19 Sep 2017 14:57:40 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: How does Alpaca compare to F#/Ocaml? After tasting some Rust and Elm, working in Erlang makes me a bit nervous too. My dream is to see a BEAM ML-like achieve Elixir stature. On Tue, Sep 19, 2017 at 8:57 AM Karl Nilsson wrote: > On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe >> wrote: >> >>> What's your opinion of F#? It seems to be the language >>> of choice for people who liked Caml but need to take advantage >>> of multiple cores. I haven't done any benchmarking; I doubt >>> that it could match OCaml in raw speed. >>> >>> >> I have not used it enough to have an opinion (yet). Were I to communicate >> a lot with the .NET platform, I'd probably pick it because it has a null >> value and this is a necessity when talking to C# I'm told. >> >> Given that it runs under a pretty powerful JIT, it could perform really >> well for a lot of tasks I think. >> >> > F# doesn't typically have any speed advantages of any other .NET languages > and in every comparison I've seen to OCaml it has performed worse. If > anything the allocation costs induced by a functional first programming > style means it is typically a bit slower than the equivalent C# code (also > there is no goto). > > As a language F# is the nicest I've ever used substantially. I find it > easy (and fun) to write reasonably correct code in. Also I hardly ever fear > refactoring (compared to erlang where I break out in cold sweats even for > code bases that pass dialyzer). > > I even like it so much I've started hacking on an fsharp to core erlang > compiler. (https://github.com/kjnilsson/fez). > > > >> My "dream" would be an industry-supported parallel MLton :P >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Sep 19 21:50:18 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 19 Sep 2017 19:50:18 +0000 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? Message-ID: Hi guys Newbie here? Can someone point me please to the C data structure representing small and large binaries inside the VM? Do they?ve the same C representation (or not)? How the VM distinguishes between them internally? Thank you. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Tue Sep 19 22:13:22 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Tue, 19 Sep 2017 22:13:22 +0200 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: Look for binary, procbin, refbin (probably), and sub binary. Use this as keywords On 19 Sep 2017 21:50, "Frank Muller" wrote: > Hi guys > > Newbie here? > > Can someone point me please to the C data structure representing small and > large binaries inside the VM? > Do they?ve the same C representation (or not)? > How the VM distinguishes between them internally? > > Thank you. > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Sep 20 00:50:22 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 20 Sep 2017 10:50:22 +1200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> Message-ID: <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> On 19/09/17 9:08 PM, Adam Lindberg wrote: > Perhaps it's not relevant for you right now, but for an interesting use-case of what is possible: https://www.grisp.org > > We're successfully running Erlang bare metal on an ARM Cortex M7 without any kernel (the VM is the kernel!). All our software is open source and available at https://github.com/grisp Great work. The software side is really impressive. One question. I can get a "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" for NZD 69 including tax and run Erlang on it. (Hmm. Next time I have some spare cash I must get one...) Why would I pay NZD300 for a 300MHz board with 64MB of memory? https://www.grisp.org/specs.html has a link "Erlang-based custom operating system", but clicking on that gives me a 404. From frank.muller.erl@REDACTED Wed Sep 20 06:08:31 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 20 Sep 2017 04:08:31 +0000 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: Hi Dmytro, Thanks for the hint. I found a lot of references to these keywords ... bit lost. How about the rest of the questions? /Frank Tue. Sept. 19 2017 ? 22:13, Dmytro Lytovchenko wrote : > Look for binary, procbin, refbin (probably), and sub binary. Use this as > keywords > > On 19 Sep 2017 21:50, "Frank Muller" wrote: > >> Hi guys >> >> Newbie here? >> >> Can someone point me please to the C data structure representing small >> and large binaries inside the VM? >> Do they?ve the same C representation (or not)? >> How the VM distinguishes between them internally? >> >> Thank you. >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Wed Sep 20 09:40:07 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Wed, 20 Sep 2017 09:40:07 +0200 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: >From what I remember (some people who recently have worked with binaries might know much more) Binaries in binary heap (large bins) have own structure, i think the master storage structure is called Binary. This is confusing even to me so have a spoon of salt with this advice :) https://github.com/erlang/otp/blob/master/erts/emulator/beam/erl_binary.h https://github.com/erlang/otp/blob/master/erts/emulator/beam/binary.c Binaries stored outside processes (proc-bins ProcBin), smaller than 64b (proc-bins, structure ErlHeapBin i think) and sub-bins (ErlSubBin) have own tag bit combinations on the process heap. So the GC and VM always know what they have found. They might share part of C structure pointing to the external binary, you can observe it as a macro included in the respective structures, think is called ErtsMagicBinary. Macros is_not_binary, binary_size, binary_bytes and ERTS_GET_REAL_BIN might become useful. Also ref-bins all contain 'next' pointer, as a part of long linked list which begins somewhere in process header (one such list belongs to every process) and during GC this list is traversed to find out which ref-bins are about to die and refcount is decremented. 2017-09-20 6:08 GMT+02:00 Frank Muller : > Hi Dmytro, > > Thanks for the hint. I found a lot of references to these keywords ... bit > lost. > > How about the rest of the questions? > > /Frank > > Tue. Sept. 19 2017 ? 22:13, Dmytro Lytovchenko < > dmytro.lytovchenko@REDACTED> wrote : > >> Look for binary, procbin, refbin (probably), and sub binary. Use this as >> keywords >> >> On 19 Sep 2017 21:50, "Frank Muller" wrote: >> >>> Hi guys >>> >>> Newbie here? >>> >>> Can someone point me please to the C data structure representing small >>> and large binaries inside the VM? >>> Do they?ve the same C representation (or not)? >>> How the VM distinguishes between them internally? >>> >>> Thank you. >>> /Frank >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Wed Sep 20 09:55:32 2017 From: zkessin@REDACTED (Zachary Kessin) Date: Wed, 20 Sep 2017 10:55:32 +0300 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: I would love to see an Elm like language on the beam too, but I don't expect to see a first-class language of that type soon. Zach ? On Tue, Sep 19, 2017 at 5:57 PM, Daniel Goertzen wrote: > How does Alpaca compare to F#/Ocaml? After tasting some Rust and Elm, > working in Erlang makes me a bit nervous too. My dream is to see a BEAM > ML-like achieve Elixir stature. > > > On Tue, Sep 19, 2017 at 8:57 AM Karl Nilsson wrote: > >> On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe >>> wrote: >>> >>>> What's your opinion of F#? It seems to be the language >>>> of choice for people who liked Caml but need to take advantage >>>> of multiple cores. I haven't done any benchmarking; I doubt >>>> that it could match OCaml in raw speed. >>>> >>>> >>> I have not used it enough to have an opinion (yet). Were I to >>> communicate a lot with the .NET platform, I'd probably pick it because it >>> has a null value and this is a necessity when talking to C# I'm told. >>> >>> Given that it runs under a pretty powerful JIT, it could perform really >>> well for a lot of tasks I think. >>> >>> >> F# doesn't typically have any speed advantages of any other .NET >> languages and in every comparison I've seen to OCaml it has performed >> worse. If anything the allocation costs induced by a functional first >> programming style means it is typically a bit slower than the equivalent C# >> code (also there is no goto). >> >> As a language F# is the nicest I've ever used substantially. I find it >> easy (and fun) to write reasonably correct code in. Also I hardly ever fear >> refactoring (compared to erlang where I break out in cold sweats even for >> code bases that pass dialyzer). >> >> I even like it so much I've started hacking on an fsharp to core erlang >> compiler. (https://github.com/kjnilsson/fez). >> >> >> >>> My "dream" would be an industry-supported parallel MLton :P >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Zach Kessin Teaching Web Developers to test code to find more bugs in less time Skype: zachkessin +972 54 234 3956 / +44 203 734 9790 / +1 617 778 7213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Wed Sep 20 11:12:04 2017 From: hello@REDACTED (Adam Lindberg) Date: Wed, 20 Sep 2017 11:12:04 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> Message-ID: <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> Hi Richard and Lloyd, We target smaller embedded systems that could benefit from Erlang, whereas on a Raspberry Pi (even zero) or Beagleboard you can easily run Linux and Erlang on top. The goal is to be able to work as much as possible with hardware in Erlang. We combine Wi-Fi with support more standard connectors, such as ports for PMODs, I?C, 1-Wire an so on. Our 300 Mhz CPU uses much less energy and the board is aimed at being an evaluation board for (then cheaper) embedded systems. As for the price, mainly it's a matter of scale. We suspect Raspberry Pi still looses money on their production, selling units at a loss. We are prepared to loose money on the development, but we are selling the hardware at cost. Cheers, Adam > On 20 Sep 2017, at 00:50, Richard A. O'Keefe wrote: > > > > On 19/09/17 9:08 PM, Adam Lindberg wrote: >> Perhaps it's not relevant for you right now, but for an interesting use-case of what is possible: https://www.grisp.org >> >> We're successfully running Erlang bare metal on an ARM Cortex M7 without any kernel (the VM is the kernel!). All our software is open source and available at https://github.com/grisp > > Great work. The software side is really impressive. > > One question. I can get a > "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" > for NZD 69 including tax and run Erlang on it. > (Hmm. Next time I have some spare cash I must get one...) > > Why would I pay NZD300 for a 300MHz board with 64MB of memory? > > https://www.grisp.org/specs.html > has a link "Erlang-based custom operating system", > but clicking on that gives me a 404. > From drasko.draskovic@REDACTED Wed Sep 20 11:36:32 2017 From: drasko.draskovic@REDACTED (Drasko DRASKOVIC) Date: Wed, 20 Sep 2017 11:36:32 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> Message-ID: On Wed, Sep 20, 2017 at 11:12 AM, Adam Lindberg wrote: > Hi Richard and Lloyd, > > We target smaller embedded systems that could benefit from Erlang, whereas on a Raspberry Pi (even zero) or Beagleboard you can easily run Linux and Erlang on top. The goal is to be able to work as much as possible with hardware in Erlang. We combine Wi-Fi with support more standard connectors, such as ports for PMODs, I?C, 1-Wire an so on. Our 300 Mhz CPU uses much less energy and the board is aimed at being an evaluation board for (then cheaper) embedded systems. Booting system from SD Card will not make it in production. I think you'll need bigger non-volatile storage on the board. And why RTEMS when there is now Zephyr and RIOT with all necessary IoT libs - like MQTT and CoAP? > As for the price, mainly it's a matter of scale. We suspect Raspberry Pi still looses money on their production, selling units at a loss. We are prepared to loose money on the development, but we are selling the hardware at cost. RPi has deals with Broadcom, I doubt that they loose money. BR, Drasko From erlang@REDACTED Wed Sep 20 12:22:03 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 12:22:03 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. Message-ID: Hello, I want to ship Erlang as a component inside a larger application to run on Windows Assuming the larger application has reserved space for Erlang at some destination directory $DEST then I need to copy a subset of the files from a standard distribution into $DEST. On a Mac I have reduced the number of necessary files to 11 (could be 10 if you remove start.script) ??? COPYRIGHT ??? bin ? ??? beam.smp ? ??? erl_child_setup ? ??? erlexec ? ??? start.boot ? ??? start.script ??? erl ??? lib ??? compiler-7.0.2.ez ??? kernel-5.1.ez ??? sasl-3.0.1.ez ??? stdlib-3.1.ez I've thrown away all the libraries apart from compiler etc. I'm trying (unsuccessfully) to do the same thing on windows - does anybody know how to make a minimal Erlang for windows? - which files must be copied and which files must be patched???? As an afterthought - it would be *very nice* to have a program which makes a minimal distributable program for Windows/Mac/Linux. I was thinking along the lines of running the program twice - in run one I'd trace all calls to discover which modules were actually called, then from this information build a minimal distributable program. Cheers /Joe From stu.bailey@REDACTED Wed Sep 20 12:35:29 2017 From: stu.bailey@REDACTED (Stu Bailey) Date: Wed, 20 Sep 2017 03:35:29 -0700 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: "As an afterthought - it would be *very nice* to have a program which makes a minimal distributable program for Windows/Mac/Linux." Joe, It would be nice if Erlang had an official Docker image (e.g. https://docs.docker.com/docker-for-windows/ ). Such an image would fulfill exactly your requirement. Here are the official Docker images for other languagesl: https://hub.docker.com/_/python/ https://hub.docker.com/_/golang/ https://hub.docker.com/_/rust/ https://hub.docker.com/_/clojure/ https://hub.docker.com/_/java/ I guess this is the start: https://github.com/erlang/docker-erlang Note this work: https://store.docker.com/community/images/library/rabbitmq Stu On Wed, Sep 20, 2017 at 3:22 AM, Joe Armstrong wrote: > Hello, > > I want to ship Erlang as a component inside a > larger application to run on Windows > > Assuming the larger application has reserved space > for Erlang at some destination directory $DEST then I need > to copy a subset of the files from a standard > distribution into $DEST. > > On a Mac I have reduced the number of necessary files to 11 > (could be 10 if you remove start.script) > > ??? COPYRIGHT > ??? bin > ? ??? beam.smp > ? ??? erl_child_setup > ? ??? erlexec > ? ??? start.boot > ? ??? start.script > ??? erl > ??? lib > ??? compiler-7.0.2.ez > ??? kernel-5.1.ez > ??? sasl-3.0.1.ez > ??? stdlib-3.1.ez > > I've thrown away all the libraries apart from compiler etc. > > I'm trying (unsuccessfully) to do the same thing on windows - does anybody > know how to make a minimal Erlang for windows? - which files must be > copied and which files must be patched???? > > As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux. > > I was thinking along the lines of running the program twice - in run one > I'd trace all calls to discover which modules were actually called, then > from this information build a minimal distributable program. > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Wed Sep 20 12:40:20 2017 From: ali.sabil@REDACTED (Ali Sabil) Date: Wed, 20 Sep 2017 12:40:20 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Like this one: https://hub.docker.com/_/erlang/ ? On Wed, Sep 20, 2017 at 12:35 PM, Stu Bailey wrote: > "As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux." > > Joe, > > It would be nice if Erlang had an official Docker image (e.g. > https://docs.docker.com/docker-for-windows/ ). Such an image would > fulfill exactly your requirement. Here are the official Docker images for > other languagesl: > > https://hub.docker.com/_/python/ > https://hub.docker.com/_/golang/ > https://hub.docker.com/_/rust/ > https://hub.docker.com/_/clojure/ > https://hub.docker.com/_/java/ > > I guess this is the start: > > https://github.com/erlang/docker-erlang > > Note this work: > > https://store.docker.com/community/images/library/rabbitmq > > Stu > > On Wed, Sep 20, 2017 at 3:22 AM, Joe Armstrong wrote: > >> Hello, >> >> I want to ship Erlang as a component inside a >> larger application to run on Windows >> >> Assuming the larger application has reserved space >> for Erlang at some destination directory $DEST then I need >> to copy a subset of the files from a standard >> distribution into $DEST. >> >> On a Mac I have reduced the number of necessary files to 11 >> (could be 10 if you remove start.script) >> >> ??? COPYRIGHT >> ??? bin >> ? ??? beam.smp >> ? ??? erl_child_setup >> ? ??? erlexec >> ? ??? start.boot >> ? ??? start.script >> ??? erl >> ??? lib >> ??? compiler-7.0.2.ez >> ??? kernel-5.1.ez >> ??? sasl-3.0.1.ez >> ??? stdlib-3.1.ez >> >> I've thrown away all the libraries apart from compiler etc. >> >> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >> know how to make a minimal Erlang for windows? - which files must be >> copied and which files must be patched???? >> >> As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux. >> >> I was thinking along the lines of running the program twice - in run one >> I'd trace all calls to discover which modules were actually called, then >> from this information build a minimal distributable program. >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Wed Sep 20 12:43:08 2017 From: stu.bailey@REDACTED (Stu Bailey) Date: Wed, 20 Sep 2017 03:43:08 -0700 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: :-) Yes like that! :-) Awesome On Wed, Sep 20, 2017 at 3:40 AM, Ali Sabil wrote: > Like this one: https://hub.docker.com/_/erlang/ ? > > On Wed, Sep 20, 2017 at 12:35 PM, Stu Bailey wrote: > >> "As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux." >> >> Joe, >> >> It would be nice if Erlang had an official Docker image (e.g. >> https://docs.docker.com/docker-for-windows/ ). Such an image would >> fulfill exactly your requirement. Here are the official Docker images for >> other languagesl: >> >> https://hub.docker.com/_/python/ >> https://hub.docker.com/_/golang/ >> https://hub.docker.com/_/rust/ >> https://hub.docker.com/_/clojure/ >> https://hub.docker.com/_/java/ >> >> I guess this is the start: >> >> https://github.com/erlang/docker-erlang >> >> Note this work: >> >> https://store.docker.com/community/images/library/rabbitmq >> >> Stu >> >> On Wed, Sep 20, 2017 at 3:22 AM, Joe Armstrong wrote: >> >>> Hello, >>> >>> I want to ship Erlang as a component inside a >>> larger application to run on Windows >>> >>> Assuming the larger application has reserved space >>> for Erlang at some destination directory $DEST then I need >>> to copy a subset of the files from a standard >>> distribution into $DEST. >>> >>> On a Mac I have reduced the number of necessary files to 11 >>> (could be 10 if you remove start.script) >>> >>> ??? COPYRIGHT >>> ??? bin >>> ? ??? beam.smp >>> ? ??? erl_child_setup >>> ? ??? erlexec >>> ? ??? start.boot >>> ? ??? start.script >>> ??? erl >>> ??? lib >>> ??? compiler-7.0.2.ez >>> ??? kernel-5.1.ez >>> ??? sasl-3.0.1.ez >>> ??? stdlib-3.1.ez >>> >>> I've thrown away all the libraries apart from compiler etc. >>> >>> I'm trying (unsuccessfully) to do the same thing on windows - does >>> anybody >>> know how to make a minimal Erlang for windows? - which files must be >>> copied and which files must be patched???? >>> >>> As an afterthought - it would be *very nice* to have a program which >>> makes a minimal distributable program for Windows/Mac/Linux. >>> >>> I was thinking along the lines of running the program twice - in run one >>> I'd trace all calls to discover which modules were actually called, then >>> from this information build a minimal distributable program. >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Sep 20 12:46:00 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 12:46:00 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: On Wed, Sep 20, 2017 at 12:35 PM, Stu Bailey wrote: > "As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux." > > Joe, > > It would be nice if Erlang had an official Docker image (e.g. > https://docs.docker.com/docker-for-windows/ ). Such an image would fulfill > exactly your requirement. I think this is a separate issue (though a good idea for Erlang developers) In my use case we are targeting nieve windows users who won't have Docker installed > Here are the official Docker images for other > languagesl: > > https://hub.docker.com/_/python/ > https://hub.docker.com/_/golang/ > https://hub.docker.com/_/rust/ > https://hub.docker.com/_/clojure/ > https://hub.docker.com/_/java/ > > I guess this is the start: > > https://github.com/erlang/docker-erlang > > Note this work: > > https://store.docker.com/community/images/library/rabbitmq > > Stu > > On Wed, Sep 20, 2017 at 3:22 AM, Joe Armstrong wrote: >> >> Hello, >> >> I want to ship Erlang as a component inside a >> larger application to run on Windows >> >> Assuming the larger application has reserved space >> for Erlang at some destination directory $DEST then I need >> to copy a subset of the files from a standard >> distribution into $DEST. >> >> On a Mac I have reduced the number of necessary files to 11 >> (could be 10 if you remove start.script) >> >> ??? COPYRIGHT >> ??? bin >> ? ??? beam.smp >> ? ??? erl_child_setup >> ? ??? erlexec >> ? ??? start.boot >> ? ??? start.script >> ??? erl >> ??? lib >> ??? compiler-7.0.2.ez >> ??? kernel-5.1.ez >> ??? sasl-3.0.1.ez >> ??? stdlib-3.1.ez >> >> I've thrown away all the libraries apart from compiler etc. >> >> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >> know how to make a minimal Erlang for windows? - which files must be >> copied and which files must be patched???? >> >> As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux. >> >> I was thinking along the lines of running the program twice - in run one >> I'd trace all calls to discover which modules were actually called, then >> from this information build a minimal distributable program. >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From matthias@REDACTED Wed Sep 20 13:03:05 2017 From: matthias@REDACTED (Matthias Lang) Date: Wed, 20 Sep 2017 13:03:05 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: <20170920110305.ds3wqhtiqft5w3mi@hec.corelatus.se> On 20. September 2017, Joe Armstrong wrote: > I was thinking along the lines of running the program twice - in run one > I'd trace all calls to discover which modules were actually called, then > from this information build a minimal distributable program. code:all_loaded() is an easy way to see which modules were loaded by a particular run of a program. What's harder is being confident that your program won't call further modules when given different input. A pragmatic approach is to run a test suite with 'sufficient' coverage. Matthias From zxq9@REDACTED Wed Sep 20 13:11:45 2017 From: zxq9@REDACTED (zxq9) Date: Wed, 20 Sep 2017 20:11:45 +0900 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: <2606917.BfgbS6XQcm@changa> On 2017?09?20? ??? 12:46:00 Joe Armstrong wrote: > On Wed, Sep 20, 2017 at 12:35 PM, Stu Bailey wrote: > > "As an afterthought - it would be *very nice* to have a program which > > makes a minimal distributable program for Windows/Mac/Linux." > > > > Joe, > > > > It would be nice if Erlang had an official Docker image (e.g. > > https://docs.docker.com/docker-for-windows/ ). Such an image would fulfill > > exactly your requirement. > > I think this is a separate issue (though a good idea for Erlang developers) > > In my use case we are targeting nieve > windows users who won't have Docker installed This. The world doesn't revolve around Docker. -Craig From vladdu55@REDACTED Wed Sep 20 13:15:50 2017 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Sep 2017 13:15:50 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Hi Joe, I got it started by adding the erts-9.0/bin directory and editing the bin/erl.ini file with the right path. Am trying to trim more, but my PC is acting weird. Regards, Vlad On 20 Sep 2017 12:22, "Joe Armstrong" wrote: > Hello, > > I want to ship Erlang as a component inside a > larger application to run on Windows > > Assuming the larger application has reserved space > for Erlang at some destination directory $DEST then I need > to copy a subset of the files from a standard > distribution into $DEST. > > On a Mac I have reduced the number of necessary files to 11 > (could be 10 if you remove start.script) > > ??? COPYRIGHT > ??? bin > ? ??? beam.smp > ? ??? erl_child_setup > ? ??? erlexec > ? ??? start.boot > ? ??? start.script > ??? erl > ??? lib > ??? compiler-7.0.2.ez > ??? kernel-5.1.ez > ??? sasl-3.0.1.ez > ??? stdlib-3.1.ez > > I've thrown away all the libraries apart from compiler etc. > > I'm trying (unsuccessfully) to do the same thing on windows - does anybody > know how to make a minimal Erlang for windows? - which files must be > copied and which files must be patched???? > > As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux. > > I was thinking along the lines of running the program twice - in run one > I'd trace all calls to discover which modules were actually called, then > from this information build a minimal distributable program. > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Sep 20 13:25:50 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 13:25:50 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: This is what I tried - actually even if it works you might accidentally be picking up files from the original C:/Program Files/erl ... directory. I looked at the Wings3D distribution - this has managed to hide away a complete erlang - but how it finds the correct directories to load the libraries from is a mystery to me /Joe On Wed, Sep 20, 2017 at 1:15 PM, Vlad Dumitrescu wrote: > Hi Joe, > I got it started by adding the erts-9.0/bin directory and editing the > bin/erl.ini file with the right path. Am trying to trim more, but my PC is > acting weird. > Regards, > Vlad > > > On 20 Sep 2017 12:22, "Joe Armstrong" wrote: >> >> Hello, >> >> I want to ship Erlang as a component inside a >> larger application to run on Windows >> >> Assuming the larger application has reserved space >> for Erlang at some destination directory $DEST then I need >> to copy a subset of the files from a standard >> distribution into $DEST. >> >> On a Mac I have reduced the number of necessary files to 11 >> (could be 10 if you remove start.script) >> >> ??? COPYRIGHT >> ??? bin >> ? ??? beam.smp >> ? ??? erl_child_setup >> ? ??? erlexec >> ? ??? start.boot >> ? ??? start.script >> ??? erl >> ??? lib >> ??? compiler-7.0.2.ez >> ??? kernel-5.1.ez >> ??? sasl-3.0.1.ez >> ??? stdlib-3.1.ez >> >> I've thrown away all the libraries apart from compiler etc. >> >> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >> know how to make a minimal Erlang for windows? - which files must be >> copied and which files must be patched???? >> >> As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux. >> >> I was thinking along the lines of running the program twice - in run one >> I'd trace all calls to discover which modules were actually called, then >> from this information build a minimal distributable program. >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From sergej.jurecko@REDACTED Wed Sep 20 13:28:16 2017 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Wed, 20 Sep 2017 13:28:16 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: <228EDB1B-E041-4282-9B68-AD390A10E87A@gmail.com> You can just delete the erl.ini file. That way the erl/werl will work no matter where you put the top level erlang folder. You can definitely also delete: - all .pdb files - all *.debug.*.dll - all .lib files - all source (.c, .h, .erl, .hrl) files A simple filelib:wildcard comprehension with delete across erl folder will achieve this quickly. Regards, Sergek > On 20 Sep 2017, at 13:15, Vlad Dumitrescu wrote: > > Hi Joe, > I got it started by adding the erts-9.0/bin directory and editing the bin/erl.ini file with the right path. Am trying to trim more, but my PC is acting weird. > Regards, > Vlad > > > On 20 Sep 2017 12:22, "Joe Armstrong" > wrote: > Hello, > > I want to ship Erlang as a component inside a > larger application to run on Windows > > Assuming the larger application has reserved space > for Erlang at some destination directory $DEST then I need > to copy a subset of the files from a standard > distribution into $DEST. > > On a Mac I have reduced the number of necessary files to 11 > (could be 10 if you remove start.script) > > ??? COPYRIGHT > ??? bin > ? ??? beam.smp > ? ??? erl_child_setup > ? ??? erlexec > ? ??? start.boot > ? ??? start.script > ??? erl > ??? lib > ??? compiler-7.0.2.ez > ??? kernel-5.1.ez > ??? sasl-3.0.1.ez > ??? stdlib-3.1.ez > > I've thrown away all the libraries apart from compiler etc. > > I'm trying (unsuccessfully) to do the same thing on windows - does anybody > know how to make a minimal Erlang for windows? - which files must be > copied and which files must be patched???? > > As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux. > > I was thinking along the lines of running the program twice - in run one > I'd trace all calls to discover which modules were actually called, then > from this information build a minimal distributable program. > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Sep 20 13:28:49 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Sep 2017 14:28:49 +0300 Subject: [erlang-questions] Logging from nif (and especially from rust nif) In-Reply-To: References: Message-ID: It also don't break current input and give more synchronized printing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Sep 20 13:42:46 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 13:42:46 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Making some progress - I have copied *everything* from the erlang distribution to my new location and edited erl.ini - the system now seems to start correctly from the new location. Now I need to start deleting things, until I discover I've deleted too much and broken it - if somebody knows which files I have to keep it would be a great help :-) /Joe On Wed, Sep 20, 2017 at 1:15 PM, Vlad Dumitrescu wrote: > Hi Joe, > I got it started by adding the erts-9.0/bin directory and editing the > bin/erl.ini file with the right path. Am trying to trim more, but my PC is > acting weird. > Regards, > Vlad > > > On 20 Sep 2017 12:22, "Joe Armstrong" wrote: >> >> Hello, >> >> I want to ship Erlang as a component inside a >> larger application to run on Windows >> >> Assuming the larger application has reserved space >> for Erlang at some destination directory $DEST then I need >> to copy a subset of the files from a standard >> distribution into $DEST. >> >> On a Mac I have reduced the number of necessary files to 11 >> (could be 10 if you remove start.script) >> >> ??? COPYRIGHT >> ??? bin >> ? ??? beam.smp >> ? ??? erl_child_setup >> ? ??? erlexec >> ? ??? start.boot >> ? ??? start.script >> ??? erl >> ??? lib >> ??? compiler-7.0.2.ez >> ??? kernel-5.1.ez >> ??? sasl-3.0.1.ez >> ??? stdlib-3.1.ez >> >> I've thrown away all the libraries apart from compiler etc. >> >> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >> know how to make a minimal Erlang for windows? - which files must be >> copied and which files must be patched???? >> >> As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux. >> >> I was thinking along the lines of running the program twice - in run one >> I'd trace all calls to discover which modules were actually called, then >> from this information build a minimal distributable program. >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Sep 20 13:51:37 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 13:51:37 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: <228EDB1B-E041-4282-9B68-AD390A10E87A@gmail.com> References: <228EDB1B-E041-4282-9B68-AD390A10E87A@gmail.com> Message-ID: On Wed, Sep 20, 2017 at 1:28 PM, Sergej Jure?ko wrote: > You can just delete the erl.ini file. That way the erl/werl will work no > matter where you put the top level erlang folder. Yes - it worked - But how does erl.exe find the dll's ?? > You can definitely also delete: > - all .pdb files > - all *.debug.*.dll > - all .lib files > - all source (.c, .h, .erl, .hrl) files How much of the bin directories do I have to keep? I just want erlang with networking and a shell no distribution - /Joe > > A simple filelib:wildcard comprehension with delete across erl folder will > achieve this quickly. > > Regards, > Sergek > > > On 20 Sep 2017, at 13:15, Vlad Dumitrescu wrote: > > Hi Joe, > I got it started by adding the erts-9.0/bin directory and editing the > bin/erl.ini file with the right path. Am trying to trim more, but my PC is > acting weird. > Regards, > Vlad > > > On 20 Sep 2017 12:22, "Joe Armstrong" wrote: >> >> Hello, >> >> I want to ship Erlang as a component inside a >> larger application to run on Windows >> >> Assuming the larger application has reserved space >> for Erlang at some destination directory $DEST then I need >> to copy a subset of the files from a standard >> distribution into $DEST. >> >> On a Mac I have reduced the number of necessary files to 11 >> (could be 10 if you remove start.script) >> >> ??? COPYRIGHT >> ??? bin >> ? ??? beam.smp >> ? ??? erl_child_setup >> ? ??? erlexec >> ? ??? start.boot >> ? ??? start.script >> ??? erl >> ??? lib >> ??? compiler-7.0.2.ez >> ??? kernel-5.1.ez >> ??? sasl-3.0.1.ez >> ??? stdlib-3.1.ez >> >> I've thrown away all the libraries apart from compiler etc. >> >> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >> know how to make a minimal Erlang for windows? - which files must be >> copied and which files must be patched???? >> >> As an afterthought - it would be *very nice* to have a program which >> makes a minimal distributable program for Windows/Mac/Linux. >> >> I was thinking along the lines of running the program twice - in run one >> I'd trace all calls to discover which modules were actually called, then >> from this information build a minimal distributable program. >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From erlang@REDACTED Wed Sep 20 13:59:41 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 13:59:41 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: <228EDB1B-E041-4282-9B68-AD390A10E87A@gmail.com> Message-ID: Seems like I need to keep erl.exe start.boot beam.smp.dll erlexec.dll And throw the rest away - seems to work (so far) - does anybody know If I've thrown way too much /Joe On Wed, Sep 20, 2017 at 1:51 PM, Joe Armstrong wrote: > On Wed, Sep 20, 2017 at 1:28 PM, Sergej Jure?ko > wrote: >> You can just delete the erl.ini file. That way the erl/werl will work no >> matter where you put the top level erlang folder. > > Yes - it worked - But how does erl.exe find the dll's ?? > > > >> You can definitely also delete: >> - all .pdb files >> - all *.debug.*.dll >> - all .lib files >> - all source (.c, .h, .erl, .hrl) files > > How much of the bin directories do I have to keep? > > I just want erlang with networking and a shell no distribution - > > > /Joe >> >> A simple filelib:wildcard comprehension with delete across erl folder will >> achieve this quickly. >> >> Regards, >> Sergek >> >> >> On 20 Sep 2017, at 13:15, Vlad Dumitrescu wrote: >> >> Hi Joe, >> I got it started by adding the erts-9.0/bin directory and editing the >> bin/erl.ini file with the right path. Am trying to trim more, but my PC is >> acting weird. >> Regards, >> Vlad >> >> >> On 20 Sep 2017 12:22, "Joe Armstrong" wrote: >>> >>> Hello, >>> >>> I want to ship Erlang as a component inside a >>> larger application to run on Windows >>> >>> Assuming the larger application has reserved space >>> for Erlang at some destination directory $DEST then I need >>> to copy a subset of the files from a standard >>> distribution into $DEST. >>> >>> On a Mac I have reduced the number of necessary files to 11 >>> (could be 10 if you remove start.script) >>> >>> ??? COPYRIGHT >>> ??? bin >>> ? ??? beam.smp >>> ? ??? erl_child_setup >>> ? ??? erlexec >>> ? ??? start.boot >>> ? ??? start.script >>> ??? erl >>> ??? lib >>> ??? compiler-7.0.2.ez >>> ??? kernel-5.1.ez >>> ??? sasl-3.0.1.ez >>> ??? stdlib-3.1.ez >>> >>> I've thrown away all the libraries apart from compiler etc. >>> >>> I'm trying (unsuccessfully) to do the same thing on windows - does anybody >>> know how to make a minimal Erlang for windows? - which files must be >>> copied and which files must be patched???? >>> >>> As an afterthought - it would be *very nice* to have a program which >>> makes a minimal distributable program for Windows/Mac/Linux. >>> >>> I was thinking along the lines of running the program twice - in run one >>> I'd trace all calls to discover which modules were actually called, then >>> from this information build a minimal distributable program. >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> From drasko.draskovic@REDACTED Wed Sep 20 14:01:57 2017 From: drasko.draskovic@REDACTED (Drasko DRASKOVIC) Date: Wed, 20 Sep 2017 14:01:57 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <535D2BCB-30CB-46AF-AA44-D358A5A776C0@stritzinger.com> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> <535D2BCB-30CB-46AF-AA44-D358A5A776C0@stritzinger.com> Message-ID: On Wed, Sep 20, 2017 at 1:20 PM, Peer Stritzinger wrote: > Hi Drasko, > >> On 20.09.2017, at 11:36, Drasko DRASKOVIC wrote: >> >> On Wed, Sep 20, 2017 at 11:12 AM, Adam Lindberg wrote: >>> Hi Richard and Lloyd, >>> >>> We target smaller embedded systems that could benefit from Erlang, whereas on a Raspberry Pi (even zero) or Beagleboard you can easily run Linux and Erlang on top. The goal is to be able to work as much as possible with hardware in Erlang. We combine Wi-Fi with support more standard connectors, such as ports for PMODs, I?C, 1-Wire an so on. Our 300 Mhz CPU uses much less energy and the board is aimed at being an evaluation board for (then cheaper) embedded systems. >> >> Booting system from SD Card will not make it in production. I think >> you'll need bigger non-volatile storage on the board. > > We have a board with the same software stack in production for a industrial customer for about 5 year. > > This board has soldered on NAND flash, but that means you need a Flash filesystem. > > We use yaffs2 but this is GPL2 dual license and we are paying for being free from GPL in our product. > SD Card is good in production for certain use cases but for some one wants NAND flash on board. > > This is a eval and maker board to build prototypes and have fun with. And yes it can be used in production for certain use cases. > > In most production cases I would build a specialized board which have all the sensors on board and not use PMODs for them > and also boot from local NAND flash. > > GRiSP-base is meant as a starting point for that. And to get started a SD-card is *very* useful Prototype - yes, SD card is useful. Production - no, SD cards break often. They are not ment to keep your system files and that you boot your system from them. They are secondary storage. RPi is totally useless in production because of this. There are industrial SD cards, but they are very expensive and do not justify not having flash soldered on the board. > >> And why RTEMS when there is now Zephyr and RIOT with all necessary IoT >> libs - like MQTT and CoAP? > > The question could be stated the other way round too. > > RTEMS is around since the 1980ties and very robust. > > Its very usably in industrial and automotive and of course also for IoT (whatever that may be) also its Space rated and used on a lot of space probes of NASA and ESA. > > One other thing that makes it useful to us: it has solid SMP support and embedded systems are having more cores more and more. > > And allowing access to SMP is kind of useful for Erlang. Not available in GRiSP base since we have a single core but we are using this as > platform on smaller systems than GRiSP base and larger ones. What you see on www.grisp.org is just what we put out so everyone can get started quickly and have as much fun as we have :-) > > But there are hundreds of embedded OS with all kinds of properties, you could pick many for porting Erlang to. > > The main goal is that the embedded OS we use as basis steps aside and Erlang looks like the OS. I understand this, but this is exactly why these modern RTOSes are made for: https://www.zephyrproject.org/ and https://riot-os.org/. These are big modern projects that try to solve various problems existing with old RTOSes, especially on communication and networking side. IoT communication protocols are not fun to port: https://lists.rtems.org/pipermail/devel/2017-March/017240.html Sticking with weird licensing: https://www.rtems.org/license instead with industry-adopted Apache-2.0 is also an error IMHO. Best regards, Drasko From peer@REDACTED Wed Sep 20 13:20:20 2017 From: peer@REDACTED (Peer Stritzinger) Date: Wed, 20 Sep 2017 13:20:20 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> Message-ID: <535D2BCB-30CB-46AF-AA44-D358A5A776C0@stritzinger.com> Hi Drasko, > On 20.09.2017, at 11:36, Drasko DRASKOVIC wrote: > > On Wed, Sep 20, 2017 at 11:12 AM, Adam Lindberg wrote: >> Hi Richard and Lloyd, >> >> We target smaller embedded systems that could benefit from Erlang, whereas on a Raspberry Pi (even zero) or Beagleboard you can easily run Linux and Erlang on top. The goal is to be able to work as much as possible with hardware in Erlang. We combine Wi-Fi with support more standard connectors, such as ports for PMODs, I?C, 1-Wire an so on. Our 300 Mhz CPU uses much less energy and the board is aimed at being an evaluation board for (then cheaper) embedded systems. > > Booting system from SD Card will not make it in production. I think > you'll need bigger non-volatile storage on the board. We have a board with the same software stack in production for a industrial customer for about 5 year. This board has soldered on NAND flash, but that means you need a Flash filesystem. We use yaffs2 but this is GPL2 dual license and we are paying for being free from GPL in our product. SD Card is good in production for certain use cases but for some one wants NAND flash on board. This is a eval and maker board to build prototypes and have fun with. And yes it can be used in production for certain use cases. In most production cases I would build a specialized board which have all the sensors on board and not use PMODs for them and also boot from local NAND flash. GRiSP-base is meant as a starting point for that. And to get started a SD-card is *very* useful > And why RTEMS when there is now Zephyr and RIOT with all necessary IoT > libs - like MQTT and CoAP? The question could be stated the other way round too. RTEMS is around since the 1980ties and very robust. Its very usably in industrial and automotive and of course also for IoT (whatever that may be) also its Space rated and used on a lot of space probes of NASA and ESA. One other thing that makes it useful to us: it has solid SMP support and embedded systems are having more cores more and more. And allowing access to SMP is kind of useful for Erlang. Not available in GRiSP base since we have a single core but we are using this as platform on smaller systems than GRiSP base and larger ones. What you see on www.grisp.org is just what we put out so everyone can get started quickly and have as much fun as we have :-) But there are hundreds of embedded OS with all kinds of properties, you could pick many for porting Erlang to. The main goal is that the embedded OS we use as basis steps aside and Erlang looks like the OS. >> As for the price, mainly it's a matter of scale. We suspect Raspberry Pi still looses money on their production, selling units at a loss. We are prepared to loose money on the development, but we are selling the hardware at cost. > > RPi has deals with Broadcom, I doubt that they loose money. RPi is a charity with rich donors, we are a small company. We are selling the boards at cost and donated and open sourced the software stack. Price can go down if you buy a lot of boards ;-) Best, -- Peer > > BR, > Drasko > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jozsef.berces@REDACTED Wed Sep 20 14:23:49 2017 From: jozsef.berces@REDACTED (=?utf-8?B?SsOzenNlZiBCw6lyY2Vz?=) Date: Wed, 20 Sep 2017 12:23:49 +0000 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: <228EDB1B-E041-4282-9B68-AD390A10E87A@gmail.com> Message-ID: Hi Joe, Here is what I have bundled with a tool used mainly inside Ericsson: erl |_ bin | |_ beam.dll | |_ beam.smp.dll | |_ epmd.exe | |_ erl.exe | |_ erl.ini | |_ erlexec.dll | |_ erlsrv.exe | |_ inet_gethost.exe | |_ libeay32.dll | |_ start.boot |_ lib |_ erts-8.1 |_ ... |_ kernel-5.1 |_ ... |_ ... What libraries to include depends on the application's needs. You might also add MSVCR120.dll into the bin directory just in case your target users do not have it installed. BR, Jozsef -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Joe Armstrong Sent: Wednesday, September 20, 2017 14:00 To: Sergej Jure?ko Cc: erlang-questions Subject: Re: [erlang-questions] Making a minimal distributable program in Erlang. Seems like I need to keep erl.exe start.boot beam.smp.dll erlexec.dll And throw the rest away - seems to work (so far) - does anybody know If I've thrown way too much /Joe On Wed, Sep 20, 2017 at 1:51 PM, Joe Armstrong wrote: > On Wed, Sep 20, 2017 at 1:28 PM, Sergej Jure?ko > wrote: >> You can just delete the erl.ini file. That way the erl/werl will work >> no matter where you put the top level erlang folder. > > Yes - it worked - But how does erl.exe find the dll's ?? > > > >> You can definitely also delete: >> - all .pdb files >> - all *.debug.*.dll >> - all .lib files >> - all source (.c, .h, .erl, .hrl) files > > How much of the bin directories do I have to keep? > > I just want erlang with networking and a shell no distribution - > > > /Joe >> >> A simple filelib:wildcard comprehension with delete across erl folder >> will achieve this quickly. >> >> Regards, >> Sergek >> >> >> On 20 Sep 2017, at 13:15, Vlad Dumitrescu wrote: >> >> Hi Joe, >> I got it started by adding the erts-9.0/bin directory and editing the >> bin/erl.ini file with the right path. Am trying to trim more, but my >> PC is acting weird. >> Regards, >> Vlad >> >> >> On 20 Sep 2017 12:22, "Joe Armstrong" wrote: >>> >>> Hello, >>> >>> I want to ship Erlang as a component inside a larger application to >>> run on Windows >>> >>> Assuming the larger application has reserved space for Erlang at >>> some destination directory $DEST then I need to copy a subset of the >>> files from a standard distribution into $DEST. >>> >>> On a Mac I have reduced the number of necessary files to 11 (could >>> be 10 if you remove start.script) >>> >>> ??? COPYRIGHT >>> ??? bin >>> ? ??? beam.smp >>> ? ??? erl_child_setup >>> ? ??? erlexec >>> ? ??? start.boot >>> ? ??? start.script >>> ??? erl >>> ??? lib >>> ??? compiler-7.0.2.ez >>> ??? kernel-5.1.ez >>> ??? sasl-3.0.1.ez >>> ??? stdlib-3.1.ez >>> >>> I've thrown away all the libraries apart from compiler etc. >>> >>> I'm trying (unsuccessfully) to do the same thing on windows - does >>> anybody know how to make a minimal Erlang for windows? - which files >>> must be copied and which files must be patched???? >>> >>> As an afterthought - it would be *very nice* to have a program which >>> makes a minimal distributable program for Windows/Mac/Linux. >>> >>> I was thinking along the lines of running the program twice - in run >>> one I'd trace all calls to discover which modules were actually >>> called, then from this information build a minimal distributable program. >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Wed Sep 20 14:24:50 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 20 Sep 2017 15:24:50 +0300 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Hello, > On 20 Sep 2017, at 13.35, Stu Bailey wrote: > > "As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux." This program is called relx. It generates a release package which are deployable to any vanilla linux or other vanilla os. The usage of Docker-image-with-Erlang complicates the process of distribution. Best Regards, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Sep 20 16:23:11 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 20 Sep 2017 16:23:11 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Thanks everybody for you help - I got it down to 5.5MB in 9 files Relocated as follows. I've also packed/compressed the libraries Here are the files that are needed %% DEST/bin/erl.exe 120320 bytes %% DEST/bin/start.boot 5603 bytes %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes %% DEST/lib/compiler-7.1.ez 409091 bytes %% DEST/lib/kernel-5.2.ez 384730 bytes %% DEST/lib/sasl-3.0.4.ez 106278 bytes %% DEST/lib/stdlib-3.4.ez 913013 bytes This seems to work :-) Cheers /Joe On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov wrote: > Hello, > > On 20 Sep 2017, at 13.35, Stu Bailey wrote: > > "As an afterthought - it would be *very nice* to have a program which > makes a minimal distributable program for Windows/Mac/Linux." > > > This program is called relx. > > It generates a release package which are deployable to any vanilla linux or > other vanilla os. > The usage of Docker-image-with-Erlang complicates the process of > distribution. > > Best Regards, > Dmitry > > From frank.muller.erl@REDACTED Wed Sep 20 16:34:50 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 20 Sep 2017 14:34:50 +0000 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Do you think it's possible to package this minimal distribution in one excutable? Imagine we can ship a whole release in one binary (like in Go). /Frank Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : > Thanks everybody for you help - I got it down to 5.5MB in 9 files > Relocated as follows. I've also packed/compressed the libraries > > Here are the files that are needed > > %% DEST/bin/erl.exe 120320 bytes > %% DEST/bin/start.boot 5603 bytes > %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes > %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes > %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes > %% DEST/lib/compiler-7.1.ez 409091 bytes > %% DEST/lib/kernel-5.2.ez 384730 bytes > %% DEST/lib/sasl-3.0.4.ez 106278 bytes > %% DEST/lib/stdlib-3.4.ez 913013 bytes > > This seems to work :-) > > Cheers > > /Joe > > On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov > wrote: > > Hello, > > > > On 20 Sep 2017, at 13.35, Stu Bailey wrote: > > > > "As an afterthought - it would be *very nice* to have a program which > > makes a minimal distributable program for Windows/Mac/Linux." > > > > > > This program is called relx. > > > > It generates a release package which are deployable to any vanilla linux > or > > other vanilla os. > > The usage of Docker-image-with-Erlang complicates the process of > > distribution. > > > > Best Regards, > > Dmitry > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peer@REDACTED Wed Sep 20 17:30:03 2017 From: peer@REDACTED (Peer Stritzinger) Date: Wed, 20 Sep 2017 17:30:03 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1C6165B7-437A-4E7D-97E2-DDADDC6A7285@alind.io> <535D2BCB-30CB-46AF-AA44-D358A5A776C0@stritzinger.com> Message-ID: <19999D4D-9A54-4D73-A13E-62E94E592CF9@stritzinger.com> > On 20.09.2017, at 14:01, Drasko DRASKOVIC wrote: > > On Wed, Sep 20, 2017 at 1:20 PM, Peer Stritzinger wrote: >> Hi Drasko, >> >>> On 20.09.2017, at 11:36, Drasko DRASKOVIC wrote: >>> >>> On Wed, Sep 20, 2017 at 11:12 AM, Adam Lindberg wrote: >>>> Hi Richard and Lloyd, >>>> >>>> We target smaller embedded systems that could benefit from Erlang, whereas on a Raspberry Pi (even zero) or Beagleboard you can easily run Linux and Erlang on top. The goal is to be able to work as much as possible with hardware in Erlang. We combine Wi-Fi with support more standard connectors, such as ports for PMODs, I?C, 1-Wire an so on. Our 300 Mhz CPU uses much less energy and the board is aimed at being an evaluation board for (then cheaper) embedded systems. >>> >>> Booting system from SD Card will not make it in production. I think >>> you'll need bigger non-volatile storage on the board. >> >> We have a board with the same software stack in production for a industrial customer for about 5 year. >> >> This board has soldered on NAND flash, but that means you need a Flash filesystem. >> >> We use yaffs2 but this is GPL2 dual license and we are paying for being free from GPL in our product. >> SD Card is good in production for certain use cases but for some one wants NAND flash on board. >> >> This is a eval and maker board to build prototypes and have fun with. And yes it can be used in production for certain use cases. >> >> In most production cases I would build a specialized board which have all the sensors on board and not use PMODs for them >> and also boot from local NAND flash. >> >> GRiSP-base is meant as a starting point for that. And to get started a SD-card is *very* useful > > Prototype - yes, SD card is useful. Production - no, SD cards break > often. They are not ment to keep your system files and that you boot > your system from them. They are secondary storage. RPi is totally > useless in production because of this. There are industrial SD cards, > but they are very expensive and do not justify not having flash > soldered on the board. I disagree, having built many products. For small batch size industrial SD cards can be better than soldered on flash. If you don want to GPL your embedded application you have to pay about 5k ? for licensing yaffs2 ? for that I can buy quite a bit of expensive SD-cards. And rPI is useless for production anyway because they have quality issues and frequent new versions. Which is not a problem since it was meant only for teaching. >>> And why RTEMS when there is now Zephyr and RIOT with all necessary IoT >>> libs - like MQTT and CoAP? >> >> The question could be stated the other way round too. >> >> RTEMS is around since the 1980ties and very robust. >> >> Its very usably in industrial and automotive and of course also for IoT (whatever that may be) also its Space rated and used on a lot of space probes of NASA and ESA. >> >> One other thing that makes it useful to us: it has solid SMP support and embedded systems are having more cores more and more. >> >> And allowing access to SMP is kind of useful for Erlang. Not available in GRiSP base since we have a single core but we are using this as >> platform on smaller systems than GRiSP base and larger ones. What you see on www.grisp.org is just what we put out so everyone can get started quickly and have as much fun as we have :-) >> >> But there are hundreds of embedded OS with all kinds of properties, you could pick many for porting Erlang to. >> >> The main goal is that the embedded OS we use as basis steps aside and Erlang looks like the OS. > > I understand this, but this is exactly why these modern RTOSes are > made for: https://www.zephyrproject.org/ and https://riot-os.org/. > These are big modern projects that try to solve various problems > existing with old RTOSes, especially on communication and networking > side. Well RTEMS in the version we use has a recent SMP capable FreeBSD network stack. We like old and seasoned better for embedded than new and big projects which pop up and disappear quickly. Thats what we like about Erlang too. > IoT communication protocols are not fun to port: > https://lists.rtems.org/pipermail/devel/2017-March/017240.html Well who wants the protocol in C when we have Erlang?? > Sticking with weird licensing: https://www.rtems.org/license instead > with industry-adopted Apache-2.0 is also an error IMHO. Well whats your problem with that license? It is an approved license for many large companies who are our customers. And not long ago Erlang had a non mainstream license too. But thanks for telling us we are doing it wrong for years ;-) Have a nice day, -- Peer > > Best regards, > Drasko -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.14159@REDACTED Wed Sep 20 17:31:22 2017 From: j.14159@REDACTED (Jeremy Pierre) Date: Wed, 20 Sep 2017 15:31:22 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: Begging the list's indulgence with tangential and possibly off-topic content. On Tue, Sep 19, 2017 at 7:57 AM Daniel Goertzen wrote: > How does Alpaca compare to F#/Ocaml? After tasting some Rust and Elm, > working in Erlang makes me a bit nervous too. My dream is to see a BEAM > ML-like achieve Elixir stature. > > In terms of syntax it's closest to OCaml but it's worth noting we still lack some significant features one would be used to, e.g. ML's module system (and functors, etc) and you'll see hints of Elm in some places like our records. We kept things like adding fields to records, e.g. let record_example () = let r1 = {x="hello", y="world"} in {pi=3.14 | r1} is perfectly legal without declaring those record types in advance. r1: {x: string, y: string} record_example: {pi: float, x: string, y: string} There's a basic tour of the language here if you're further curious: https://github.com/alpaca-lang/alpaca/blob/master/Tour.md The pending v0.2.8 release (held up by me being super slow on a couple of bugs I should have fixed months ago) adds some substantial stuff like type annotations and better compiler feedback (both community contributions!) Jeremy > > On Tue, Sep 19, 2017 at 8:57 AM Karl Nilsson wrote: > >> On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe >>> wrote: >>> >>>> What's your opinion of F#? It seems to be the language >>>> of choice for people who liked Caml but need to take advantage >>>> of multiple cores. I haven't done any benchmarking; I doubt >>>> that it could match OCaml in raw speed. >>>> >>>> >>> I have not used it enough to have an opinion (yet). Were I to >>> communicate a lot with the .NET platform, I'd probably pick it because it >>> has a null value and this is a necessity when talking to C# I'm told. >>> >>> Given that it runs under a pretty powerful JIT, it could perform really >>> well for a lot of tasks I think. >>> >>> >> F# doesn't typically have any speed advantages of any other .NET >> languages and in every comparison I've seen to OCaml it has performed >> worse. If anything the allocation costs induced by a functional first >> programming style means it is typically a bit slower than the equivalent C# >> code (also there is no goto). >> >> As a language F# is the nicest I've ever used substantially. I find it >> easy (and fun) to write reasonably correct code in. Also I hardly ever fear >> refactoring (compared to erlang where I break out in cold sweats even for >> code bases that pass dialyzer). >> >> I even like it so much I've started hacking on an fsharp to core erlang >> compiler. (https://github.com/kjnilsson/fez). >> >> >> >>> My "dream" would be an industry-supported parallel MLton :P >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drasko.draskovic@REDACTED Wed Sep 20 17:36:26 2017 From: drasko.draskovic@REDACTED (Drasko DRASKOVIC) Date: Wed, 20 Sep 2017 17:36:26 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller wrote: > Do you think it's possible to package this minimal distribution in one > excutable? > > Imagine we can ship a whole release in one binary (like in Go). +1 for this because we could put it in the "FROM_SCRATCH" Docker container and have minimal size, something like this: https://github.com/shortishly/erlang-in-docker-from-scratch BR, Drasko From frank.muller.erl@REDACTED Wed Sep 20 18:39:25 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 20 Sep 2017 16:39:25 +0000 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: Thanks again Dmytro for your time. Let assume I?m inside my NIF and would like to increment the refcount of a binary (hope this is the right call: enif_keep_resource). Can i increment refcount for let say 1 million binaries (or even more)? What are the consequences of doing that? I assume I'll release them at the end. /Frank >From what I remember (some people who recently have worked with binaries > might know much more) > > Binaries in binary heap (large bins) have own structure, i think the > master storage structure is called Binary. This is confusing even to me > so have a spoon of salt with this advice :) > > https://github.com/erlang/otp/blob/master/erts/emulator/beam/erl_binary.h > https://github.com/erlang/otp/blob/master/erts/emulator/beam/binary.c > > Binaries stored outside processes (proc-bins ProcBin), smaller than 64b > (proc-bins, structure ErlHeapBin i think) and sub-bins (ErlSubBin) have > own tag bit combinations on the process heap. So the GC and VM always know > what they have found. They might share part of C structure pointing to the > external binary, you can observe it as a macro included in the respective > structures, think is called ErtsMagicBinary. Macros is_not_binary, binary_size, > binary_bytes and ERTS_GET_REAL_BIN might become useful. > > Also ref-bins all contain 'next' pointer, as a part of long linked list > which begins somewhere in process header (one such list belongs to every > process) and during GC this list is traversed to find out which ref-bins > are about to die and refcount is decremented. > > 2017-09-20 6:08 GMT+02:00 Frank Muller : > >> Hi Dmytro, >> >> Thanks for the hint. I found a lot of references to these keywords ... >> bit lost. >> >> How about the rest of the questions? >> >> /Frank >> >> Tue. Sept. 19 2017 ? 22:13, Dmytro Lytovchenko < >> dmytro.lytovchenko@REDACTED> wrote : >> >>> Look for binary, procbin, refbin (probably), and sub binary. Use this as >>> keywords >>> >>> On 19 Sep 2017 21:50, "Frank Muller" wrote: >>> >>>> Hi guys >>>> >>>> Newbie here? >>>> >>>> Can someone point me please to the C data structure representing small >>>> and large binaries inside the VM? >>>> Do they?ve the same C representation (or not)? >>>> How the VM distinguishes between them internally? >>>> >>>> Thank you. >>>> /Frank >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Thu Sep 21 00:09:07 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 20 Sep 2017 18:09:07 -0400 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: References: Message-ID: <15ea1566164-c04-1fec4@webjas-vac175.srv.aolmail.net> Thanks for the tips Lukas. That process killing issue did actually bite me in the ar*e a few seemingly random times. Luckily, for my use-case, the processes depending on the controlled hot-swapping are known so I was able to mitigate against this by keeping track of the which versions each process knows about. - Edmond - On Tuesday, 19 September 2017 at 19:13 Adam Lindberg wrote: Cost wise I'm not sure, but I would advice you to benchmark for your use case. Operations wise the most common problem you'll see with module reloads is crashed processes. Erlang only keeps two versions of a module in memory at the same time. If you have processes lingering in the old code when reloading yet another version, those processes will get killed. Cheers, Adam > On 16 Sep 2017, at 23:57, Edmond Begumisa wrote: > > On Sun, 17 Sep 2017 03:39:50 +1000, Edmond Begumisa wrote: > >> Hello all, >> >> (Not new here--just haven't posted in a while!) >> >> I'm using merl to dynamically generate and regenerate modules at runtime that make use of the ERTS constant pool in order to efficiently avail data to all processes. I'm aware that this technique is known amongst more experienced Erlangers for situations where the data is updated infrequently (e.g. sharing global configuration parameters). >> >> The question I'm investigating is how frequent is "infrequent" w.r.t reloading of modules[1]? I'm not entirely sure about the costs involved when (re)loading a module so I've got no idea where to start in order to measure the impact on the emulator. Could anybody with sufficient knowledge of emulator internals point me in the right direction? >> > > To clarify: When measuring whether or not I'm hot-code swapping too frequently, what are the negative signs to look for? What does the emulator actually do when swapping out code w.r.t potentially disruptive/costly operations like locking, GC, etc which could negatively impact scheduling, memory and responsiveness of the system? > > I'm trying to figure out when the best time to reload a dynamically generated module/group of modules is and how frequently I should allow reloading to occur. > > - Edmond - > >> Thanks in advance. >> >> - Edmond - >> >> [1] The more exotic use-cases for use/abuse of the constant pool and module reloading I'm looking into are to do with caching and incremental computation. >> > > > -- > Using Opera's mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Thu Sep 21 00:10:56 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 20 Sep 2017 18:10:56 -0400 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: <15ea1566164-c04-1fec4@webjas-vac175.srv.aolmail.net> References: <15ea1566164-c04-1fec4@webjas-vac175.srv.aolmail.net> Message-ID: <15ea1580bf4-c0a-167f8@webjas-vac193.srv.aolmail.net> Sorry, I meant Adam not Lukas! - Edmond - On Thursday, 21 September 2017 at 08:09 Edmond Begumisa wrote: Thanks for the tips Lukas. That process killing issue did actually bite me in the ar*e a few seemingly random times. Luckily, for my use-case, the processes depending on the controlled hot-swapping are known so I was able to mitigate against this by keeping track of the which versions each process knows about. - Edmond - On Tuesday, 19 September 2017 at 19:13 Adam Lindberg wrote: Cost wise I'm not sure, but I would advice you to benchmark for your use case. Operations wise the most common problem you'll see with module reloads is crashed processes. Erlang only keeps two versions of a module in memory at the same time. If you have processes lingering in the old code when reloading yet another version, those processes will get killed. Cheers, Adam > On 16 Sep 2017, at 23:57, Edmond Begumisa wrote: > > On Sun, 17 Sep 2017 03:39:50 +1000, Edmond Begumisa wrote: > >> Hello all, >> >> (Not new here--just haven't posted in a while!) >> >> I'm using merl to dynamically generate and regenerate modules at runtime that make use of the ERTS constant pool in order to efficiently avail data to all processes. I'm aware that this technique is known amongst more experienced Erlangers for situations where the data is updated infrequently (e.g. sharing global configuration parameters). >> >> The question I'm investigating is how frequent is "infrequent" w.r.t reloading of modules[1]? I'm not entirely sure about the costs involved when (re)loading a module so I've got no idea where to start in order to measure the impact on the emulator. Could anybody with sufficient knowledge of emulator internals point me in the right direction? >> > > To clarify: When measuring whether or not I'm hot-code swapping too frequently, what are the negative signs to look for? What does the emulator actually do when swapping out code w.r.t potentially disruptive/costly operations like locking, GC, etc which could negatively impact scheduling, memory and responsiveness of the system? > > I'm trying to figure out when the best time to reload a dynamically generated module/group of modules is and how frequently I should allow reloading to occur. > > - Edmond - > >> Thanks in advance. >> >> - Edmond - >> >> [1] The more exotic use-cases for use/abuse of the constant pool and module reloading I'm looking into are to do with caching and incremental computation. >> > > > -- > Using Opera's mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Thu Sep 21 00:23:01 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 20 Sep 2017 18:23:01 -0400 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: null References: null Message-ID: <15ea1631d50-c05-11e40@webjas-vaa103.srv.aolmail.net> Thanks Lukas, That is very very helpful information as it not only gives me knowledge of what to benchmark, it also gives me some surprising insights on how to reorganise my (now obviously naive) code. Apart from GC, is there anything else I should be concerned about like costly locks and memory barriers? For instance, is a fat mutex held for all schedulers when scanning all the process heaps? - Edmond - On Tuesday, 19 September 2017 at 19:40 Lukas Larsson wrote: On Tue, Sep 19, 2017 at 11:13 AM, Adam Lindberg wrote: Cost wise I'm not sure, but I would advice you to benchmark for your use case. Every time you purge a module, each process heap is scanned for any sign of the old literals (aka constants) and if a literal is found a GC is done of that process which copies the literal onto its heap. So the cost depends on the total heap size of all running processes and the number of processes that have a reference to those literals on their heaps at the moment of the purge. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Sep 21 08:19:38 2017 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 21 Sep 2017 08:19:38 +0200 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: <15ea1631d50-c05-11e40@webjas-vaa103.srv.aolmail.net> References: <15ea1631d50-c05-11e40@webjas-vaa103.srv.aolmail.net> Message-ID: On Thu, Sep 21, 2017 at 12:23 AM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > Apart from GC, is there anything else I should be concerned about like > costly locks and memory barriers? For instance, is a fat mutex held for all > schedulers when scanning all the process heaps? > No, there are no locks taken. Exactly how it is done is described here: https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CodeLoading.md -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 21 11:00:02 2017 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 21 Sep 2017 11:00:02 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: Yes^1000 I would very much like this to happen. The windows version I put together was 5.5 MB in 9 files The emulator is 3.5 MB and all the erlang code (compiled and compressed a MB or so) - so realistically we could make an stand-along program in c. 5MB Things like Hugo (static site generator in GO) are single file executables - Hugo is massive 18MB on my machine (probably a lot of templates - who knows?) I did an experiment a while back with self-modifing executables Basicicall the structure of a executable is: Stub 3.5 MB Data Len bytes Len 4 bytes Magic 4 bytes The Stub is always the same and gets copied from app to app. Following the sub is the program. It's just Data (compress beam code) Then a length counter (length of Data) then a magic number. When Stub starts it reads itself, checks there's a magic byte on the end, if so reads the Length of the Data, then the data and then boot's itself. (This is how PDF files work, reading backwards from the end - it's nice because you could always daisy-chain things on the end) How do we write Stub? - this needs "yet another top-level program to start Erlang" What you can't do is include the erlang DLL's in the Data area of the program I think you have to statically link *everything* into the stub. I don't know how squashable the object code is - it might be nice to compress the object code for (Mac,Windows,Linux) and put them in the Data area of the program, then decompress them and load them on-the-fly. This kind of code goes way beyond my knowledge of the low-level code needed to boot C applications. I have a feeling it might not be a lot of code - but it's (really) tricky stuff. So my first question is - how can I compile Erlang into a single file executable? - I want no scripts/batch files to launch erlang no DLLs and a statically linked binary? /Joe On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller wrote: > Do you think it's possible to package this minimal distribution in one > excutable? > > Imagine we can ship a whole release in one binary (like in Go). > > /Frank > > Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : >> >> Thanks everybody for you help - I got it down to 5.5MB in 9 files >> Relocated as follows. I've also packed/compressed the libraries >> >> Here are the files that are needed >> >> %% DEST/bin/erl.exe 120320 bytes >> %% DEST/bin/start.boot 5603 bytes >> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >> %% DEST/lib/compiler-7.1.ez 409091 bytes >> %% DEST/lib/kernel-5.2.ez 384730 bytes >> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >> %% DEST/lib/stdlib-3.4.ez 913013 bytes >> >> This seems to work :-) >> >> Cheers >> >> /Joe >> >> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >> wrote: >> > Hello, >> > >> > On 20 Sep 2017, at 13.35, Stu Bailey wrote: >> > >> > "As an afterthought - it would be *very nice* to have a program which >> > makes a minimal distributable program for Windows/Mac/Linux." >> > >> > >> > This program is called relx. >> > >> > It generates a release package which are deployable to any vanilla linux >> > or >> > other vanilla os. >> > The usage of Docker-image-with-Erlang complicates the process of >> > distribution. >> > >> > Best Regards, >> > Dmitry >> > >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Thu Sep 21 12:10:56 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 21 Sep 2017 12:10:56 +0200 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: On 09/20/2017 06:39 PM, Frank Muller wrote: > Thanks again Dmytro for your time. > > Let assume I?m inside my NIF and would like to increment the refcount of a > binary (hope this is the right call: enif_keep_resource). No, enif_keep_resource does not work on binaries. "Resources" are a concept of their own and binaries are not resources. Unfortunately there is no enif_keep_binary. The only use for such a function is if you want to keep binaries alive even no processes are referring to them. And you would need some native data structure with pointers to those binaries, in order to find them without a binary term. /Sverker From jesper.louis.andersen@REDACTED Thu Sep 21 12:23:06 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Sep 2017 10:23:06 +0000 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> Message-ID: On Wed, Sep 20, 2017 at 12:50 AM Richard A. O'Keefe wrote: > One question. I can get a > "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" > for NZD 69 including tax and run Erlang on it. > (Hmm. Next time I have some spare cash I must get one...) > > Why would I pay NZD300 for a 300MHz board with 64MB of memory? > > My prediction is that eventually we'll see the "small embedded" market squeezed down by rather cheap powerful devices such as the rPI. My argument for this prediction hinges on power becoming free in the future (due to the current massive advances in solar and wind energy on top of better batteries). Currently, the power draw of the stronger chips is too high and production at scale means that shaving off a few cents here and there is worth it. But it looks like the price tag of a 1.2 GHZ 64 bit chip is only going one way, and this makes the added development effort of targeting a smaller chip more expensive. In turn, you need an even bigger scale of production. And so on. GRiSP is interesting because it can offer you a far faster time to market and lower development effort at the same or better robustness/resilience. You can thus get the lower production costs while amortizing development effort better. It is a good bet until the niche squeeze I predict above happens. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Thu Sep 21 12:34:19 2017 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Thu, 21 Sep 2017 12:34:19 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> > > So my first question is - how can I compile Erlang into a single file > executable? - I want no scripts/batch files to launch erlang no DLLs > and a statically linked binary? Getting rid of scripts/batch files you can do if you generalize the erlexec binary a bit (in Docker for instance, as it was mentioned before, you perhaps want to be able to configure more things with environment variables for instance) but what would "statically linked" mean in the world of modules that load shared objects and even modules that can be upgraded? To statically link one would have to sacrifice the dynamic code reload, right? /Stefan > > /Joe > > > > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller > wrote: >> Do you think it's possible to package this minimal distribution in one >> excutable? >> >> Imagine we can ship a whole release in one binary (like in Go). >> >> /Frank >> >> Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : >>> >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files >>> Relocated as follows. I've also packed/compressed the libraries >>> >>> Here are the files that are needed >>> >>> %% DEST/bin/erl.exe 120320 bytes >>> %% DEST/bin/start.boot 5603 bytes >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >>> %% DEST/lib/compiler-7.1.ez 409091 bytes >>> %% DEST/lib/kernel-5.2.ez 384730 bytes >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes >>> >>> This seems to work :-) >>> >>> Cheers >>> >>> /Joe >>> >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >>> wrote: >>>> Hello, >>>> >>>> On 20 Sep 2017, at 13.35, Stu Bailey wrote: >>>> >>>> "As an afterthought - it would be *very nice* to have a program which >>>> makes a minimal distributable program for Windows/Mac/Linux." >>>> >>>> >>>> This program is called relx. >>>> >>>> It generates a release package which are deployable to any vanilla linux >>>> or >>>> other vanilla os. >>>> The usage of Docker-image-with-Erlang complicates the process of >>>> distribution. >>>> >>>> Best Regards, >>>> Dmitry >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From frank.muller.erl@REDACTED Thu Sep 21 13:15:24 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 21 Sep 2017 11:15:24 +0000 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> Message-ID: Stefan, By statically linking bins, Joe is talking about the executables here (erl.exe, ...), not the beam files IMHO. You'll be able to reload your beam modules on the fly, obviously. Moreover, nothing prevent a statically built executable to load a shared lib while running. @Joe: just made a quick search and found UPX binary packer. It's cross-platform: https://upx.github.io/ Can you please try to shrink the minimal VM binaries on Windows and report back the new size? Use: upx.exe -9 ... /Frank > > > > So my first question is - how can I compile Erlang into a single file > > executable? - I want no scripts/batch files to launch erlang no DLLs > > and a statically linked binary? > > Getting rid of scripts/batch files you can do if you generalize the > erlexec binary a bit (in Docker for instance, as it was mentioned before, > you perhaps want to be able to configure more things with environment > variables for instance) but what would "statically linked" mean in the > world of modules that load shared objects and even modules that can be > upgraded? To statically link one would have to sacrifice the dynamic code > reload, right? > > /Stefan > > > > > /Joe > > > > > > > > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller > > wrote: > >> Do you think it's possible to package this minimal distribution in one > >> excutable? > >> > >> Imagine we can ship a whole release in one binary (like in Go). > >> > >> /Frank > >> > >> Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : > >>> > >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files > >>> Relocated as follows. I've also packed/compressed the libraries > >>> > >>> Here are the files that are needed > >>> > >>> %% DEST/bin/erl.exe 120320 bytes > >>> %% DEST/bin/start.boot 5603 bytes > >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes > >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes > >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes > >>> %% DEST/lib/compiler-7.1.ez 409091 bytes > >>> %% DEST/lib/kernel-5.2.ez 384730 bytes > >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes > >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes > >>> > >>> This seems to work :-) > >>> > >>> Cheers > >>> > >>> /Joe > >>> > >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov > >>> wrote: > >>>> Hello, > >>>> > >>>> On 20 Sep 2017, at 13.35, Stu Bailey wrote: > >>>> > >>>> "As an afterthought - it would be *very nice* to have a program which > >>>> makes a minimal distributable program for Windows/Mac/Linux." > >>>> > >>>> > >>>> This program is called relx. > >>>> > >>>> It generates a release package which are deployable to any vanilla > linux > >>>> or > >>>> other vanilla os. > >>>> The usage of Docker-image-with-Erlang complicates the process of > >>>> distribution. > >>>> > >>>> Best Regards, > >>>> Dmitry > >>>> > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Sep 21 13:22:05 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Sep 2017 11:22:05 +0000 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> Message-ID: I think one of the reasons for the directory structure has to do with hot-code-upgrades. It is way easier if your system is able to load stuff individually from files rather than having to work with static binaries. Static binaries would be nice. But a full implementation must also be able to load patches into the binary later on, if you decide to upgrade the running code. This may not be that simple to pull off, though ELF can do tricks. On Thu, Sep 21, 2017 at 1:15 PM Frank Muller wrote: > Stefan, > > By statically linking bins, Joe is talking about the executables here > (erl.exe, ...), not the beam files IMHO. > > You'll be able to reload your beam modules on the fly, obviously. > > Moreover, nothing prevent a statically built executable to load a shared > lib while running. > > @Joe: just made a quick search and found UPX binary packer. It's > cross-platform: > https://upx.github.io/ > > Can you please try to shrink the minimal VM binaries on Windows and report > back the new size? > Use: upx.exe -9 ... > > /Frank > > > >> > >> > So my first question is - how can I compile Erlang into a single file >> > executable? - I want no scripts/batch files to launch erlang no DLLs >> > and a statically linked binary? >> >> Getting rid of scripts/batch files you can do if you generalize the >> erlexec binary a bit (in Docker for instance, as it was mentioned before, >> you perhaps want to be able to configure more things with environment >> variables for instance) but what would "statically linked" mean in the >> world of modules that load shared objects and even modules that can be >> upgraded? To statically link one would have to sacrifice the dynamic code >> reload, right? >> >> /Stefan >> >> > >> > /Joe >> > >> > >> > >> > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller >> > wrote: >> >> Do you think it's possible to package this minimal distribution in one >> >> excutable? >> >> >> >> Imagine we can ship a whole release in one binary (like in Go). >> >> >> >> /Frank >> >> >> >> Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : >> >>> >> >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files >> >>> Relocated as follows. I've also packed/compressed the libraries >> >>> >> >>> Here are the files that are needed >> >>> >> >>> %% DEST/bin/erl.exe 120320 bytes >> >>> %% DEST/bin/start.boot 5603 bytes >> >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >> >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >> >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >> >>> %% DEST/lib/compiler-7.1.ez 409091 bytes >> >>> %% DEST/lib/kernel-5.2.ez 384730 bytes >> >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >> >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes >> >>> >> >>> This seems to work :-) >> >>> >> >>> Cheers >> >>> >> >>> /Joe >> >>> >> >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >> >>> wrote: >> >>>> Hello, >> >>>> >> >>>> On 20 Sep 2017, at 13.35, Stu Bailey wrote: >> >>>> >> >>>> "As an afterthought - it would be *very nice* to have a program which >> >>>> makes a minimal distributable program for Windows/Mac/Linux." >> >>>> >> >>>> >> >>>> This program is called relx. >> >>>> >> >>>> It generates a release package which are deployable to any vanilla >> linux >> >>>> or >> >>>> other vanilla os. >> >>>> The usage of Docker-image-with-Erlang complicates the process of >> >>>> distribution. >> >>>> >> >>>> Best Regards, >> >>>> Dmitry >> >>>> >> >>>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Thu Sep 21 16:23:46 2017 From: list1@REDACTED (Grzegorz Junka) Date: Thu, 21 Sep 2017 14:23:46 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> Message-ID: <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> Many thanks to all for your responses and suggestions. To answer your question Richard, I don't actually need a cyclical structure. As I mentioned, this is a simplification of a more specific problem. I didn't want to describe all the specifics to ask a question. I need to design a tree, a variant of a HAMT tree, but which can be traversed in both directions, from the root to leafs and from leafs to the root (independently, i.e. starting from a reference to a leaf rather than recursively). Tuples that reference each other is one solution but not the only one fortunately, i.e. I can add ids to each node and use another data structure to store the mapping id-node, then store the id of the parent in the child. Thank you for the pointer to the book. GrzegorzJ On 18/09/2017 00:53, Richard A. O'Keefe wrote: > On further reflection, I suppose I should ask: > Why is it that you think you need cyclic structures? > What do you actually want to *do* with them? > > As a matter of curiosity, have you read Okasaki's > book on functional data structures? > > (Didn't someone convert Okasaki's Edison library > to Erlang at some point? What happened to that?) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Sep 21 16:54:13 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 21 Sep 2017 14:54:13 +0000 Subject: [erlang-questions] Small and large (>=64B) binaries? How do they differ? In-Reply-To: References: Message-ID: Sverker, Okay thanks for the clarifications. How about the opposite: an Erlang process which holds pointers to a large number of RefCounted Bins ? Is there any performance penalty (ex. affected by GC) of doing that? /Frank > > On 09/20/2017 06:39 PM, Frank Muller wrote: > > Thanks again Dmytro for your time. > > > > Let assume I?m inside my NIF and would like to increment the refcount of > a > > binary (hope this is the right call: enif_keep_resource). > No, enif_keep_resource does not work on binaries. > "Resources" are a concept of their own and binaries are not resources. > > Unfortunately there is no enif_keep_binary. The only use for such a > function > is if you want to keep binaries alive even no processes are referring to > them. > And you would need some native data structure with pointers to those > binaries, > in order to find them without a binary term. > > /Sverker > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peer@REDACTED Thu Sep 21 17:30:05 2017 From: peer@REDACTED (Peer Stritzinger) Date: Thu, 21 Sep 2017 17:30:05 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> Message-ID: <1B1D5F1D-6D25-4751-ACA0-075E513A4146@stritzinger.com> > On 21.09.2017, at 12:23, Jesper Louis Andersen wrote: > > On Wed, Sep 20, 2017 at 12:50 AM Richard A. O'Keefe > wrote: > One question. I can get a > "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" > for NZD 69 including tax and run Erlang on it. > (Hmm. Next time I have some spare cash I must get one...) > > Why would I pay NZD300 for a 300MHz board with 64MB of memory? > > > My prediction is that eventually we'll see the "small embedded" market squeezed down by rather cheap powerful devices such as the rPI. My argument for this prediction hinges on power becoming free in the future (due to the current massive advances in solar and wind energy on top of better batteries). Small embedded being current rPI *sized* CPUs for some definition of small will be happening. Except rPI is not usable in production for several reasons (supply, quality, hardware changing beyond your control). > Currently, the power draw of the stronger chips is too high and production at scale means that shaving off a few cents here and there is worth it. Exactly. We have a few industrial applications where we would love to get a larger CPU but we can?t because of price and power (meaning in this case thermal energy output). > But it looks like the price tag of a 1.2 GHZ 64 bit chip is only going one way, and this makes the added development effort of targeting a smaller chip more expensive. In turn, you need an even bigger scale of production. And so on. Except that with a 300MHz Cortex M7 size CPU you can do lots of interesting applications just in Erlang already and save the costs. > GRiSP is interesting because it can offer you a far faster time to market and lower development effort at the same or better robustness/resilience. You can thus get the lower production costs while amortizing development effort better. It is a good bet until the niche squeeze I predict above happens. Thanks :-) Also we plan not to be squeezed but adapt. One size which is just getting viable with Erlang are the automotive controllers with flash and ram on the CPU only (for the automotive industry any external memory is too unreliable and a problem for EMC). Thats one direction what I want to target with the GRiSP software stack. We could have easily also targeted a 24 core communication controller with 16 times Gbit I/O which is a tempting target also and noone would complain about the compute power there ;-) For the GRiSP-base we just tried to pick the right sized board for today. Others might follow? Best, -- Peer > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From peer@REDACTED Thu Sep 21 17:45:47 2017 From: peer@REDACTED (Peer Stritzinger) Date: Thu, 21 Sep 2017 17:45:47 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> Message-ID: <1F720108-859A-42BE-A652-F893905AD41B@stritzinger.com> > On 21.09.2017, at 13:15, Frank Muller wrote: > > Stefan, > > By statically linking bins, Joe is talking about the executables here (erl.exe, ...), not the beam files IMHO. But what you can do is preload all needed beam files, which means they get baked into your erts executable. So you can even have statically linked beam files. Every Erlang system has some of those since quite a lot of the system startup is done in Erlang already. And to get to the point where the runtime can load beam files you need some preexisting beam files. When building this is done by this perl script and its surrounding Makefile magic: erts/emulator/utils/make_preload Which generates a C file called preload.c starting like: /* * DO NOT EDIT THIS FILE. It was automatically generated * by the `make_preload' program on Fri Jun 23 12:55:01 2017. */ unsigned preloaded_size_otp_ring0 = 532; unsigned char preloaded_otp_ring0[] = { 0x46,0x4f,0x52,0x31,0x00,0x00,0x02,0x0c, /* FOR1.... */ 0x42,0x45,0x41,0x4d,0x41,0x74,0x6f,0x6d, /* BEAMAtom */ 0x00,0x00,0x00,0x82,0x00,0x00,0x00,0x10, /* ........ */ 0x09,0x6f,0x74,0x70,0x5f,0x72,0x69,0x6e, /* .otp_rin */ 0x67,0x30,0x05,0x73,0x74,0x61,0x72,0x74, /* g0.start */ 0x04,0x62,0x6f,0x6f,0x74,0x04,0x69,0x6e, /* .boot.in */ 0x69,0x74,0x03,0x72,0x75,0x6e,0x06,0x65, /* it.run.e */ 0x72,0x6c,0x61,0x6e,0x67,0x11,0x66,0x75, /* rlang.fu */ 0x6e,0x63,0x74,0x69,0x6f,0x6e,0x5f,0x65, /* nction_e */ 0x78,0x70,0x6f,0x72,0x74,0x65,0x64,0x05, /* xported. */ 0x66,0x61,0x6c,0x73,0x65,0x04,0x74,0x72, /* false.tr */ 0x75,0x65,0x05,0x66,0x61,0x74,0x61,0x6c, /* ue.fatal */ 0x05,0x65,0x72,0x72,0x6f,0x72,0x06,0x6d, /* .error.m */ 0x6f,0x64,0x75,0x6c,0x65,0x07,0x64,0x69, /* odule.di */ And I suppose that you still can hot code load preloaded files, its just that their memory isn?t freed since it is in the erts executable. Best, -- Peer > > You'll be able to reload your beam modules on the fly, obviously. > > Moreover, nothing prevent a statically built executable to load a shared lib while running. > > @Joe: just made a quick search and found UPX binary packer. It's cross-platform: > https://upx.github.io/ > > Can you please try to shrink the minimal VM binaries on Windows and report back the new size? > Use: upx.exe -9 ... > > /Frank > > > > > > > So my first question is - how can I compile Erlang into a single file > > executable? - I want no scripts/batch files to launch erlang no DLLs > > and a statically linked binary? > > Getting rid of scripts/batch files you can do if you generalize the erlexec binary a bit (in Docker for instance, as it was mentioned before, you perhaps want to be able to configure more things with environment variables for instance) but what would "statically linked" mean in the world of modules that load shared objects and even modules that can be upgraded? To statically link one would have to sacrifice the dynamic code reload, right? > > /Stefan > > > > > /Joe > > > > > > > > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller > > > wrote: > >> Do you think it's possible to package this minimal distribution in one > >> excutable? > >> > >> Imagine we can ship a whole release in one binary (like in Go). > >> > >> /Frank > >> > >> Wed 20 sept. 2017 at 16:23, Joe Armstrong > wrote : > >>> > >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files > >>> Relocated as follows. I've also packed/compressed the libraries > >>> > >>> Here are the files that are needed > >>> > >>> %% DEST/bin/erl.exe 120320 bytes > >>> %% DEST/bin/start.boot 5603 bytes > >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes > >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes > >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes > >>> %% DEST/lib/compiler-7.1.ez 409091 bytes > >>> %% DEST/lib/kernel-5.2.ez 384730 bytes > >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes > >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes > >>> > >>> This seems to work :-) > >>> > >>> Cheers > >>> > >>> /Joe > >>> > >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov > >>> > wrote: > >>>> Hello, > >>>> > >>>> On 20 Sep 2017, at 13.35, Stu Bailey > wrote: > >>>> > >>>> "As an afterthought - it would be *very nice* to have a program which > >>>> makes a minimal distributable program for Windows/Mac/Linux." > >>>> > >>>> > >>>> This program is called relx. > >>>> > >>>> It generates a release package which are deployable to any vanilla linux > >>>> or > >>>> other vanilla os. > >>>> The usage of Docker-image-with-Erlang complicates the process of > >>>> distribution. > >>>> > >>>> Best Regards, > >>>> Dmitry > >>>> > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From powers_brady@REDACTED Thu Sep 21 17:20:58 2017 From: powers_brady@REDACTED (Brady Powers) Date: Thu, 21 Sep 2017 15:20:58 +0000 (UTC) Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> Message-ID: <1029578522.6888872.1506007258526@mail.yahoo.com> When I think about a data structure that needs to be traversed in both directions, I immediately think of a zipper. https://ferd.ca/yet-another-article-on-zippers.html It might not be appropriate, it's just where my mind goes.? On Thursday, September 21, 2017, 10:24:17 AM EDT, Grzegorz Junka wrote: Many thanks to all for your responses and suggestions. To answer your question Richard, I don't actually need a cyclical structure. As I mentioned, this is a simplification of a more specific problem. I didn't want to describe all the specifics to ask a question. I need to design a tree, a variant of a HAMT tree, but which can be traversed in both directions, from the root to leafs and from leafs to the root (independently, i.e. starting from a reference to a leaf rather than recursively). Tuples that reference each other is one solution but not the only one fortunately, i.e. I can add ids to each node and use another data structure to store the mapping id-node, then store the id of the parent in the child. Thank you for the pointer to the book. GrzegorzJ On 18/09/2017 00:53, Richard A. O'Keefe wrote: On further reflection, I suppose I should ask: ?Why is it that you think you need cyclic structures? ?What do you actually want to *do* with them? As a matter of curiosity, have you read Okasaki's book on functional data structures? (Didn't someone convert Okasaki's Edison library to Erlang at some point?? What happened to that?) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.khlopotov@REDACTED Thu Sep 21 18:11:24 2017 From: ilya.khlopotov@REDACTED (ILYA Khlopotov) Date: Thu, 21 Sep 2017 09:11:24 -0700 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: <1F720108-859A-42BE-A652-F893905AD41B@stritzinger.com> References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> <1F720108-859A-42BE-A652-F893905AD41B@stritzinger.com> Message-ID: hi, I wanted a single file distribution of Erlang programs for a long time. I was thinking to implement it as follows: 1. extend the loader for *.ez files to support offset 2. extend erlexec to support detection of offset in the executable. - in order to avoid patching otp files I was thinking to use -eentry flag for linker to pick main function from my own file. 3. use any technique to bundle beam and *.ez - in Linux the files can be just concatenated - I don't know how to make it crossplatform is there any documentation how to use make_preload or upx to make the solution cross platform? best regards, ILYA On September 21, 2017 8:45:47 AM PDT, Peer Stritzinger wrote: > >> On 21.09.2017, at 13:15, Frank Muller >wrote: >> >> Stefan, >> >> By statically linking bins, Joe is talking about the executables here >(erl.exe, ...), not the beam files IMHO. > >But what you can do is preload all needed beam files, which means they >get baked into your erts executable. > >So you can even have statically linked beam files. Every Erlang system >has some of those since quite a lot >of the system startup is done in Erlang already. And to get to the >point where the runtime can load beam files >you need some preexisting beam files. > >When building this is done by this perl script and its surrounding >Makefile magic: > >erts/emulator/utils/make_preload > >Which generates a C file called preload.c starting like: > >/* > * DO NOT EDIT THIS FILE. It was automatically generated > * by the `make_preload' program on Fri Jun 23 12:55:01 2017. > */ >unsigned preloaded_size_otp_ring0 = 532; >unsigned char preloaded_otp_ring0[] = { >0x46,0x4f,0x52,0x31,0x00,0x00,0x02,0x0c, /* FOR1.... */ > 0x42,0x45,0x41,0x4d,0x41,0x74,0x6f,0x6d, /* BEAMAtom */ > 0x00,0x00,0x00,0x82,0x00,0x00,0x00,0x10, /* ........ */ > 0x09,0x6f,0x74,0x70,0x5f,0x72,0x69,0x6e, /* .otp_rin */ > 0x67,0x30,0x05,0x73,0x74,0x61,0x72,0x74, /* g0.start */ > 0x04,0x62,0x6f,0x6f,0x74,0x04,0x69,0x6e, /* .boot.in */ > 0x69,0x74,0x03,0x72,0x75,0x6e,0x06,0x65, /* it.run.e */ > 0x72,0x6c,0x61,0x6e,0x67,0x11,0x66,0x75, /* rlang.fu */ > 0x6e,0x63,0x74,0x69,0x6f,0x6e,0x5f,0x65, /* nction_e */ > 0x78,0x70,0x6f,0x72,0x74,0x65,0x64,0x05, /* xported. */ > 0x66,0x61,0x6c,0x73,0x65,0x04,0x74,0x72, /* false.tr */ > 0x75,0x65,0x05,0x66,0x61,0x74,0x61,0x6c, /* ue.fatal */ > 0x05,0x65,0x72,0x72,0x6f,0x72,0x06,0x6d, /* .error.m */ > 0x6f,0x64,0x75,0x6c,0x65,0x07,0x64,0x69, /* odule.di */ > >And I suppose that you still can hot code load preloaded files, its >just that their memory isn?t freed >since it is in the erts executable. > >Best, >-- Peer > > >> >> You'll be able to reload your beam modules on the fly, obviously. >> >> Moreover, nothing prevent a statically built executable to load a >shared lib while running. >> >> @Joe: just made a quick search and found UPX binary packer. It's >cross-platform: >> https://upx.github.io/ >> >> Can you please try to shrink the minimal VM binaries on Windows and >report back the new size? >> Use: upx.exe -9 ... >> >> /Frank >> >> >> >> > >> > So my first question is - how can I compile Erlang into a single >file >> > executable? - I want no scripts/batch files to launch erlang no >DLLs >> > and a statically linked binary? >> >> Getting rid of scripts/batch files you can do if you generalize the >erlexec binary a bit (in Docker for instance, as it was mentioned >before, you perhaps want to be able to configure more things with >environment variables for instance) but what would "statically linked" >mean in the world of modules that load shared objects and even modules >that can be upgraded? To statically link one would have to sacrifice >the dynamic code reload, right? >> >> /Stefan >> >> > >> > /Joe >> > >> > >> > >> > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller >> > > >wrote: >> >> Do you think it's possible to package this minimal distribution in >one >> >> excutable? >> >> >> >> Imagine we can ship a whole release in one binary (like in Go). >> >> >> >> /Frank >> >> >> >> Wed 20 sept. 2017 at 16:23, Joe Armstrong > wrote : >> >>> >> >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files >> >>> Relocated as follows. I've also packed/compressed the libraries >> >>> >> >>> Here are the files that are needed >> >>> >> >>> %% DEST/bin/erl.exe 120320 bytes >> >>> %% DEST/bin/start.boot 5603 bytes >> >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >> >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >> >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >> >>> %% DEST/lib/compiler-7.1.ez 409091 bytes >> >>> %% DEST/lib/kernel-5.2.ez 384730 bytes >> >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >> >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes >> >>> >> >>> This seems to work :-) >> >>> >> >>> Cheers >> >>> >> >>> /Joe >> >>> >> >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >> >>> > wrote: >> >>>> Hello, >> >>>> >> >>>> On 20 Sep 2017, at 13.35, Stu Bailey > wrote: >> >>>> >> >>>> "As an afterthought - it would be *very nice* to have a program >which >> >>>> makes a minimal distributable program for Windows/Mac/Linux." >> >>>> >> >>>> >> >>>> This program is called relx. >> >>>> >> >>>> It generates a release package which are deployable to any >vanilla linux >> >>>> or >> >>>> other vanilla os. >> >>>> The usage of Docker-image-with-Erlang complicates the process of >> >>>> distribution. >> >>>> >> >>>> Best Regards, >> >>>> Dmitry >> >>>> >> >>>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Thu Sep 21 18:15:22 2017 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 21 Sep 2017 17:15:22 +0100 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: My opinion is that a full re-write of all VM's in Rust, especially the Erlang VM should be a very very high priority in the world of computing .. the benefit of memory safety in all languages, including low level, can not be underestimated in all of security, reliability and developer productivity. Over time, code that is not written in memory safe languages will be viewed in the same way as coding full of goto's or writing full apps in assembler. The cost of the rewrite in rust would be quite large, including: 1) A time consuming, complex process for a full rewrite 2) New bugs in the new implementation that will take a while to trace down ( tho the prevalence of large code bases written in Erlang with randomised property tests would help find these a lot faster tho ) 3) Incompatibilities that are likely inevitable 4) Wide performance differences : tho its lightly the rewrite could be gigantically faster ( judging by the performance of Hype alone ) there would be large scale Erlang systems that are tuned for the current VM .. some of these may expect very specific behaviour in scheduling, the GC, networking & more and would lightly need some heavy profiling and re-tuning to perform well. Also as the new VM would become popular the opposite would happen: some library's would run very fast on the new VM and slower on the old one 5) New users asking "which VM should I chose for my deployment" and the lightly very complex answers that come back causing some confusion. 6) Breaking NIF & C driver compatibility ( tho, it may be possible to provide a compatibility layer in some circumstances ) 7) Some tooling ( e.g. the debugger if its used by anyone etc.. ) no longer working 8) Platform portability being slightly different vs the current vm 9) A pause in curent VM development while the new one is built if its a full rebuild, tho an incremental rebuild or the development of a small mini proof of concept / embeddable vm would not have this issue The advantages going forward would be huge : 1) An ahead-of-time (AOT) optimising compiler targeting a high level machine code ( say LLVR IR ) could give a 10x performance boost, if not more. This would be far easer to write in rust 2) Hugely higher level of innovation ( once written & complete ) and future progress in the VM. Rust is so much more productive than C its crazy .. the borrow checker, tighter type system, no memory allocation errors, pattern matching, better macros, cleaner higher level syntax, inbuilt & safe mutex's, better inbuilt data structures and standard library, type classes, a package manager etc.. 3) Encouraging more open source contributions to the VM 4) Fantastic parallelism in the language + libraries too ( e.g. Rayon https://github.com/nikomatsakis/rayon ) 5) Far higher security in the VM .. this is not to say the current VM has any issues with security, but this is ensured by very high quality of coding and putting trust in that. Rust will eliminate the most common security issues found in code today. This extends into both future developments ( you can't guarantee secure code to day is secure after code changes ) and security of all included libaries. Tho Erlang was not effected by openSSL issues ( due to what parts it did / did not use ) it still included a code base with a gigantic security issue .. importing ( mostly ) native rust libraries and their updates would improve this greatly 6) Reducing the need for nifs : most NIF's ( tho not all ) are written for performance, with a much faster VM these may not be needed 7) Writing more of the VM / BIF's in Erlang, due to the higher performance 8) Secure, crash resistant NIFS written in Rust 9) Far smaller more modern implementation 10) Far more portable & embeddable implementation. It would be fantastic to be able to compile the VM and run it in the browser using WebAssembly, be able to embed Erlang into a desktop app, maybe simply to use its networking functionality, embed it into other libraries for other languages etc.. ( like Lua embedded for example ) 11) Making use of the Cargo packages and package manager for Rust. A huge benefit to rust is how good the package manager is and the amazing number of amazing packages. There are for example a huge range of lock free concurrent data structures right there in Cargo. These could be very useful in building the erlang VM 12) Far more modular implementation. Modern Rust applications are a collection of modest sized packages, built using Cargo rather than one gigantic code base. This allows far better code sharing between unrelated projects + often far better testing of each model & often more stable / better API designs per module. 13) A formally verified vm? 14) Setting a future direction where all low level code is written in secure, modern, memory safe languages Anyhow my 2 pence ! On 20 September 2017 at 16:31, Jeremy Pierre wrote: > Begging the list's indulgence with tangential and possibly off-topic > content. > > On Tue, Sep 19, 2017 at 7:57 AM Daniel Goertzen > wrote: > >> How does Alpaca compare to F#/Ocaml? After tasting some Rust and Elm, >> working in Erlang makes me a bit nervous too. My dream is to see a BEAM >> ML-like achieve Elixir stature. >> >> > In terms of syntax it's closest to OCaml but it's worth noting we still > lack some significant features one would be used to, e.g. ML's module > system (and functors, etc) and you'll see hints of Elm in some places like > our records. We kept things like adding fields to records, e.g. > > let record_example () = > let r1 = {x="hello", y="world"} in > {pi=3.14 | r1} > > is perfectly legal without declaring those record types in advance. > > r1: {x: string, y: string} > record_example: {pi: float, x: string, y: string} > > There's a basic tour of the language here if you're further curious: > https://github.com/alpaca-lang/alpaca/blob/master/Tour.md > > The pending v0.2.8 release (held up by me being super slow on a couple of > bugs I should have fixed months ago) adds some substantial stuff like type > annotations and better compiler feedback (both community contributions!) > > Jeremy > > >> >> On Tue, Sep 19, 2017 at 8:57 AM Karl Nilsson wrote: >> >>> On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < >>> jesper.louis.andersen@REDACTED> wrote: >>> >>>> On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe >>>> wrote: >>>> >>>>> What's your opinion of F#? It seems to be the language >>>>> of choice for people who liked Caml but need to take advantage >>>>> of multiple cores. I haven't done any benchmarking; I doubt >>>>> that it could match OCaml in raw speed. >>>>> >>>>> >>>> I have not used it enough to have an opinion (yet). Were I to >>>> communicate a lot with the .NET platform, I'd probably pick it because it >>>> has a null value and this is a necessity when talking to C# I'm told. >>>> >>>> Given that it runs under a pretty powerful JIT, it could perform really >>>> well for a lot of tasks I think. >>>> >>>> >>> F# doesn't typically have any speed advantages of any other .NET >>> languages and in every comparison I've seen to OCaml it has performed >>> worse. If anything the allocation costs induced by a functional first >>> programming style means it is typically a bit slower than the equivalent C# >>> code (also there is no goto). >>> >>> As a language F# is the nicest I've ever used substantially. I find it >>> easy (and fun) to write reasonably correct code in. Also I hardly ever fear >>> refactoring (compared to erlang where I break out in cold sweats even for >>> code bases that pass dialyzer). >>> >>> I even like it so much I've started hacking on an fsharp to core erlang >>> compiler. (https://github.com/kjnilsson/fez). >>> >>> >>> >>>> My "dream" would be an industry-supported parallel MLton :P >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 21 18:25:42 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 22 Sep 2017 01:25:42 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <1646853.Mco9nqNM1N@changa> On 2017?09?21? ??? 17:15:22 James Churchman wrote: > ... or writing full apps in assembler ... Have you any clue how many oversimplifications this implies? I get the point you think you are trying to make but srsly, there is this thing called a processor, and IT IS ITS OWN GODDAM LANGUAGE WTF. Trade one parade of blind-leading-the-blind for another. -Craig From felixgallo@REDACTED Thu Sep 21 18:42:06 2017 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 21 Sep 2017 09:42:06 -0700 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: People interested in Rust and Erlang should look into Pony ( https://www.ponylang.org), which is an LLVM-targeting, actor-model, memory-safe, race-free language with rich types and a small but growing userbase. On Thu, Sep 21, 2017 at 9:15 AM, James Churchman wrote: > My opinion is that a full re-write of all VM's in Rust, especially the > Erlang VM should be a very very high priority in the world of computing .. > the benefit of memory safety in all languages, including low level, can not > be underestimated in all of security, reliability and developer > productivity. Over time, code that is not written in memory safe languages > will be viewed in the same way as coding full of goto's or writing full > apps in assembler. > > The cost of the rewrite in rust would be quite large, including: > > 1) A time consuming, complex process for a full rewrite > > 2) New bugs in the new implementation that will take a while to trace down > ( tho the prevalence of large code bases written in Erlang with randomised > property tests would help find these a lot faster tho ) > > 3) Incompatibilities that are likely inevitable > > 4) Wide performance differences : tho its lightly the rewrite could be > gigantically faster ( judging by the performance of Hype alone ) there > would be large scale Erlang systems that are tuned for the current VM .. > some of these may expect very specific behaviour in scheduling, the GC, > networking & more and would lightly need some heavy profiling and re-tuning > to perform well. Also as the new VM would become popular the opposite would > happen: some library's would run very fast on the new VM and slower on the > old one > > 5) New users asking "which VM should I chose for my deployment" and the > lightly very complex answers that come back causing some confusion. > > 6) Breaking NIF & C driver compatibility ( tho, it may be possible to > provide a compatibility layer in some circumstances ) > > 7) Some tooling ( e.g. the debugger if its used by anyone etc.. ) no > longer working > > 8) Platform portability being slightly different vs the current vm > > 9) A pause in curent VM development while the new one is built if its a > full rebuild, tho an incremental rebuild or the development of a small mini > proof of concept / embeddable vm would not have this issue > > > The advantages going forward would be huge : > > 1) An ahead-of-time (AOT) optimising compiler targeting a high level > machine code ( say LLVR IR ) could give a 10x performance boost, if not > more. This would be far easer to write in rust > > 2) Hugely higher level of innovation ( once written & complete ) and > future progress in the VM. Rust is so much more productive than C its crazy > .. the borrow checker, tighter type system, no memory allocation errors, > pattern matching, better macros, cleaner higher level syntax, inbuilt & > safe mutex's, better inbuilt data structures and standard library, type > classes, a package manager etc.. > > 3) Encouraging more open source contributions to the VM > > 4) Fantastic parallelism in the language + libraries too ( e.g. Rayon > https://github.com/nikomatsakis/rayon ) > > 5) Far higher security in the VM .. this is not to say the current VM has > any issues with security, but this is ensured by very high quality of > coding and putting trust in that. Rust will eliminate the most common > security issues found in code today. This extends into both future > developments ( you can't guarantee secure code to day is secure after code > changes ) and security of all included libaries. Tho Erlang was not > effected by openSSL issues ( due to what parts it did / did not use ) it > still included a code base with a gigantic security issue .. importing ( > mostly ) native rust libraries and their updates would improve this greatly > > 6) Reducing the need for nifs : most NIF's ( tho not all ) are written for > performance, with a much faster VM these may not be needed > > 7) Writing more of the VM / BIF's in Erlang, due to the higher performance > > 8) Secure, crash resistant NIFS written in Rust > > 9) Far smaller more modern implementation > > 10) Far more portable & embeddable implementation. It would be fantastic > to be able to compile the VM and run it in the browser using WebAssembly, > be able to embed Erlang into a desktop app, maybe simply to use its > networking functionality, embed it into other libraries for other languages > etc.. ( like Lua embedded for example ) > > 11) Making use of the Cargo packages and package manager for Rust. A huge > benefit to rust is how good the package manager is and the amazing number > of amazing packages. There are for example a huge range of lock free > concurrent data structures right there in Cargo. These could be very useful > in building the erlang VM > > 12) Far more modular implementation. Modern Rust applications are a > collection of modest sized packages, built using Cargo rather than one > gigantic code base. This allows far better code sharing between unrelated > projects + often far better testing of each model & often more stable / > better API designs per module. > > 13) A formally verified vm? > > 14) Setting a future direction where all low level code is written in > secure, modern, memory safe languages > > > Anyhow my 2 pence ! > > > > > On 20 September 2017 at 16:31, Jeremy Pierre wrote: > >> Begging the list's indulgence with tangential and possibly off-topic >> content. >> >> On Tue, Sep 19, 2017 at 7:57 AM Daniel Goertzen < >> daniel.goertzen@REDACTED> wrote: >> >>> How does Alpaca compare to F#/Ocaml? After tasting some Rust and Elm, >>> working in Erlang makes me a bit nervous too. My dream is to see a BEAM >>> ML-like achieve Elixir stature. >>> >>> >> In terms of syntax it's closest to OCaml but it's worth noting we still >> lack some significant features one would be used to, e.g. ML's module >> system (and functors, etc) and you'll see hints of Elm in some places like >> our records. We kept things like adding fields to records, e.g. >> >> let record_example () = >> let r1 = {x="hello", y="world"} in >> {pi=3.14 | r1} >> >> is perfectly legal without declaring those record types in advance. >> >> r1: {x: string, y: string} >> record_example: {pi: float, x: string, y: string} >> >> There's a basic tour of the language here if you're further curious: >> https://github.com/alpaca-lang/alpaca/blob/master/Tour.md >> >> The pending v0.2.8 release (held up by me being super slow on a couple of >> bugs I should have fixed months ago) adds some substantial stuff like type >> annotations and better compiler feedback (both community contributions!) >> >> Jeremy >> >> >>> >>> On Tue, Sep 19, 2017 at 8:57 AM Karl Nilsson >>> wrote: >>> >>>> On Tue, 19 Sep 2017 at 12:41 Jesper Louis Andersen < >>>> jesper.louis.andersen@REDACTED> wrote: >>>> >>>>> On Wed, Sep 13, 2017 at 11:38 PM Richard A. O'Keefe >>>>> wrote: >>>>> >>>>>> What's your opinion of F#? It seems to be the language >>>>>> of choice for people who liked Caml but need to take advantage >>>>>> of multiple cores. I haven't done any benchmarking; I doubt >>>>>> that it could match OCaml in raw speed. >>>>>> >>>>>> >>>>> I have not used it enough to have an opinion (yet). Were I to >>>>> communicate a lot with the .NET platform, I'd probably pick it because it >>>>> has a null value and this is a necessity when talking to C# I'm told. >>>>> >>>>> Given that it runs under a pretty powerful JIT, it could perform >>>>> really well for a lot of tasks I think. >>>>> >>>>> >>>> F# doesn't typically have any speed advantages of any other .NET >>>> languages and in every comparison I've seen to OCaml it has performed >>>> worse. If anything the allocation costs induced by a functional first >>>> programming style means it is typically a bit slower than the equivalent C# >>>> code (also there is no goto). >>>> >>>> As a language F# is the nicest I've ever used substantially. I find it >>>> easy (and fun) to write reasonably correct code in. Also I hardly ever fear >>>> refactoring (compared to erlang where I break out in cold sweats even for >>>> code bases that pass dialyzer). >>>> >>>> I even like it so much I've started hacking on an fsharp to core erlang >>>> compiler. (https://github.com/kjnilsson/fez). >>>> >>>> >>>> >>>>> My "dream" would be an industry-supported parallel MLton :P >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Sep 21 18:45:01 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 21 Sep 2017 18:45:01 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On 09/21/2017 06:15 PM, James Churchman wrote: > My opinion is that a full re-write of all VM's in Rust, especially the > Erlang VM should be a very very high priority in the world of computing There's a reason Netscape died... Perhaps Rust can and should be used for parts of the VM, but a full rewrite would be suicide, especially considering BEAM development does not have gigantic resources to begin with. Even if you rewrite only parts of the VM, while you're rewriting you're not improving the features stack. -- Lo?c Hoguin https://ninenines.eu From frank.muller.erl@REDACTED Thu Sep 21 19:12:14 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 21 Sep 2017 17:12:14 +0000 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: Erlang newbie here ... jumping into the subject. While agree with most of the ideas about security, speed ... I still can't get some really basic things. 1. Why one should trade a time-proven, close to metal, fars language like C with more than ~40yrs of existence with a new one? We don't even know if Rust will exist in the near future. That's not gonna be the case for C apparently (IoT, etc.). 2. Why simply not simply learn how to better code our NIF/Drivers instead? C was/is my main programming language for many years now, and I didn't have any major issue with it (medium to large projects) in production environment so far. Maybe I'm just lucky, maybe not. Thanks for your comments and feedbacks. /Frank On 09/21/2017 06:15 PM, James Churchman wrote: > > My opinion is that a full re-write of all VM's in Rust, especially the > > Erlang VM should be a very very high priority in the world of computing > > There's a reason Netscape died... > > Perhaps Rust can and should be used for parts of the VM, but a full > rewrite would be suicide, especially considering BEAM development does > not have gigantic resources to begin with. > > Even if you rewrite only parts of the VM, while you're rewriting you're > not improving the features stack. > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Sep 21 20:26:32 2017 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 21 Sep 2017 11:26:32 -0700 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: I think Rust takes several steps in wrong directions, but the answer to (2) is obvious -- even though we've had 40 years to learn how to 'better code our drivers', the world of software is a shaky, broken, rickety pile of insecure nonsense and it's only getting worse over time. There is apparently no amount of learning we can do and we need the machines to help us. Erlang solves the memory safety problem by enforcing immutability, which has incredibly low mechanical sympathy and ends up being unperformant for a large and useful set of problems. Rust solves it by giving the developer a bewilderingly bedazzled straitjacket and telling them to sort it out if they want performance. Pony's straitjacket has better affordances in my opinion but is still deeply confusing to developers. The fact that we are all trying is no accident. F. F. On Thu, Sep 21, 2017 at 10:12 AM, Frank Muller wrote: > Erlang newbie here ... jumping into the subject. > > While agree with most of the ideas about security, speed ... I still can't > get some really basic things. > > 1. Why one should trade a time-proven, close to metal, fars language like > C with more than ~40yrs of existence with a new one? We don't even know if > Rust will exist in the near future. That's not gonna be the case for C > apparently (IoT, etc.). > > 2. Why simply not simply learn how to better code our NIF/Drivers instead? > C was/is my main programming language for many years now, and I didn't have > any major issue with it (medium to large projects) in production > environment so far. Maybe I'm just lucky, maybe not. > > Thanks for your comments and feedbacks. > > /Frank > > On 09/21/2017 06:15 PM, James Churchman wrote: >> > My opinion is that a full re-write of all VM's in Rust, especially the >> > Erlang VM should be a very very high priority in the world of computing >> >> There's a reason Netscape died... >> >> Perhaps Rust can and should be used for parts of the VM, but a full >> rewrite would be suicide, especially considering BEAM development does >> not have gigantic resources to begin with. >> >> Even if you rewrite only parts of the VM, while you're rewriting you're >> not improving the features stack. >> >> -- >> Lo?c Hoguin >> https://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 21 21:27:09 2017 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 21 Sep 2017 21:27:09 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: <1F720108-859A-42BE-A652-F893905AD41B@stritzinger.com> References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> <1F720108-859A-42BE-A652-F893905AD41B@stritzinger.com> Message-ID: On Thu, Sep 21, 2017 at 5:45 PM, Peer Stritzinger wrote: > > On 21.09.2017, at 13:15, Frank Muller wrote: > > Stefan, > > By statically linking bins, Joe is talking about the executables here > (erl.exe, ...), not the beam files IMHO. > > > But what you can do is preload all needed beam files, which means they get > baked into your erts executable. > > So you can even have statically linked beam files. Every Erlang system has > some of those since quite a lot > of the system startup is done in Erlang already. And to get to the point > where the runtime can load beam files > you need some preexisting beam files. > > When building this is done by this perl script and its surrounding Makefile > magic: > > erts/emulator/utils/make_preload > > Which generates a C file called preload.c starting like: > > /* > * DO NOT EDIT THIS FILE. It was automatically generated > * by the `make_preload' program on Fri Jun 23 12:55:01 2017. > */ > unsigned preloaded_size_otp_ring0 = 532; > unsigned char preloaded_otp_ring0[] = { > 0x46,0x4f,0x52,0x31,0x00,0x00,0x02,0x0c, /* FOR1.... */ > 0x42,0x45,0x41,0x4d,0x41,0x74,0x6f,0x6d, /* BEAMAtom */ > 0x00,0x00,0x00,0x82,0x00,0x00,0x00,0x10, /* ........ */ > 0x09,0x6f,0x74,0x70,0x5f,0x72,0x69,0x6e, /* .otp_rin */ > 0x67,0x30,0x05,0x73,0x74,0x61,0x72,0x74, /* g0.start */ > 0x04,0x62,0x6f,0x6f,0x74,0x04,0x69,0x6e, /* .boot.in */ > 0x69,0x74,0x03,0x72,0x75,0x6e,0x06,0x65, /* it.run.e */ > 0x72,0x6c,0x61,0x6e,0x67,0x11,0x66,0x75, /* rlang.fu */ > 0x6e,0x63,0x74,0x69,0x6f,0x6e,0x5f,0x65, /* nction_e */ > 0x78,0x70,0x6f,0x72,0x74,0x65,0x64,0x05, /* xported. */ > 0x66,0x61,0x6c,0x73,0x65,0x04,0x74,0x72, /* false.tr */ > 0x75,0x65,0x05,0x66,0x61,0x74,0x61,0x6c, /* ue.fatal */ > 0x05,0x65,0x72,0x72,0x6f,0x72,0x06,0x6d, /* .error.m */ > 0x6f,0x64,0x75,0x6c,0x65,0x07,0x64,0x69, /* odule.di */ > > And I suppose that you still can hot code load preloaded files, its just > that their memory isn?t freed > since it is in the erts executable. > > Best, > -- Peer Yup - the last time I did this I changed the preloaded erlang files to read the data that I had appended to the stub ... /Joe > > > You'll be able to reload your beam modules on the fly, obviously. > > Moreover, nothing prevent a statically built executable to load a shared lib > while running. > > @Joe: just made a quick search and found UPX binary packer. It's > cross-platform: > https://upx.github.io/ > > Can you please try to shrink the minimal VM binaries on Windows and report > back the new size? > Use: upx.exe -9 ... > > /Frank > > >> >> > >> > So my first question is - how can I compile Erlang into a single file >> > executable? - I want no scripts/batch files to launch erlang no DLLs >> > and a statically linked binary? >> >> Getting rid of scripts/batch files you can do if you generalize the >> erlexec binary a bit (in Docker for instance, as it was mentioned before, >> you perhaps want to be able to configure more things with environment >> variables for instance) but what would "statically linked" mean in the world >> of modules that load shared objects and even modules that can be upgraded? >> To statically link one would have to sacrifice the dynamic code reload, >> right? >> >> /Stefan >> >> > >> > /Joe >> > >> > >> > >> > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller >> > wrote: >> >> Do you think it's possible to package this minimal distribution in one >> >> excutable? >> >> >> >> Imagine we can ship a whole release in one binary (like in Go). >> >> >> >> /Frank >> >> >> >> Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : >> >>> >> >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files >> >>> Relocated as follows. I've also packed/compressed the libraries >> >>> >> >>> Here are the files that are needed >> >>> >> >>> %% DEST/bin/erl.exe 120320 bytes >> >>> %% DEST/bin/start.boot 5603 bytes >> >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >> >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >> >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >> >>> %% DEST/lib/compiler-7.1.ez 409091 bytes >> >>> %% DEST/lib/kernel-5.2.ez 384730 bytes >> >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >> >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes >> >>> >> >>> This seems to work :-) >> >>> >> >>> Cheers >> >>> >> >>> /Joe >> >>> >> >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >> >>> wrote: >> >>>> Hello, >> >>>> >> >>>> On 20 Sep 2017, at 13.35, Stu Bailey wrote: >> >>>> >> >>>> "As an afterthought - it would be *very nice* to have a program which >> >>>> makes a minimal distributable program for Windows/Mac/Linux." >> >>>> >> >>>> >> >>>> This program is called relx. >> >>>> >> >>>> It generates a release package which are deployable to any vanilla >> >>>> linux >> >>>> or >> >>>> other vanilla os. >> >>>> The usage of Docker-image-with-Erlang complicates the process of >> >>>> distribution. >> >>>> >> >>>> Best Regards, >> >>>> Dmitry >> >>>> >> >>>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From peer@REDACTED Thu Sep 21 21:30:00 2017 From: peer@REDACTED (Peer Stritzinger) Date: Thu, 21 Sep 2017 21:30:00 +0200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1B1D5F1D-6D25-4751-ACA0-075E513A4146@stritzinger.com> Message-ID: <19A7D75B-8CCD-48C6-985F-8D3ECD49F109@stritzinger.com> Hi Steve, > On 21.09.2017, at 21:16, Steve Rogers wrote: > > So when can we expect to get our hands on a GRiSP board/ This week we finalized the hardware for production. From now there is a minimum of 8 weeks production time if there is delivery time on crucial parts possibly a bit longer, waiting for the schedule from the manufacturer. So hoping to ship mid November. Definitely in time for Christmas season ;-) There is a newsletter subscription on the bottom of https://www.grisp.org will post details of producing the first batch there. Best, -- Peer > # Steve > > On Thu, Sep 21, 2017 at 10:29 AM Peer Stritzinger > wrote: >> On 21.09.2017, at 12:23, Jesper Louis Andersen > wrote: >> >> On Wed, Sep 20, 2017 at 12:50 AM Richard A. O'Keefe > wrote: >> One question. I can get a >> "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" >> for NZD 69 including tax and run Erlang on it. >> (Hmm. Next time I have some spare cash I must get one...) >> >> Why would I pay NZD300 for a 300MHz board with 64MB of memory? >> >> >> My prediction is that eventually we'll see the "small embedded" market squeezed down by rather cheap powerful devices such as the rPI. My argument for this prediction hinges on power becoming free in the future (due to the current massive advances in solar and wind energy on top of better batteries). > > Small embedded being current rPI *sized* CPUs for some definition of small will be happening. Except rPI is not usable in production for several reasons (supply, quality, hardware changing beyond your control). > >> Currently, the power draw of the stronger chips is too high and production at scale means that shaving off a few cents here and there is worth it. > > Exactly. We have a few industrial applications where we would love to get a larger CPU but we can?t because of price and power (meaning in this case thermal energy output). > >> But it looks like the price tag of a 1.2 GHZ 64 bit chip is only going one way, and this makes the added development effort of targeting a smaller chip more expensive. In turn, you need an even bigger scale of production. And so on. > > Except that with a 300MHz Cortex M7 size CPU you can do lots of interesting applications just in Erlang already and save the costs. > >> GRiSP is interesting because it can offer you a far faster time to market and lower development effort at the same or better robustness/resilience. You can thus get the lower production costs while amortizing development effort better. It is a good bet until the niche squeeze I predict above happens. > > Thanks :-) > > Also we plan not to be squeezed but adapt. One size which is just getting viable with Erlang are the automotive controllers with flash and ram on the CPU only (for the automotive industry any external memory is too unreliable and a problem for EMC). Thats one direction what I want to target with the GRiSP software stack. > > We could have easily also targeted a 24 core communication controller with 16 times Gbit I/O which is a tempting target also and noone would complain about the compute power there ;-) > > For the GRiSP-base we just tried to pick the right sized board for today. Others might follow? > > Best, > -- Peer > > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Thu Sep 21 21:42:11 2017 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 21 Sep 2017 20:42:11 +0100 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On 21 September 2017 at 19:26, Felix Gallo wrote: > I think Rust takes several steps in wrong directions, but the answer to > (2) is obvious -- even though we've had 40 years to learn how to 'better > code our drivers', the world of software is a shaky, broken, rickety pile > of insecure nonsense and it's only getting worse over time. There is > apparently no amount of learning we can do and we need the machines to help > us. > > Erlang solves the memory safety problem by enforcing immutability, which > has incredibly low mechanical sympathy and ends up being unperformant for a > large and useful set of problems. Rust solves it by giving the developer a > bewilderingly bedazzled straitjacket and telling them to sort it out if > they want performance. Pony's straitjacket has better affordances in my > opinion but is still deeply confusing to developers. The fact that we are > all trying is no accident. > Indeed... there are some algorithms that are orders of magnitude slower to write with immutability. The systems that Erlang is designed for quite often are not these tho, + NIF's can fill the gap, tho not in an elegant way ( embedding a totally different language that forces you to give up all guarantees that Erlang has, tho rust would help here as it should not crash the VM ) You should try the borrow checker in rust.. it takes time to get used to and there are few times you have rethink a way of coding something but it gives memory safety with no GC .. really amazing .. on top of that you can write "unsafe" rust with less guarantees, and do as you feel .. no restrictions at all. Its also possible to write GC code too, have yet to try it but was originally optional in the language & all the hooks left in the language in the type system, so a few are available as installable as packages ... some algorithms ( maybe writing a graph database or similar ? ) are easier with a GC so you just create those objects as being handled by the GC .. Its also worth remembering that the entire Erlang runtime has already been re-written, in Java, with performance between beam and hype, able to run apps like riak and the only downside being some small gc pauses, that may not even happen on a modern JVM > F. > > F. > On 21 September 2017 at 18:12, Frank Muller > wrote: > >> Erlang newbie here ... jumping into the subject. >> >> While agree with most of the ideas about security, speed ... I still >> can't get some really basic things. >> >> 1. Why one should trade a time-proven, close to metal, fars language like >> C with more than ~40yrs of existence with a new one? We don't even know if >> Rust will exist in the near future. That's not gonna be the case for C >> apparently (IoT, etc.). >> > Well the existence of Rust ( or any language ) will depend entirely on how many new projects and existing ( c ) projects written in it! Large sections of Firefox are now written in Rust ( in a version thats shipping very soon ) cross ported from the Servo project, which gave rise to rust in order to build a totally new browser engine .. given that browsers are now some of the largest software projects on the planet this is a good sign. These include the CSS parser, CSS matcher, the Compositor and several more .. the eventual plan is to move everything over. The benefits are many, and it still maintains C ABI compatibility if you need it. > > >> 2. Why simply not simply learn how to better code our NIF/Drivers >> instead? C was/is my main programming language for many years now, and I >> didn't have any major issue with it (medium to large projects) in >> production environment so far. Maybe I'm just lucky, maybe not. >> > > Well the more the tooling and language can do for you the better, and the more guarantees of correctness the more secure your software is likely to be. One of many reasons for rewriting Firefox in rust is security. Most C projects probably don't have hundreds/thousands of security guys trying to cause memory overflow errors, but for things like browsers, VM's, OS's they do, and with the increase of IOT many products that were not traditionally exposed to the internet now are .. and when one is discovered it's usually game over! -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 21 22:08:03 2017 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 21 Sep 2017 22:08:03 +0200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: <91A88F78-E681-4831-B970-CB0BAD148687@gmail.com> Message-ID: On Thu, Sep 21, 2017 at 1:22 PM, Jesper Louis Andersen wrote: > I think one of the reasons for the directory structure has to do with > hot-code-upgrades. It is way easier if your system is able to load stuff > individually from files rather than having to work with static binaries. > > Static binaries would be nice. But a full implementation must also be able > to load patches into the binary later on, if you decide to upgrade the > running code. This may not be that simple to pull off, though ELF can do > tricks. > > > On Thu, Sep 21, 2017 at 1:15 PM Frank Muller > wrote: >> >> Stefan, >> >> By statically linking bins, Joe is talking about the executables here >> (erl.exe, ...), not the beam files IMHO. >> >> You'll be able to reload your beam modules on the fly, obviously. >> >> Moreover, nothing prevent a statically built executable to load a shared >> lib while running. Problem is that I want to distribute only one file. As it is now we have to distribute a shell script erlexec and a DLL that gets (somehow) loaded when erlexec is run. Is there some way to statically link the code that erlexec causes to be dynamically loaded? /Joe >> >> @Joe: just made a quick search and found UPX binary packer. It's >> cross-platform: >> https://upx.github.io/ >> >> Can you please try to shrink the minimal VM binaries on Windows and report >> back the new size? >> Use: upx.exe -9 ... >> >> /Frank >> >> >>> >>> > >>> > So my first question is - how can I compile Erlang into a single file >>> > executable? - I want no scripts/batch files to launch erlang no DLLs >>> > and a statically linked binary? >>> >>> Getting rid of scripts/batch files you can do if you generalize the >>> erlexec binary a bit (in Docker for instance, as it was mentioned before, >>> you perhaps want to be able to configure more things with environment >>> variables for instance) but what would "statically linked" mean in the world >>> of modules that load shared objects and even modules that can be upgraded? >>> To statically link one would have to sacrifice the dynamic code reload, >>> right? >>> >>> /Stefan >>> >>> > >>> > /Joe >>> > >>> > >>> > >>> > On Wed, Sep 20, 2017 at 4:34 PM, Frank Muller >>> > wrote: >>> >> Do you think it's possible to package this minimal distribution in one >>> >> excutable? >>> >> >>> >> Imagine we can ship a whole release in one binary (like in Go). >>> >> >>> >> /Frank >>> >> >>> >> Wed 20 sept. 2017 at 16:23, Joe Armstrong wrote : >>> >>> >>> >>> Thanks everybody for you help - I got it down to 5.5MB in 9 files >>> >>> Relocated as follows. I've also packed/compressed the libraries >>> >>> >>> >>> Here are the files that are needed >>> >>> >>> >>> %% DEST/bin/erl.exe 120320 bytes >>> >>> %% DEST/bin/start.boot 5603 bytes >>> >>> %% DEST/erts-9.0/bin/beam.smp.dll 3346944 bytes >>> >>> %% DEST/erts-9.0/bin/erlexec.dll 162304 bytes >>> >>> %% DEST/erts-9.0/bin/inet_gethost.exec 45568 bytes >>> >>> %% DEST/lib/compiler-7.1.ez 409091 bytes >>> >>> %% DEST/lib/kernel-5.2.ez 384730 bytes >>> >>> %% DEST/lib/sasl-3.0.4.ez 106278 bytes >>> >>> %% DEST/lib/stdlib-3.4.ez 913013 bytes >>> >>> >>> >>> This seems to work :-) >>> >>> >>> >>> Cheers >>> >>> >>> >>> /Joe >>> >>> >>> >>> On Wed, Sep 20, 2017 at 2:24 PM, Dmitry Kolesnikov >>> >>> wrote: >>> >>>> Hello, >>> >>>> >>> >>>> On 20 Sep 2017, at 13.35, Stu Bailey wrote: >>> >>>> >>> >>>> "As an afterthought - it would be *very nice* to have a program >>> >>>> which >>> >>>> makes a minimal distributable program for Windows/Mac/Linux." >>> >>>> >>> >>>> >>> >>>> This program is called relx. >>> >>>> >>> >>>> It generates a release package which are deployable to any vanilla >>> >>>> linux >>> >>>> or >>> >>>> other vanilla os. >>> >>>> The usage of Docker-image-with-Erlang complicates the process of >>> >>>> distribution. >>> >>>> >>> >>>> Best Regards, >>> >>>> Dmitry >>> >>>> >>> >>>> >>> >>> _______________________________________________ >>> >>> erlang-questions mailing list >>> >>> erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Thu Sep 21 22:37:17 2017 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 21 Sep 2017 22:37:17 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: The project that interest me would go in the opposite direction to reimplementing the VM in Rust. I'd like to make an extremely small extremely slow Ertang targeted to IOT devices - low power devices with small memory and slow clocks. The goal would be a smaller VM with fewer instructions and a highly portable ANSI C interpreter. I think the only way to make secure systems is to throw away as much as possible and retain a small kernel with very limited ability. I'd like to see lots of very small machines talking to each other with defined protocols. I managed to find an early erlang from 1991 (the compiler still works) the compiler in was 4000 line of Ertang and the emulator was 3800 lines of C We didn't have binarys and maps but I think the *goodness* comes from links and mailboxes. Software gets more complex with time *because* we build upon earlier work *without* throwing away stuff. The problem is we when we build on old stuff we don't know which of the old stuff we can throw away, so we include it "just in case" Then it gets so complex we give up - and we seal it off in a virtual machine or container and carry on. N. Wirth said when you add a new feature you should remove an old one but we don't do this. A new VM in Rust or anything would be a good excuse to re-look at the architectures we want and see how to implement them - Just reimplementing Erlang would be miss a good opportunity to chuck out some weird design decisions and do some good things. Pids should be planetary wide references with a DHT to find them - processes should be first class and movable - Protocols should be first class ... My 10 cents worth /Joe On Thu, Sep 21, 2017 at 9:42 PM, James Churchman wrote: > > > On 21 September 2017 at 19:26, Felix Gallo wrote: >> >> I think Rust takes several steps in wrong directions, but the answer to >> (2) is obvious -- even though we've had 40 years to learn how to 'better >> code our drivers', the world of software is a shaky, broken, rickety pile of >> insecure nonsense and it's only getting worse over time. There is >> apparently no amount of learning we can do and we need the machines to help >> us. >> >> Erlang solves the memory safety problem by enforcing immutability, which >> has incredibly low mechanical sympathy and ends up being unperformant for a >> large and useful set of problems. Rust solves it by giving the developer a >> bewilderingly bedazzled straitjacket and telling them to sort it out if they >> want performance. Pony's straitjacket has better affordances in my opinion >> but is still deeply confusing to developers. The fact that we are all >> trying is no accident. > > > Indeed... there are some algorithms that are orders of magnitude slower to > write with immutability. The systems that Erlang is designed for quite often > are not these tho, + NIF's can fill the gap, tho not in an elegant way ( > embedding a totally different language that forces you to give up all > guarantees that Erlang has, tho rust would help here as it should not crash > the VM ) > > You should try the borrow checker in rust.. it takes time to get used to and > there are few times you have rethink a way of coding something but it gives > memory safety with no GC .. really amazing .. on top of that you can write > "unsafe" rust with less guarantees, and do as you feel .. no restrictions at > all. Its also possible to write GC code too, have yet to try it but was > originally optional in the language & all the hooks left in the language in > the type system, so a few are available as installable as packages ... some > algorithms ( maybe writing a graph database or similar ? ) are easier with a > GC so you just create those objects as being handled by the GC .. > > > Its also worth remembering that the entire Erlang runtime has already been > re-written, in Java, with performance between beam and hype, able to run > apps like riak and the only downside being some small gc pauses, that may > not even happen on a modern JVM > >> >> F. >> >> F. >> On 21 September 2017 at 18:12, Frank Muller >> wrote: >>> >>> Erlang newbie here ... jumping into the subject. >>> >>> While agree with most of the ideas about security, speed ... I still >>> can't get some really basic things. >>> >>> 1. Why one should trade a time-proven, close to metal, fars language like >>> C with more than ~40yrs of existence with a new one? We don't even know if >>> Rust will exist in the near future. That's not gonna be the case for C >>> apparently (IoT, etc.). > > > Well the existence of Rust ( or any language ) will depend entirely on how > many new projects and existing ( c ) projects written in it! > Large sections of Firefox are now written in Rust ( in a version thats > shipping very soon ) cross ported from the Servo project, which gave rise to > rust in order to build a totally new browser engine .. given that browsers > are now some of the largest software projects on the planet this is a good > sign. These include the CSS parser, CSS matcher, the Compositor and several > more .. the eventual plan is to move everything over. > The benefits are many, and it still maintains C ABI compatibility if you > need it. > > >> >> >>> >>> 2. Why simply not simply learn how to better code our NIF/Drivers >>> instead? C was/is my main programming language for many years now, and I >>> didn't have any major issue with it (medium to large projects) in production >>> environment so far. Maybe I'm just lucky, maybe not. >> >> > > Well the more the tooling and language can do for you the better, and the > more guarantees of correctness the more secure your software is likely to > be. One of many reasons for rewriting Firefox in rust is security. Most C > projects probably don't have hundreds/thousands of security guys trying to > cause memory overflow errors, but for things like browsers, VM's, OS's they > do, and with the increase of IOT many products that were not traditionally > exposed to the internet now are .. and when one is discovered it's usually > game over! > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jameschurchman@REDACTED Thu Sep 21 22:48:37 2017 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 21 Sep 2017 21:48:37 +0100 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On 21 September 2017 at 21:37, Joe Armstrong wrote: > The project that interest me would go in the opposite direction to > reimplementing the VM in Rust. > > I'd like to make an extremely small extremely slow Ertang targeted to > IOT devices - low power devices with small memory and slow clocks. > > The goal would be a smaller VM with fewer instructions > and a highly portable ANSI C interpreter. > > I think the only way to make secure systems is to throw away as much as > possible and retain a small kernel with very limited ability. > > I'd like to see lots of very small machines talking to each other with > defined protocols. > > I managed to find an early erlang from 1991 (the compiler still works) > the compiler in was 4000 line of Ertang and the emulator was 3800 lines of > C > > We didn't have binarys and maps but I think the *goodness* comes from > links and mailboxes. > > Software gets more complex with time *because* we build upon earlier work > *without* throwing away stuff. > > The problem is we when we build on old stuff we don't know which of the old > stuff we can throw away, so we include it "just in case" > > Then it gets so complex we give up - and we seal it off in a virtual > machine > or container and carry on. > > N. Wirth said when you add a new feature you should remove an old one > but we don't do this. > > A new VM in Rust or anything would be a good excuse to re-look at the > architectures we want and see how to implement them - Just reimplementing > Erlang would be miss a good opportunity to chuck out some weird design > decisions and do some good things. > Sounds fantastic and with a truly ideal implementation the same small implementation would also be performant at large scale as well > > Pids should be planetary wide references with a DHT to find them - > processes > should be first class and movable - Protocols should be first class ... > Would be amazing tho pids would become very complex with the lightly security needs added on top, becoming language agnostic, ( message reliability ) etc.. Ps what does protocols being first class mean btw? > > My 10 cents worth > > > /Joe > > > > On Thu, Sep 21, 2017 at 9:42 PM, James Churchman > wrote: > > > > > > On 21 September 2017 at 19:26, Felix Gallo wrote: > >> > >> I think Rust takes several steps in wrong directions, but the answer to > >> (2) is obvious -- even though we've had 40 years to learn how to 'better > >> code our drivers', the world of software is a shaky, broken, rickety > pile of > >> insecure nonsense and it's only getting worse over time. There is > >> apparently no amount of learning we can do and we need the machines to > help > >> us. > >> > >> Erlang solves the memory safety problem by enforcing immutability, which > >> has incredibly low mechanical sympathy and ends up being unperformant > for a > >> large and useful set of problems. Rust solves it by giving the > developer a > >> bewilderingly bedazzled straitjacket and telling them to sort it out if > they > >> want performance. Pony's straitjacket has better affordances in my > opinion > >> but is still deeply confusing to developers. The fact that we are all > >> trying is no accident. > > > > > > Indeed... there are some algorithms that are orders of magnitude slower > to > > write with immutability. The systems that Erlang is designed for quite > often > > are not these tho, + NIF's can fill the gap, tho not in an elegant way ( > > embedding a totally different language that forces you to give up all > > guarantees that Erlang has, tho rust would help here as it should not > crash > > the VM ) > > > > You should try the borrow checker in rust.. it takes time to get used to > and > > there are few times you have rethink a way of coding something but it > gives > > memory safety with no GC .. really amazing .. on top of that you can > write > > "unsafe" rust with less guarantees, and do as you feel .. no > restrictions at > > all. Its also possible to write GC code too, have yet to try it but was > > originally optional in the language & all the hooks left in the language > in > > the type system, so a few are available as installable as packages ... > some > > algorithms ( maybe writing a graph database or similar ? ) are easier > with a > > GC so you just create those objects as being handled by the GC .. > > > > > > Its also worth remembering that the entire Erlang runtime has already > been > > re-written, in Java, with performance between beam and hype, able to run > > apps like riak and the only downside being some small gc pauses, that may > > not even happen on a modern JVM > > > >> > >> F. > >> > >> F. > >> On 21 September 2017 at 18:12, Frank Muller > > >> wrote: > >>> > >>> Erlang newbie here ... jumping into the subject. > >>> > >>> While agree with most of the ideas about security, speed ... I still > >>> can't get some really basic things. > >>> > >>> 1. Why one should trade a time-proven, close to metal, fars language > like > >>> C with more than ~40yrs of existence with a new one? We don't even > know if > >>> Rust will exist in the near future. That's not gonna be the case for C > >>> apparently (IoT, etc.). > > > > > > Well the existence of Rust ( or any language ) will depend entirely on > how > > many new projects and existing ( c ) projects written in it! > > Large sections of Firefox are now written in Rust ( in a version thats > > shipping very soon ) cross ported from the Servo project, which gave > rise to > > rust in order to build a totally new browser engine .. given that > browsers > > are now some of the largest software projects on the planet this is a > good > > sign. These include the CSS parser, CSS matcher, the Compositor and > several > > more .. the eventual plan is to move everything over. > > The benefits are many, and it still maintains C ABI compatibility if you > > need it. > > > > > >> > >> > >>> > >>> 2. Why simply not simply learn how to better code our NIF/Drivers > >>> instead? C was/is my main programming language for many years now, and > I > >>> didn't have any major issue with it (medium to large projects) in > production > >>> environment so far. Maybe I'm just lucky, maybe not. > >> > >> > > > > Well the more the tooling and language can do for you the better, and the > > more guarantees of correctness the more secure your software is likely to > > be. One of many reasons for rewriting Firefox in rust is security. Most C > > projects probably don't have hundreds/thousands of security guys trying > to > > cause memory overflow errors, but for things like browsers, VM's, OS's > they > > do, and with the increase of IOT many products that were not > traditionally > > exposed to the internet now are .. and when one is discovered it's > usually > > game over! > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 21 22:56:43 2017 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 21 Sep 2017 22:56:43 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: On Thu, Sep 21, 2017 at 10:48 PM, James Churchman wrote: > > > On 21 September 2017 at 21:37, Joe Armstrong wrote: >> >> The project that interest me would go in the opposite direction to >> reimplementing the VM in Rust. >> >> I'd like to make an extremely small extremely slow Ertang targeted to >> IOT devices - low power devices with small memory and slow clocks. >> >> The goal would be a smaller VM with fewer instructions >> and a highly portable ANSI C interpreter. >> >> I think the only way to make secure systems is to throw away as much as >> possible and retain a small kernel with very limited ability. >> >> I'd like to see lots of very small machines talking to each other with >> defined protocols. >> >> I managed to find an early erlang from 1991 (the compiler still works) >> the compiler in was 4000 line of Ertang and the emulator was 3800 lines of >> C >> >> We didn't have binarys and maps but I think the *goodness* comes from >> links and mailboxes. >> >> Software gets more complex with time *because* we build upon earlier work >> *without* throwing away stuff. >> >> The problem is we when we build on old stuff we don't know which of the >> old >> stuff we can throw away, so we include it "just in case" >> >> Then it gets so complex we give up - and we seal it off in a virtual >> machine >> or container and carry on. >> >> N. Wirth said when you add a new feature you should remove an old one >> but we don't do this. >> >> A new VM in Rust or anything would be a good excuse to re-look at the >> architectures we want and see how to implement them - Just reimplementing >> Erlang would be miss a good opportunity to chuck out some weird design >> decisions and do some good things. > > Sounds fantastic and with a truly ideal implementation the same small > implementation would also be performant at large scale as well >> >> >> Pids should be planetary wide references with a DHT to find them - >> processes >> should be first class and movable - Protocols should be first class ... > > Would be amazing tho pids would become very complex with the lightly > security needs added on top, becoming language agnostic, ( message > reliability ) etc.. > Ps what does protocols being first class mean btw? Protocols are implicit and you can imply them by reading the code. I'd like a protocol declaration (like a module definition) that *defines* the sequence of allowed operations. Some people call these session types :-) -protocol(file_server) -agents(client, server) client ! server {send_file,F} -> server ! client ({ok,Bin} |enofile) ... -end(protocol). Then at run time you could query the protocol -process(c1) -implements(file_server, client) ... ... /Joe >> >> >> My 10 cents worth >> > > > >> >> /Joe >> >> >> >> On Thu, Sep 21, 2017 at 9:42 PM, James Churchman >> wrote: >> > >> > >> > On 21 September 2017 at 19:26, Felix Gallo wrote: >> >> >> >> I think Rust takes several steps in wrong directions, but the answer to >> >> (2) is obvious -- even though we've had 40 years to learn how to >> >> 'better >> >> code our drivers', the world of software is a shaky, broken, rickety >> >> pile of >> >> insecure nonsense and it's only getting worse over time. There is >> >> apparently no amount of learning we can do and we need the machines to >> >> help >> >> us. >> >> >> >> Erlang solves the memory safety problem by enforcing immutability, >> >> which >> >> has incredibly low mechanical sympathy and ends up being unperformant >> >> for a >> >> large and useful set of problems. Rust solves it by giving the >> >> developer a >> >> bewilderingly bedazzled straitjacket and telling them to sort it out if >> >> they >> >> want performance. Pony's straitjacket has better affordances in my >> >> opinion >> >> but is still deeply confusing to developers. The fact that we are all >> >> trying is no accident. >> > >> > >> > Indeed... there are some algorithms that are orders of magnitude slower >> > to >> > write with immutability. The systems that Erlang is designed for quite >> > often >> > are not these tho, + NIF's can fill the gap, tho not in an elegant way ( >> > embedding a totally different language that forces you to give up all >> > guarantees that Erlang has, tho rust would help here as it should not >> > crash >> > the VM ) >> > >> > You should try the borrow checker in rust.. it takes time to get used to >> > and >> > there are few times you have rethink a way of coding something but it >> > gives >> > memory safety with no GC .. really amazing .. on top of that you can >> > write >> > "unsafe" rust with less guarantees, and do as you feel .. no >> > restrictions at >> > all. Its also possible to write GC code too, have yet to try it but was >> > originally optional in the language & all the hooks left in the language >> > in >> > the type system, so a few are available as installable as packages ... >> > some >> > algorithms ( maybe writing a graph database or similar ? ) are easier >> > with a >> > GC so you just create those objects as being handled by the GC .. >> > >> > >> > Its also worth remembering that the entire Erlang runtime has already >> > been >> > re-written, in Java, with performance between beam and hype, able to run >> > apps like riak and the only downside being some small gc pauses, that >> > may >> > not even happen on a modern JVM >> > >> >> >> >> F. >> >> >> >> F. >> >> On 21 September 2017 at 18:12, Frank Muller >> >> >> >> wrote: >> >>> >> >>> Erlang newbie here ... jumping into the subject. >> >>> >> >>> While agree with most of the ideas about security, speed ... I still >> >>> can't get some really basic things. >> >>> >> >>> 1. Why one should trade a time-proven, close to metal, fars language >> >>> like >> >>> C with more than ~40yrs of existence with a new one? We don't even >> >>> know if >> >>> Rust will exist in the near future. That's not gonna be the case for C >> >>> apparently (IoT, etc.). >> > >> > >> > Well the existence of Rust ( or any language ) will depend entirely on >> > how >> > many new projects and existing ( c ) projects written in it! >> > Large sections of Firefox are now written in Rust ( in a version thats >> > shipping very soon ) cross ported from the Servo project, which gave >> > rise to >> > rust in order to build a totally new browser engine .. given that >> > browsers >> > are now some of the largest software projects on the planet this is a >> > good >> > sign. These include the CSS parser, CSS matcher, the Compositor and >> > several >> > more .. the eventual plan is to move everything over. >> > The benefits are many, and it still maintains C ABI compatibility if you >> > need it. >> > >> > >> >> >> >> >> >>> >> >>> 2. Why simply not simply learn how to better code our NIF/Drivers >> >>> instead? C was/is my main programming language for many years now, and >> >>> I >> >>> didn't have any major issue with it (medium to large projects) in >> >>> production >> >>> environment so far. Maybe I'm just lucky, maybe not. >> >> >> >> >> > >> > Well the more the tooling and language can do for you the better, and >> > the >> > more guarantees of correctness the more secure your software is likely >> > to >> > be. One of many reasons for rewriting Firefox in rust is security. Most >> > C >> > projects probably don't have hundreds/thousands of security guys trying >> > to >> > cause memory overflow errors, but for things like browsers, VM's, OS's >> > they >> > do, and with the increase of IOT many products that were not >> > traditionally >> > exposed to the internet now are .. and when one is discovered it's >> > usually >> > game over! >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > From davidnwelton@REDACTED Fri Sep 22 01:07:42 2017 From: davidnwelton@REDACTED (David Welton) Date: Thu, 21 Sep 2017 16:07:42 -0700 Subject: [erlang-questions] relx relup, appups and dependencies Message-ID: Hi, I'm working on some inherited code that uses relx to manage releases, and I've run into something that I'm not quite sure about: In previous release - let's call it 1.0.0, say, I have {cowboy, "1.0.0", {git, "git://github.com/extend/cowboy.git", {tag, "1.0.4"}}} In my rebar.config In the new one, 2.0.0 for the sake of argument, I have {cowboy, "1.0.4", {git, "git://github.com/extend/cowboy.git", {tag, "1.0.4"}}}, When I run relx relup, it complains: Could not open file erlapp/_rel/xxxxxxxxx/lib/cowboy-1.0.4/ebin/cowboy.appup How are other people handling this? I'm confident that cowboy is a well behaving Erlang app, so I'm confident I should be able to do this in my code without having to add anything to the cowboy code I've checked out. Thanks -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From ok@REDACTED Fri Sep 22 02:08:40 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 22 Sep 2017 12:08:40 +1200 Subject: [erlang-questions] Making a minimal distributable program in Erlang. In-Reply-To: References: Message-ID: On 21/09/17 9:00 PM, Joe Armstrong wrote: > I don't know how squashable the object code is - it might be nice to compress > the object code for (Mac,Windows,Linux) and put them in the Data area > of the program, then decompress them and load them on-the-fly. I happened to have a 2 MB a.out handy on my Mac. gzip -9 reduced it to 30% of the original size. I recompiled it from source on a SPARC/Solaris box. gzip -9 reduced that to 21% of its original size. (The a.out was bigger on the SPARC, but the a.out.gz was smaller.) Object file formats are optimized for speed of loading and for being able to page from them, so it's not surprising that they are pretty squashable. It seems rather likely that a custom compressor could beat gzip. The BEAM code of course would be system-independent. All the BEAM files in an Erlang/OTP distribution I had handy came to 170 MB. That did *not* compress well: gzip -9 just reduced it to (not by) 75%. I suspect that a custom compressor could do better. Taking all the .erl files in the same distribution (and yes, I know that is an under-estimate, but it isn't a *huge* underestimate) and stripping out the comments gave me 19 MB, which gzipped to 2.2 MB, so sending the sources instead of BEAM files would compress the Erlang part to 1.3% (to, not by). Again, BEAM files were not designed to save space. But this does very strongly suggest that BEAM files are not a good *distribution* format, and that a system using intelligently compress ASTs like Thomas Kistler's Juice system for Oberon might do very well, an observation I've made before. From ok@REDACTED Fri Sep 22 02:29:08 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 22 Sep 2017 12:29:08 +1200 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> Message-ID: On 21/09/17 10:23 PM, Jesper Louis Andersen wrote: > On Wed, Sep 20, 2017 at 12:50 AM Richard A. O'Keefe > wrote: > > One question. I can get a > "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" > for NZD 69 including tax and run Erlang on it. > (Hmm. Next time I have some spare cash I must get one...) > > Why would I pay NZD300 for a 300MHz board with 64MB of memory? > > > My prediction is that eventually we'll see the "small embedded" market > squeezed down by rather cheap powerful devices such as the rPI. My > argument for this prediction hinges on power becoming free in the future > (due to the current massive advances in solar and wind energy on top of > better batteries). I am rather intrigued by the "massive advances in solar"; solar firms seem to be going broke like so many autumn leaves. Suniva, Solar World, Suntech, Solyndra, ... I'm old enough to remember when the gospel was that nuclear power plants would make electricity "too cheap to meter". Supposing the cost of *generation* to go to zero, that would not make power "free" because there is still the cost of *distributing* it (to get economies of scale and to provide redundancy) and *storing* it. The makers of the better batteries will not be handing them out free for the sheer joy of giving, and nothing lasts forever (maybe the Pyramids; is pyramid power still a thing (:-)) so they will have an ongoing replacement cost. We've got people looking at sensor networks here. In fact I have a paper on the subject I must finish urgently. Sensor nodes are pretty amazing. In deep sleep they take so little power that you can practically ignore it compared with leakage in the battery. If you want a couple of sensors in every field, well, that's affordable nowadays, but *not* if you want to plug them into any kind of generator. (Wires and boxes -- distribution -- will be much more expensive than the nodes.) The problem with insolation is that it varies a lot. In winter, there are fewer daylight hours, and the *peak* insolation is about half of what it is in summer. At least, that's what the measurements I've been looking at say. So you are limited by the average power density available over the course of a day. It's enough for a small panel to power a sensor node, but assuming the technology per se to be free, the material and processing costs of producing and shipping a panel looks like making panels dearer than nodes. Maybe the right choice for sensor nodes might be microbial fuel cells, e.g., https://phys.org/news/2016-07-urinal-electricity-urine.html The farmer could recharge his sensors using a different form of leakage current. (Half fun and full earnest. This might actually work. It's certainly funny enough to try...) From ok@REDACTED Fri Sep 22 02:37:52 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 22 Sep 2017 12:37:52 +1200 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> Message-ID: <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> On 22/09/17 2:23 AM, Grzegorz Junka wrote: > To answer your question Richard, I don't actually need a cyclical > structure. As I mentioned, this is a simplification of a more specific > problem. I didn't want to describe all the specifics to ask a question. > I need to design a tree, a variant of a HAMT tree, but which can be > traversed in both directions, from the root to leafs and from leafs to > the root (independently, i.e. starting from a reference to a leaf rather > than recursively). Of course this pushes back the question a further step. Why do you want to traverse the tree in both directions? Might the "zipper" approach be any use? If you want references to leaves, such references could be paths. I used that technique once in a programming language that used reference counting, so that link cycles meant no collection. (Yes, I know modern reference-based collectors have solved that problem.) I've also used that technique for processing XML in a pure functional language. In an imperative context, that doesn't work very well because the tree structure might change. The DOM suffers from trying to make this "work" in some sense, but trying to iterate over a graph while something else is changing it is never going to be easy. It is possible that the Haskell "Functional Graph Library" may be relevant to your needs. From davidnwelton@REDACTED Fri Sep 22 03:49:54 2017 From: davidnwelton@REDACTED (David Welton) Date: Thu, 21 Sep 2017 18:49:54 -0700 Subject: [erlang-questions] relx relup, appups and dependencies In-Reply-To: References: Message-ID: I should add: The problem "goes away" if I don't specify the newer version in relx.config - but then it doesn't seem to actually use the newer version. Also, various pieces of this system I have inherited are older (~2 years), so that *could* be relevant. It looks like there may be some tools out there to help with this ("knit"?) - recommendations for what's worked for you would be welcome. Thanks -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From shr066@REDACTED Thu Sep 21 21:37:45 2017 From: shr066@REDACTED (Steve Rogers) Date: Thu, 21 Sep 2017 19:37:45 +0000 Subject: [erlang-questions] Light-weight operating systems supporting Erlang in production web servers In-Reply-To: <19A7D75B-8CCD-48C6-985F-8D3ECD49F109@stritzinger.com> References: <1505508481.2000934.1107665816.309201B6@webmail.messagingengine.com> <5dcca9da-bcb9-84e8-46d1-8b9e277c9940@cs.otago.ac.nz> <32fc9f19-5095-f09f-8d82-f06325d65138@cs.otago.ac.nz> <1B1D5F1D-6D25-4751-ACA0-075E513A4146@stritzinger.com> <19A7D75B-8CCD-48C6-985F-8D3ECD49F109@stritzinger.com> Message-ID: Great, thanks for the update! On Thu, Sep 21, 2017 at 2:29 PM Peer Stritzinger wrote: > Hi Steve, > > On 21.09.2017, at 21:16, Steve Rogers wrote: > > So when can we expect to get our hands on a GRiSP board/ > > > This week we finalized the hardware for production. From now there is a > minimum of 8 weeks production time > if there is delivery time on crucial parts possibly a bit longer, waiting > for the schedule from the manufacturer. > > So hoping to ship mid November. > > Definitely in time for Christmas season ;-) > > There is a newsletter subscription on the bottom of https://www.grisp.org > will post details of producing the first batch there. > > Best, > -- Peer > > # Steve > > On Thu, Sep 21, 2017 at 10:29 AM Peer Stritzinger > wrote: > >> On 21.09.2017, at 12:23, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >> On Wed, Sep 20, 2017 at 12:50 AM Richard A. O'Keefe >> wrote: >> >>> One question. I can get a >>> "Raspberry Pi 3 Model B with 1.2GHz 64-bit Quad Core and 1GB Ram" >>> for NZD 69 including tax and run Erlang on it. >>> (Hmm. Next time I have some spare cash I must get one...) >>> >>> Why would I pay NZD300 for a 300MHz board with 64MB of memory? >>> >>> >> My prediction is that eventually we'll see the "small embedded" market >> squeezed down by rather cheap powerful devices such as the rPI. My argument >> for this prediction hinges on power becoming free in the future (due to the >> current massive advances in solar and wind energy on top of better >> batteries). >> >> >> Small embedded being current rPI *sized* CPUs for some definition of >> small will be happening. Except rPI is not usable in production for >> several reasons (supply, quality, hardware changing beyond your control). >> >> Currently, the power draw of the stronger chips is too high and >> production at scale means that shaving off a few cents here and there is >> worth it. >> >> >> Exactly. We have a few industrial applications where we would love to get >> a larger CPU but we can?t because of price and power (meaning in this case >> thermal energy output). >> >> But it looks like the price tag of a 1.2 GHZ 64 bit chip is only going >> one way, and this makes the added development effort of targeting a smaller >> chip more expensive. In turn, you need an even bigger scale of production. >> And so on. >> >> >> Except that with a 300MHz Cortex M7 size CPU you can do lots of >> interesting applications just in Erlang already and save the costs. >> >> GRiSP is interesting because it can offer you a far faster time to market >> and lower development effort at the same or better robustness/resilience. >> You can thus get the lower production costs while amortizing development >> effort better. It is a good bet until the niche squeeze I predict above >> happens. >> >> >> Thanks :-) >> >> Also we plan not to be squeezed but adapt. One size which is just >> getting viable with Erlang are the automotive controllers with flash and >> ram on the CPU only (for the automotive industry any external memory is too >> unreliable and a problem for EMC). Thats one direction what I want to >> target with the GRiSP software stack. >> >> We could have easily also targeted a 24 core communication controller >> with 16 times Gbit I/O which is a tempting target also and noone would >> complain about the compute power there ;-) >> >> For the GRiSP-base we just tried to pick the right sized board for >> today. Others might follow? >> >> Best, >> -- Peer >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Sep 22 12:42:58 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 22 Sep 2017 12:42:58 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: 2017-09-21 19:12 GMT+02:00 Frank Muller : [...] > 2. Why simply not simply learn how to better code our NIF/Drivers instead? C > was/is my main programming language for many years now, and I didn't have > any major issue with it (medium to large projects) in production environment > so far. Maybe I'm just lucky, maybe not. Probably you're lucky. Last time I worked on a C code, there was a mysterious (but thankfully reproducable) crash. According to the core file the process tried to follow a null pointer variable, but if I checked the memory where the variable was, it was not null. I went down to the assembly code and the contents of the registers, there was nothing that explained the crash. The variable was in a structure that was initialized at startup, then never written. In final despair I moved the initialization from runtime to compile time and made that structure const hoping that the compiler/linker/kernel will put the structure into read-only memory so if anything tries to write to it, I'd get the crash at that time. Guess what happened: the original crash went away. Probably the memory was reorganized in a way that the stray overwrite now didn't touch this particular variable. And no, memory debuggers didn't help, ElectricFence didn't found the error, Valgrind didn't worked in that environment. I'm so glad that I don't have to work on that kind of C code anymore. From ebegumisa@REDACTED Fri Sep 22 12:58:55 2017 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 22 Sep 2017 20:58:55 +1000 Subject: [erlang-questions] Frequency of module reloading and the constant pool In-Reply-To: References: <15ea1631d50-c05-11e40@webjas-vaa103.srv.aolmail.net> Message-ID: On Thu, 21 Sep 2017 16:19:38 +1000, Lukas Larsson wrote: > > On Thu, Sep 21, 2017 at 12:23 AM, Edmond Begumisa > wrote: >> >> Apart from GC, is there anything else I should be concerned about like >> costly locks and memory >>barriers? For instance, is a fat mutex held >> for all schedulers when scanning all the process >>heaps? >> > No, there are no locks taken. Exactly how it is done is described here: > https://github.com/>erlang/otp/blob/master/erts/emulator/internal_doc/CodeLoading.md > Excellent resource! This was exactly what I needed. Thanks again Lukas. - Edmond - -- Using Opera's mail client: http://www.opera.com/mail/ From scott_ribe@REDACTED Fri Sep 22 13:08:36 2017 From: scott_ribe@REDACTED (scott ribe) Date: Fri, 22 Sep 2017 05:08:36 -0600 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: > Probably you're lucky. Last time I worked on a C code, there was a > mysterious (but thankfully reproducable) crash. According to the core > file the process tried to follow a null pointer variable, but if I > checked the memory where the variable was, it was not null. I went > down to the assembly code and the contents of the registers, there was > nothing that explained the crash. The variable was in a structure that > was initialized at startup, then never written. In final despair I > moved the initialization from runtime to compile time and made that > structure const hoping that the compiler/linker/kernel will put the > structure into read-only memory so if anything tries to write to it, > I'd get the crash at that time. Guess what happened: the original > crash went away. Probably the memory was reorganized in a way that the > stray overwrite now didn't touch this particular variable. And no, > memory debuggers didn't help, ElectricFence didn't found the error, > Valgrind didn't worked in that environment. I'm so glad that I don't > have to work on that kind of C code anymore. And that's why I'm hesitant to ever start another major project in C. I've got them out there that are pretty reliable, but it takes a lot of work to get to that point, and one single tiny error within a huge code base can lead to what you describe. -- Scott Ribe scott_ribe@REDACTED (303) 722-0567 From n.oxyde@REDACTED Fri Sep 22 14:39:17 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 22 Sep 2017 14:39:17 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: Joe you are still ignoring the elephant in the room that C is a memory-unsafe programming language and that you are suggesting putting it in IoT stuff. That's a security disaster waiting to happen. > Le 21 sept. 2017 ? 22:37, Joe Armstrong a ?crit : > > I'd like to make an extremely small extremely slow Ertang targeted to > IOT devices - low power devices with small memory and slow clocks. From zxq9@REDACTED Fri Sep 22 15:21:53 2017 From: zxq9@REDACTED (zxq9) Date: Fri, 22 Sep 2017 22:21:53 +0900 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: Message-ID: <9130534.8N4gR8Ot4x@changa> On 2017?09?22? ??? 14:39:17 Anthony Ramine wrote: > That's a security disaster waiting to happen. Oh, its not waiting for anything or anyone... /(O.o)\ -Craig From erlang@REDACTED Fri Sep 22 16:33:47 2017 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 22 Sep 2017 16:33:47 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: You're quite right C is memory unsafe, and the large quantity of C code in IoT devices is as you rightly say a disaster - only it has already happened no waiting is involved. I would like to *reduce* the amount of C - write an emulator for language X in C, then write everything in X. X would be a "better" language than C in some sense. X should compile to a small instruction set such that the implementation of the VM would be a simple and hopefully correct program. if we go back to the P-code machine the design was very simple and the implementation a few hundred lines of pascal (or C) - I'm pretty sure one could write a memory safe P-code interpreter in a memory-unsafe language like C. Security (since you mentioned it) ultimately relies on trust. Do you trust the compiler? do you trust the programmers? Do you trust the hardware? If I were building a secure system I would try to trust as little as possible putting firewalls and checks between components. Given a pure choice between a language that offered memory safety and one that did not I'd obviously choose the memory-safe language all other things being equal. Trouble is all other things are not equal. For me: Time to solve problem = Time to understand problem (T1) + Time to learn language X (T2) + Time to write program in X (T3) If I choose C then T2 is very small. Usually T1 >> T3. The "elephant in the room" is the time to be productive and know the idioms of programming in X. T3 is small when you are proficient in the language otherwise large. I have said before - I think it takes about 3 weeks to learn a language 6-24 months to know your way around the libraries and 10+ years to know how to solve a problem in your favorite language. I dabble with new languages, not to become productive in them, but to see what new ideas they embody. Niklas Wirth said many years ago that it was far better to be very good at one language than having superficial knowledge of many. Cheers /Joe On Fri, Sep 22, 2017 at 2:39 PM, Anthony Ramine wrote: > Joe you are still ignoring the elephant in the room that C is a memory-unsafe programming language and that you are suggesting putting it in IoT stuff. That's a security disaster waiting to happen. > >> Le 21 sept. 2017 ? 22:37, Joe Armstrong a ?crit : >> >> I'd like to make an extremely small extremely slow Ertang targeted to >> IOT devices - low power devices with small memory and slow clocks. > From bchesneau@REDACTED Fri Sep 22 17:00:48 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 22 Sep 2017 08:00:48 -0700 Subject: [erlang-questions] slave feature Message-ID: i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? Beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Fri Sep 22 17:08:01 2017 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Fri, 22 Sep 2017 12:08:01 -0300 Subject: [erlang-questions] slave feature In-Reply-To: References: Message-ID: Most likely not what you were looking for, but it?s used in gold_fever :) Brujo Benavides > On Sep 22, 2017, at 12:00, Benoit Chesneau wrote: > > i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? > > Beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Sep 22 17:55:20 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Sep 2017 18:55:20 +0300 Subject: [erlang-questions] slave feature In-Reply-To: References: Message-ID: We are running code that works with ffmpeg in nif in separate process. It is a protection from a code that works in nif. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfisher@REDACTED Fri Sep 22 18:01:10 2017 From: pfisher@REDACTED (Fisher, Paul) Date: Fri, 22 Sep 2017 16:01:10 +0000 Subject: [erlang-questions] slave feature In-Reply-To: References: Message-ID: We use it to start separate vms, which are set with different OS priorities, and then schedule work to run in the appropriate one. Basically letting the OS deal with prioritization of different CPU heavy workloads. -- paul Fellow, Engineering ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Benoit Chesneau Sent: Friday, September 22, 2017 10:00:48 AM To: erlang-questions@REDACTED Subject: [erlang-questions] slave feature i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? Beno?t Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang.org@REDACTED Fri Sep 22 18:06:13 2017 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Fri, 22 Sep 2017 18:06:13 +0200 Subject: [erlang-questions] slave feature In-Reply-To: References: Message-ID: <20170922160613.GA27126@jarowit.net> On Fri, Sep 22, 2017 at 06:55:20PM +0300, Max Lapshin wrote: > We are running code that works with ffmpeg in nif in separate process. It > is a protection from a code that works in nif. Oh, look, and I'm spawning FFmpegs as ports, and wanting to kill them is one of the reasons I'm writing a "my better subprocesses for Erlang" application. -- Stanislaw Klekot From peer@REDACTED Fri Sep 22 18:37:33 2017 From: peer@REDACTED (Peer Stritzinger) Date: Fri, 22 Sep 2017 18:37:33 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: <71671BBE-102D-461F-9706-AB734BCB61E2@stritzinger.com> If you are not aiming at super small hardware but lets say in the 100-300 MHz 32bit controller range there is another way to protect agains C problems at least in the application domain: Running Erlang directly on the hardware as we do frees the Translation Lookaside Buffers that are used for virtual memory. These small chips have some which allows very inefficiently to run Unix alike systems. If we use those VM features to jail in the C code in a NIF it can?t escape its little box. Therefore we can use Erlang and safe C NIFs on these embedded systems without the overhead of Unix alike OS, full virtual memory and OS processes. That eliminates trusting the application level C code. Unikernel and Erlang VM remain though. Best, -- Peer > On 22.09.2017, at 16:33, Joe Armstrong wrote: > > You're quite right C is memory unsafe, and the large quantity of C code > in IoT devices is as you rightly say a disaster - only it has already happened > no waiting is involved. > > I would like to *reduce* the amount of C - write an emulator for language X > in C, then write everything in X. X would be a "better" language than C > in some sense. X should compile to a small instruction set such that > the implementation of the VM would be a simple and hopefully correct program. > > if we go back to the P-code machine the design was very simple and the > implementation a few hundred lines of pascal (or C) - I'm pretty sure > one could write a memory safe P-code interpreter in a memory-unsafe > language like C. > > Security (since you mentioned it) ultimately relies on trust. Do you trust the > compiler? do you trust the programmers? Do you trust the hardware? > > If I were building a secure system I would try to trust as little as possible > putting firewalls and checks between components. > > Given a pure choice between a language that offered memory safety > and one that did not I'd obviously choose the memory-safe language > all other things being equal. > > Trouble is all other things are not equal. > > For me: > > Time to solve problem = > Time to understand problem (T1) + > Time to learn language X (T2) + > Time to write program in X (T3) > > If I choose C then T2 is very small. Usually T1 >> T3. > > The "elephant in the room" is the time to be productive and know the > idioms of programming in X. T3 is small when you are proficient > in the language otherwise large. > > I have said before - I think it takes about 3 weeks to learn a language > 6-24 months to know your way around the libraries and 10+ years to know > how to solve a problem in your favorite language. > > I dabble with new languages, not to become productive in them, > but to see what new ideas they embody. > > Niklas Wirth said many years ago that it was far better to be very good at one > language than having superficial knowledge of many. > > Cheers > > /Joe > > > > On Fri, Sep 22, 2017 at 2:39 PM, Anthony Ramine wrote: >> Joe you are still ignoring the elephant in the room that C is a memory-unsafe programming language and that you are suggesting putting it in IoT stuff. That's a security disaster waiting to happen. >> >>> Le 21 sept. 2017 ? 22:37, Joe Armstrong a ?crit : >>> >>> I'd like to make an extremely small extremely slow Ertang targeted to >>> IOT devices - low power devices with small memory and slow clocks. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Sep 22 18:36:43 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Sep 2017 19:36:43 +0300 Subject: [erlang-questions] slave feature In-Reply-To: <20170922160613.GA27126@jarowit.net> References: <20170922160613.GA27126@jarowit.net> Message-ID: We work with libavcodec, not libavformat and also we are working with nvidia, decklink and quicksync. All this stuff must be run in separate process* -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Sep 22 18:40:29 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Sep 2017 19:40:29 +0300 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <71671BBE-102D-461F-9706-AB734BCB61E2@stritzinger.com> References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> <71671BBE-102D-461F-9706-AB734BCB61E2@stritzinger.com> Message-ID: This is a nice discussion and it is hard to track all messages here. I just want to insert that rust is in a beginning of its way. There are lots of places that can degrade performance very seriously and are far for C level. So keep it calm =) However it is possible to make parts of system in it and it is good. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Sep 22 19:26:23 2017 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 22 Sep 2017 10:26:23 -0700 Subject: [erlang-questions] slave feature In-Reply-To: References: Message-ID: <59C547BF.80709@gmail.com> On 09/22/2017 08:00 AM, Benoit Chesneau wrote: > i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? > > You should be aware that nodes started with slave try to make the slave node use the master's filesystem automatically, though this doesn't work in all places (as shown in the bug https://bugs.erlang.org/browse/ERL-446 which affects tsung). This behavior doesn't seem to be well documented (nor the -master command line argument used on the erl executable) but it appears to be the main deciding factor for whether slave is usable for a given situation (in addition to the linking that occurs between the master node, making the nodes not exist independently). Best Regards, Michael From pfisher@REDACTED Fri Sep 22 19:35:47 2017 From: pfisher@REDACTED (Fisher, Paul) Date: Fri, 22 Sep 2017 17:35:47 +0000 Subject: [erlang-questions] slave feature In-Reply-To: <59C547BF.80709@gmail.com> References: , <59C547BF.80709@gmail.com> Message-ID: I should have clarified our use... we have a version of the slave module that is exactly the same except for that feature. We want to have each node handle its own IO directly to the system. -- paul Fellow, Engineering ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Michael Truog Sent: Friday, September 22, 2017 12:26:23 PM To: Benoit Chesneau; erlang-questions@REDACTED Subject: Re: [erlang-questions] slave feature On 09/22/2017 08:00 AM, Benoit Chesneau wrote: > i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? > > You should be aware that nodes started with slave try to make the slave node use the master's filesystem automatically, though this doesn't work in all places (as shown in the bug https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.erlang.org_browse_ERL-2D446&d=DwIFEA&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=ZYaInvQKkF5t9wYb-bJCCzIq2vg5EMJ-yPQV5oGSzhE&s=6hwbTzX8VVRetMB-wwsC4dH_o0qsG_TWFYjhbCw7Hbs&e= which affects tsung). This behavior doesn't seem to be well documented (nor the -master command line argument used on the erl executable) but it appears to be the main deciding factor for whether slave is usable for a given situation (in addition to the linking that occurs between the master node, making the nodes not exist independently). Best Regards, Michael _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwIFEA&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=ZYaInvQKkF5t9wYb-bJCCzIq2vg5EMJ-yPQV5oGSzhE&s=vNTGFf8Z6SLM093TNOTb5lcV2cYPKArsfp5_0u6JpuQ&e= Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sat Sep 23 09:09:19 2017 From: ok@REDACTED (ok@REDACTED) Date: Sat, 23 Sep 2017 19:09:19 +1200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> Message-ID: <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> > Joe you are still ignoring the elephant in the room that C is a > memory-unsafe programming language and that you are suggesting putting it > in IoT stuff. That's a security disaster waiting to happen. These days I am getting a little confused about what "IoT" actually means. I thought it was lots of small devices, but the last couple of talks I've view seem to take it as synonymous with the cloud. Let's go with the first definition: talking teddy bears, internet- connected lightbulbs, sensors using MPS430 CPUs and the like. All the "IoT" operating systems I know of are written in C (like Zephyr and RIOT) or C++ (like mbed OS). Putting C in IoT stuff is not a new suggestion. (Nor is the claim that it's the internet of insecure things new (:-).) Joe was explicitly talking about much as much as practically possible to Erlang, reducing the amount of C to perhaps just the emulator. C was originally designed for small systems (MPS430 size, in fact) where it was possible for one person to read all the code carefully in a reasonable time. I note that there are a number of tools to dramatically improve the reliability of C programs. For just one example, there is the "Memory-Safe C compiler". http://www.seclab.cs.sunysb.edu/mscc/ (It's remarkable how many let's-make-C-better tools have been developed in CAML.) There's even a memory safety checker nesCheck for the nesC C-like language used with TinyOS. https://nebelwelt.net/publications/files/17AsiaCCS2.pdf I could sit here all night citing papers about static and dynamic checkers and verifiers for C. "In C" and "in a memory-safe language" do NOT have to be exclusive alternatives. From Oliver.Korpilla@REDACTED Sat Sep 23 09:28:37 2017 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 23 Sep 2017 09:28:37 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> Message-ID: Hello. >These days I am getting a little confused about what "IoT" actually >means. I thought it was lots of small devices, but the last couple >of talks I've view seem to take it as synonymous with the cloud. >Let's go with the first definition: talking teddy bears, internet- >connected lightbulbs, sensors using MPS430 CPUs and the like. I think what makes it an "Internet of Things" is really the cloud aspect, not the micro-controller aspect. So, the foremost concern would be to allow these devices to interact with each other locally in a safe and defined way and to let them occasionally connect to the cloud as needed. So, it's not basically about the first definition. We already had that. It's like saying "Let's not have an internet of things." Having said that I see no immense security risk in writing code for a remote sensor in C. It avoids one of the prime security risks: human user interactions and all the buffer overrun problems and string processing stuff that is so hard to do safely that there are still books being written about. The question is when an individual device gets complex enough to warrant using proper abstractions - like using Erlang. That might not be a good business case for the smallest micro-controllers, but any device actually having to manage a more serious kind of network interaction might hugely benefit. The more the device is intended to do outside the cloud - or the more complex its interaction with other devices or the cloud becomes - the more readily I would say moving away from C would yield benefits. Of course that is a huge sidetrack from the discussion if the Erlang VM could benefit from a Rust rewrite. ;-) I mean, are we really arguing here that C is a good answer to the problem of writing networked software? Really?? Tongue in cheek implied. Cheers, Oliver From n.oxyde@REDACTED Sat Sep 23 10:22:20 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 23 Sep 2017 10:22:20 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> Message-ID: It is if you don't have the deep pockets to actually get licenses or the checkers, and developers experimented enough to comprehend them. > Le 23 sept. 2017 ? 09:09, ok@REDACTED a ?crit : > > I could sit here all night citing papers about static > and dynamic checkers and verifiers for C. "In C" and > "in a memory-safe language" do NOT have to be exclusive > alternatives. From n.oxyde@REDACTED Sat Sep 23 10:29:15 2017 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 23 Sep 2017 10:29:15 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> Message-ID: Just forge input to the sensor in such a way it triggers a buffer overflow or something else in it. Nowadays we can even forge DNA strands to confuse DNA readers so? https://twitter.com/bldgblog/status/895728956724322304 > Le 23 sept. 2017 ? 09:28, Oliver Korpilla a ?crit : > > Having said that I see no immense security risk in writing code for a remote sensor in C. It avoids one of the prime security risks: human user interactions From Oliver.Korpilla@REDACTED Sat Sep 23 11:23:40 2017 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 23 Sep 2017 11:23:40 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> Message-ID: An HTML attachment was scrubbed... URL: From palani@REDACTED Sat Sep 23 11:50:10 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Sat, 23 Sep 2017 15:20:10 +0530 Subject: [erlang-questions] how to reply message using erlang timer module Message-ID: Hi Guys, I want to send message to the front-end via websocket in particular time interval. So i use timer module for this purpose. *timer:apply_interval(1000, self(), <<"How' you doin'?">>),* But it returns the following error *15:18:20.558 [error] CRASH REPORT Process <0.570.0> with 0 neighbours crashed with reason: call to undefined function timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>)15:18:20.558 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>)* Its there any way to solve this problem. I already tried with *erlang:start_timer(60000, self(), <<"How' you doin'?">>)* Its only send message after particular timer. I didn't found method in this module to send message in particular interval. -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Sat Sep 23 12:17:23 2017 From: igor.clark@REDACTED (Igor Clark) Date: Sat, 23 Sep 2017 11:17:23 +0100 Subject: [erlang-questions] how to reply message using erlang timer module In-Reply-To: References: Message-ID: <58cd9fd3-65b5-696a-f5e2-a53d5c86333f@gmail.com> Hi Palanikumar, It looks like you're passing the erlang:start_timer/3 arguments to timer:apply_interval/4 : erlang:start_timer(Time, Dest, Msg) vs apply_interval(Time, Module, Function, Arguments) erlang:start_timer/3 sends Msg to Dest after Time ms; timer:apply_interval/4 calls Module:Function(Arguments) every Time ms. So the error message is telling you that the function timer:apply_interval/3 you're trying to call doesn't exist. HTH, Igor On 23/09/2017 10:50, Palanikumar Gopalakrishnan wrote: > Hi Guys, > > ???????????? I want? to send? message to the front-end via websocket > in particular time interval. > So i use timer module for this purpose. > *timer:apply_interval(1000, self(), <<"How' you doin'?">>),* > * > * > * > * > But it returns the following error > > *15:18:20.558 [error] CRASH REPORT Process <0.570.0> with 0 neighbours > crashed with reason: call to undefined function > timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) > 15:18:20.558 [error] Ranch listener my_cowboy_ws_server terminated > with reason: call to undefined function timer:apply_interval(6000, > <0.570.0>, <<"How' you doin'?">>) > > > * > Its there any way to solve this problem. > > I already tried with > > *erlang:start_timer(60000, self(), <<"How' you doin'?">>) > > * > Its only send message after particular timer. I didn't found method in > this module to send message in particular interval. > > > > > > > > > > > -- > > *Warm Regards,* > > *Palanikumar Gopalakrishnan *? > *Developer* > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Sat Sep 23 12:54:46 2017 From: igor.clark@REDACTED (Igor Clark) Date: Sat, 23 Sep 2017 11:54:46 +0100 Subject: [erlang-questions] how to reply message using erlang timer module In-Reply-To: References: <58cd9fd3-65b5-696a-f5e2-a53d5c86333f@gmail.com> Message-ID: You could try timer:send_interval/3 : > send_interval(Time, Pid, Message) > Evaluates Pid ! Message repeatedly after Time milliseconds. (Pid can > also be an atom of a registered name.) There's even a timer:send_interval/2 that assumes self() as the Pid, which looks like it might be what you want: > send_interval(Time, Message) > Same as send_interval(Time, self(), Message). One thing to note though is that the Common Caveats section says: > Creating timers usingerlang:send_after/3 > anderlang:start_timer/3 > , is much > more efficient than using the timers provided by thetimer > module in STDLIB. Thetimermodule > uses a separate process to manage the timers. That process can easily > become overloaded if many processes create and cancel timers > frequently (especially when using the SMP emulator). > I'm not sure if that applies to send_interval/{2,3} as well, or quite how many "many" means there - I guess it's probably "a really large number", but perhaps somebody could jump in here? But if so, and if you expect heavy traffic with an interval like this on every WS connection, an alternative might be to create a loop to send the message using send_after/3 instead of using the timer module. I.e. the process which handles Msg could call erlang:send_after( Time, self(), Msg ) when it's done its other work. Working out which process does that would depend on your app design - it might work to just do it in the process which handles the message you're sending, or you might need another process. Cheers i On 23/09/2017 11:21, Palanikumar Gopalakrishnan wrote: > Hi lgor Clark, > ?????????????????????? Thanks for the help.? yes you? are correct. Its > there any way to send message to websocket using > *timer:apply_interval/4*. I didn't find any examples. > > > > > On 23 September 2017 at 15:47, Igor Clark > wrote: > > Hi Palanikumar, > > It looks like you're passing the erlang:start_timer/3 > arguments to > timer:apply_interval/4 > : > > erlang:start_timer(Time, Dest, Msg) > > vs > > apply_interval(Time, Module, Function, Arguments) > > erlang:start_timer/3 sends Msg to Dest after Time ms; > timer:apply_interval/4 calls Module:Function(Arguments) every Time ms. > > So the error message is telling you that the function > timer:apply_interval/3 you're trying to call doesn't exist. > > HTH, > Igor > > > On 23/09/2017 10:50, Palanikumar Gopalakrishnan wrote: >> Hi Guys, >> >> ???????????? I want? to send? message to the front-end via >> websocket in particular time interval. >> So i use timer module for this purpose. >> *timer:apply_interval(1000, self(), <<"How' you doin'?">>),* >> * >> * >> * >> * >> But it returns the following error >> >> *15:18:20.558 [error] CRASH REPORT Process <0.570.0> with 0 >> neighbours crashed with reason: call to undefined function >> timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) >> 15:18:20.558 [error] Ranch listener my_cowboy_ws_server >> terminated with reason: call to undefined function >> timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) >> >> >> * >> Its there any way to solve this problem. >> >> I already tried with >> >> *erlang:start_timer(60000, self(), <<"How' you doin'?">>) >> >> * >> Its only send message after particular timer. I didn't found >> method in this module to send message in particular interval. >> >> >> >> >> >> >> >> >> >> >> -- >> >> *Warm Regards,* >> >> *Palanikumar Gopalakrishnan *? >> *Developer* >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > -- > > *Warm Regards,* > > *Palanikumar Gopalakrishnan *? > *Developer* > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From palani@REDACTED Sat Sep 23 14:34:07 2017 From: palani@REDACTED (Palanikumar Gopalakrishnan) Date: Sat, 23 Sep 2017 18:04:07 +0530 Subject: [erlang-questions] how to reply message using erlang timer module In-Reply-To: References: <58cd9fd3-65b5-696a-f5e2-a53d5c86333f@gmail.com> <161a00ca-08fe-391a-feba-06444145be1b@gmail.com> Message-ID: Hi Guys, I tried with* timer:send_interval/2 *, and *timer:send _interval/3 *also . see my trial and error result . Both them returns 17:54:47.250 [error] CRASH REPORT Process <0.575.0> with 0 neighbours crashed with reason: call to undefined function timer:sent_interval(1000, <<"Time announcement from erlang server">>) 17:54:47.250 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:sent_interval(1000, <<"Time announcement from erlang server">>) 17:59:06.678 [error] CRASH REPORT Process <0.570.0> with 0 neighbours crashed with reason: call to undefined function timer:sent_interval(1000, <0.570.0>, "Time announcement from erlang server") 17:59:06.678 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:sent_interval(1000, <0.570.0>, "Time announcement from erlang server") On 23 September 2017 at 17:00, Igor Clark wrote: > Hi again Palanikumar, check the error message - you're still calling > timer:apply_interval instead of timer:send_interval, except now you're only > passing 2 arguments: > > *call to undefined function timer:apply_interval(1000, "Time announcement > from erlang server")* > > Be careful to check the signatures of the functions you're calling and the > argument lists you're providing. This is just basic error checking ;-) > > NB, please hit "reply to all" in future rather than just "reply", as > you'll include the mailing list, and other people will be able to help too. > > Cheers > i > > On 23/09/2017 12:07, Palanikumar Gopalakrishnan wrote: > > Hi lgor, > > I tried with > *timer:send_interval/2 with this line >>> timer:apply_interval(1000, > "Time announcement from erlang server"),* > > returns the following error > > > > > > *16:33:20.264 [error] CRASH REPORT Process <0.570.0> with 0 neighbours > crashed with reason: call to undefined function timer:apply_interval(1000, > "Time announcement from erlang server") 16:33:20.265 [error] Ranch listener > my_cowboy_ws_server terminated with reason: call to undefined function > timer:apply_interval(1000, "Time announcement from erlang server") * > same as *timer:send_interval/3 *throw the error > > On 23 September 2017 at 16:24, Igor Clark wrote: > >> You could try timer:send_interval/3 >> : >> >> send_interval(Time, Pid, Message) >> Evaluates Pid ! Message repeatedly after Time milliseconds. (Pid can >> also be an atom of a registered name.) >> >> >> There's even a timer:send_interval/2 >> that assumes >> self() as the Pid, which looks like it might be what you want: >> >> send_interval(Time, Message) >> Same as send_interval(Time, self(), Message). >> >> >> One thing to note though is that the Common Caveats >> >> section says: >> >> Creating timers using erlang:send_after/3 >> and >> erlang:start_timer/3 >> , is much >> more efficient than using the timers provided by the timer >> module in STDLIB. The timermodule >> uses a separate process to manage the timers. That process can easily >> become overloaded if many processes create and cancel timers frequently >> (especially when using the SMP emulator). >> >> >> I'm not sure if that applies to send_interval/{2,3} as well, or quite how >> many "many" means there - I guess it's probably "a really large number", >> but perhaps somebody could jump in here? >> >> But if so, and if you expect heavy traffic with an interval like this on >> every WS connection, an alternative might be to create a loop to send the >> message using send_after/3 instead of using the timer module. I.e. the >> process which handles Msg could call erlang:send_after( Time, self(), >> Msg ) when it's done its other work. Working out which process does that >> would depend on your app design - it might work to just do it in the >> process which handles the message you're sending, or you might need another >> process. >> >> Cheers >> i >> >> >> >> On 23/09/2017 11:21, Palanikumar Gopalakrishnan wrote: >> >> Hi lgor Clark, >> Thanks for the help. yes you are correct. Its >> there any way to send message to websocket using *timer:apply_interval/4*. >> I didn't find any examples. >> >> >> >> >> On 23 September 2017 at 15:47, Igor Clark wrote: >> >>> Hi Palanikumar, >>> >>> It looks like you're passing the erlang:start_timer/3 >>> arguments to >>> timer:apply_interval/4 >>> : >>> >>> erlang:start_timer(Time, Dest, Msg) >>> >>> vs >>> >>> apply_interval(Time, Module, Function, Arguments) >>> >>> erlang:start_timer/3 sends Msg to Dest after Time ms; >>> timer:apply_interval/4 calls Module:Function(Arguments) every Time ms. >>> >>> So the error message is telling you that the function >>> timer:apply_interval/3 you're trying to call doesn't exist. >>> >>> HTH, >>> Igor >>> >>> >>> On 23/09/2017 10:50, Palanikumar Gopalakrishnan wrote: >>> >>> Hi Guys, >>> >>> I want to send message to the front-end via websocket in >>> particular time interval. >>> So i use timer module for this purpose. >>> *timer:apply_interval(1000, self(), <<"How' you doin'?">>),* >>> >>> >>> But it returns the following error >>> >>> >>> >>> >>> >>> *15:18:20.558 [error] CRASH REPORT Process <0.570.0> with 0 neighbours >>> crashed with reason: call to undefined function timer:apply_interval(6000, >>> <0.570.0>, <<"How' you doin'?">>) 15:18:20.558 [error] Ranch listener >>> my_cowboy_ws_server terminated with reason: call to undefined function >>> timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) * >>> Its there any way to solve this problem. >>> >>> I already tried with >>> >>> >>> >>> *erlang:start_timer(60000, self(), <<"How' you doin'?">>) * >>> Its only send message after particular timer. I didn't found method in >>> this module to send message in particular interval. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> *Warm Regards,* >>> >>> *Palanikumar Gopalakrishnan *[image: ?] >>> *Developer* >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >> >> >> -- >> >> *Warm Regards,* >> >> *Palanikumar Gopalakrishnan *[image: ?] >> *Developer* >> >> >> >> > > > -- > > *Warm Regards,* > > *Palanikumar Gopalakrishnan *[image: ?] > *Developer* > > > > -- *Warm Regards,* *Palanikumar Gopalakrishnan *[image: ?] *Developer* -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Sat Sep 23 15:39:55 2017 From: igor.clark@REDACTED (Igor Clark) Date: Sat, 23 Sep 2017 14:39:55 +0100 Subject: [erlang-questions] how to reply message using erlang timer module In-Reply-To: References: <58cd9fd3-65b5-696a-f5e2-a53d5c86333f@gmail.com> <161a00ca-08fe-391a-feba-06444145be1b@gmail.com> Message-ID: <543A29F8-9CB8-4517-B0EE-ECAE42B3591B@gmail.com> Hi Palanikumar, looks like you've got a typo there, it's send_interval not sent_interval: > call to undefined function timer:sent_interval(1000, <<"Time announcement from erlang server">>) > On 23 Sep 2017, at 13:34, Palanikumar Gopalakrishnan wrote: > > Hi Guys, > I tried with timer:send_interval/2 , and timer:send _interval/3 also . see my trial and error result . Both them returns > > > 17:54:47.250 [error] CRASH REPORT Process <0.575.0> with 0 neighbours crashed with reason: call to undefined function timer:sent_interval(1000, <<"Time announcement from erlang server">>) > 17:54:47.250 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:sent_interval(1000, <<"Time announcement from erlang server">>) > > > 17:59:06.678 [error] CRASH REPORT Process <0.570.0> with 0 neighbours crashed with reason: call to undefined function timer:sent_interval(1000, <0.570.0>, "Time announcement from erlang server") > 17:59:06.678 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:sent_interval(1000, <0.570.0>, "Time announcement from erlang server") > > > > >> On 23 September 2017 at 17:00, Igor Clark wrote: >> Hi again Palanikumar, check the error message - you're still calling timer:apply_interval instead of timer:send_interval, except now you're only passing 2 arguments: >> >> call to undefined function timer:apply_interval(1000, "Time announcement from erlang server") >> >> Be careful to check the signatures of the functions you're calling and the argument lists you're providing. This is just basic error checking ;-) >> >> NB, please hit "reply to all" in future rather than just "reply", as you'll include the mailing list, and other people will be able to help too. >> >> Cheers >> i >> >>> On 23/09/2017 12:07, Palanikumar Gopalakrishnan wrote: >>> Hi lgor, >>> >>> I tried with timer:send_interval/2 with this line >>> >>> timer:apply_interval(1000, "Time announcement from erlang server"), >>> >>> returns the following error >>> >>> 16:33:20.264 [error] CRASH REPORT Process <0.570.0> with 0 neighbours crashed with reason: call to undefined function timer:apply_interval(1000, "Time announcement from erlang server") >>> 16:33:20.265 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:apply_interval(1000, "Time announcement from erlang server") >>> >>> >>> same as timer:send_interval/3 throw the error >>> >>>> On 23 September 2017 at 16:24, Igor Clark wrote: >>>> You could try timer:send_interval/3: >>>> >>>>> send_interval(Time, Pid, Message) >>>>> Evaluates Pid ! Message repeatedly after Time milliseconds. (Pid can also be an atom of a registered name.) >>>> >>>> There's even a timer:send_interval/2 that assumes self() as the Pid, which looks like it might be what you want: >>>> >>>>> send_interval(Time, Message) >>>>> Same as send_interval(Time, self(), Message). >>>> >>>> One thing to note though is that the Common Caveats section says: >>>> >>>>> Creating timers using erlang:send_after/3 and erlang:start_timer/3 , is much more efficient than using the timers provided by the timer module in STDLIB. The timermodule uses a separate process to manage the timers. That process can easily become overloaded if many processes create and cancel timers frequently (especially when using the SMP emulator). >>>>> >>>> >>>> I'm not sure if that applies to send_interval/{2,3} as well, or quite how many "many" means there - I guess it's probably "a really large number", but perhaps somebody could jump in here? >>>> >>>> But if so, and if you expect heavy traffic with an interval like this on every WS connection, an alternative might be to create a loop to send the message using send_after/3 instead of using the timer module. I.e. the process which handles Msg could call erlang:send_after( Time, self(), Msg ) when it's done its other work. Working out which process does that would depend on your app design - it might work to just do it in the process which handles the message you're sending, or you might need another process. >>>> >>>> Cheers >>>> i >>>> >>>> >>>> >>>>> On 23/09/2017 11:21, Palanikumar Gopalakrishnan wrote: >>>>> Hi lgor Clark, >>>>> Thanks for the help. yes you are correct. Its there any way to send message to websocket using timer:apply_interval/4. I didn't find any examples. >>>>> >>>>> >>>>> >>>>> >>>>>> On 23 September 2017 at 15:47, Igor Clark wrote: >>>>>> Hi Palanikumar, >>>>>> >>>>>> It looks like you're passing the erlang:start_timer/3 arguments to timer:apply_interval/4: >>>>>> >>>>>> erlang:start_timer(Time, Dest, Msg) >>>>>> >>>>>> vs >>>>>> >>>>>> apply_interval(Time, Module, Function, Arguments) >>>>>> >>>>>> erlang:start_timer/3 sends Msg to Dest after Time ms; timer:apply_interval/4 calls Module:Function(Arguments) every Time ms. >>>>>> >>>>>> So the error message is telling you that the function timer:apply_interval/3 you're trying to call doesn't exist. >>>>>> >>>>>> HTH, >>>>>> Igor >>>>>> >>>>>> >>>>>> On 23/09/2017 10:50, Palanikumar Gopalakrishnan wrote: >>>>>>> Hi Guys, >>>>>>> >>>>>>> I want to send message to the front-end via websocket in particular time interval. >>>>>>> So i use timer module for this purpose. >>>>>>> timer:apply_interval(1000, self(), <<"How' you doin'?">>), >>>>>>> >>>>>>> >>>>>>> But it returns the following error >>>>>>> >>>>>>> 15:18:20.558 [error] CRASH REPORT Process <0.570.0> with 0 neighbours crashed with reason: call to undefined function timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) >>>>>>> 15:18:20.558 [error] Ranch listener my_cowboy_ws_server terminated with reason: call to undefined function timer:apply_interval(6000, <0.570.0>, <<"How' you doin'?">>) >>>>>>> >>>>>>> >>>>>>> Its there any way to solve this problem. >>>>>>> >>>>>>> I already tried with >>>>>>> >>>>>>> erlang:start_timer(60000, self(), <<"How' you doin'?">>) >>>>>>> >>>>>>> Its only send message after particular timer. I didn't found method in this module to send message in particular interval. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Warm Regards, >>>>>>> >>>>>>> Palanikumar Gopalakrishnan >>>>>>> Developer >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Warm Regards, >>>>> >>>>> Palanikumar Gopalakrishnan >>>>> Developer >>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> >>> Warm Regards, >>> >>> Palanikumar Gopalakrishnan >>> Developer >>> >>> >> > > > > -- > > Warm Regards, > > Palanikumar Gopalakrishnan > Developer > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Sat Sep 23 15:50:30 2017 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 23 Sep 2017 13:50:30 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: Zipper looks interesting but I don't think it would suit here. Keeping paths to leafs could work but since the tree is dynamic those paths would often change and it would be very difficult to update paths that have already been stored. In short, I have keys, which may be any Erlang terms, and numerical Ids assigned to those terms. Keys must be sorted. Numerical Ids are increasing monotonically. Then the following lookups should be efficient: 1. Having a key quickly search for its numeric Id 2. Having a numeric Id quickly get back the key Also the following conditions should be met: 3. The same key should be always assigned the same numeric Id 4. One key has always one numeric Id (and vice versa), so they are always added or removed together Since keys can be of any length I don't want to store them more than once. And since they must be sorted I was thinking about using a B+Tree. Then another structure to map the numeric Id back to their keys. An obvious solution would be to make the B+Tree bidirectional - the key would be recreated by traversing the tree back from a leaf to the root. The other structure would then need to keep a mapping between the numeric Id and the reference to the leaf for the key corresponding to that numeric Id. But an equally valid solution would be to tag all nodes of the B+Tree with numbers and store tags to parents in their children. Then in another structure, e.g. an array, the mapping between the tag and its node. That would also allow to traverse the tree backward from leafs to the root. GrzegorzJ On 22/09/2017 00:37, Richard A. O'Keefe wrote: > > > On 22/09/17 2:23 AM, Grzegorz Junka wrote: >> To answer your question Richard, I don't actually need a cyclical >> structure. As I mentioned, this is a simplification of a more specific >> problem. I didn't want to describe all the specifics to ask a question. >> I need to design a tree, a variant of a HAMT tree, but which can be >> traversed in both directions, from the root to leafs and from leafs to >> the root (independently, i.e. starting from a reference to a leaf rather >> than recursively). > > Of course this pushes back the question a further step. > Why do you want to traverse the tree in both directions? > > Might the "zipper" approach be any use? > > If you want references to leaves, such references could be paths. > I used that technique once in a programming language that used > reference counting, so that link cycles meant no collection. > (Yes, I know modern reference-based collectors have solved that > problem.) I've also used that technique for processing XML in > a pure functional language. > > In an imperative context, that doesn't work very well because the > tree structure might change. The DOM suffers from trying to make > this "work" in some sense, but trying to iterate over a graph > while something else is changing it is never going to be easy. > > It is possible that the Haskell "Functional Graph Library" > may be relevant to your needs. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Sat Sep 23 21:52:20 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 23 Sep 2017 21:52:20 +0200 Subject: [erlang-questions] nif: how to check if a resource is given as an arg Message-ID: Is there any better method than first trying enif_get_resource to check if a resourcce is given as an argument? What would be the impact of doing it? Should I use an explicit function instead of trying to match it? - benoit From g@REDACTED Sat Sep 23 22:10:43 2017 From: g@REDACTED (Guilherme Andrade) Date: Sat, 23 Sep 2017 21:10:43 +0100 Subject: [erlang-questions] [ANN] Backwater: an RPC framework for Erlang and Elixir Message-ID: Hi list, I'm pleased to announce the release of backwater 1.0.0. Backwater aims to make execution of calls to modules/functions on individual remote machines as close to local calls as they can get, with little to no boilerplate code. The remote calls are executed over HTTP(S) and don't depend on the Erlang distribution protocol nor on both ends being clustered. Backwater is built on top of cowboy and hackney. Goodies include generation of Erlang client code when using rebar3 (a homologous Elixir compiler task for Mix is still brewing.) Hoping it can be of use to someone. Cheers, -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Sep 23 22:11:52 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 23 Sep 2017 22:11:52 +0200 Subject: [erlang-questions] [ANN] Backwater: an RPC framework for Erlang and Elixir In-Reply-To: References: Message-ID: <6C58AF9F-0F99-4D8F-8852-2D8A36CB59C2@gmail.com> You forgot the link :) - beno?t > On 23 Sep 2017, at 22:10, Guilherme Andrade wrote: > > Hi list, > > I'm pleased to announce the release of backwater 1.0.0. > > Backwater aims to make execution of calls to modules/functions on individual remote machines as close to local calls as they can get, with little to no boilerplate code. The remote calls are executed over HTTP(S) and don't depend on the Erlang distribution protocol nor on both ends being clustered. > > Backwater is built on top of cowboy and hackney. Goodies include generation of Erlang client code when using rebar3 (a homologous Elixir compiler task for Mix is still brewing.) > > Hoping it can be of use to someone. > > Cheers, > > -- > Guilherme > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Sat Sep 23 22:11:55 2017 From: g@REDACTED (Guilherme Andrade) Date: Sat, 23 Sep 2017 21:11:55 +0100 Subject: [erlang-questions] [ANN] Backwater: an RPC framework for Erlang and Elixir In-Reply-To: References: Message-ID: My last message missed a rather critical piece of information[1]. [1]: Github: https://github.com/g-andrade/backwater On 23 September 2017 at 21:10, Guilherme Andrade wrote: > Hi list, > > I'm pleased to announce the release of backwater 1.0.0. > > Backwater aims to make execution of calls to modules/functions on > individual remote machines as close to local calls as they can get, with > little to no boilerplate code. The remote calls are executed over HTTP(S) > and don't depend on the Erlang distribution protocol nor on both ends being > clustered. > > Backwater is built on top of cowboy and hackney. Goodies include > generation of Erlang client code when using rebar3 (a homologous Elixir > compiler task for Mix is still brewing.) > > Hoping it can be of use to someone. > > Cheers, > > -- > Guilherme > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From sargun@REDACTED Sun Sep 24 10:25:54 2017 From: sargun@REDACTED (Sargun Dhillon) Date: Sun, 24 Sep 2017 01:25:54 -0700 Subject: [erlang-questions] Customizing Beam Build Message-ID: I've customized my OTP build to have a custom Erlang build. The reason I've done this to begin the process of building a fully-self contained Erlang program. One of the biggest benefits of the infrastructure in Go is that they're fully self-contained statically linked binaries. In Erlang, although you can statically link beam itself, unfortunately, the releases themselves are messy, in that you have a lot of files and getting the execution environment right isn't always trivial. Eventually, the purpose is to build an init system -- something that will be primarily used by sysadmin-types. These need to be statically linked, and often times, the file system isn't fully-functioning when they're starting. I've done two things: 1) I've modified the set of preloaded files in the VM. It would be really nice if there was a way to do this without forking OTP. I didn't add any existing beam files, but I just changed around some of the file, and code loading code to my need. 2) I've modified the final, linked product to include custom objects in the final ELF binary. This is kind of terrible, as it requires I maintain a fork of OTP. Does anyone have any recommendations for solving this problem another way? My proposal would be to output a beam.a file with the requisite objects to build a BEAM installation. It would be valuable if I could statically link against this binary, and own main, if it had a separate entrypoint that I could then call into. I've toyed around with this, and for the unix build, it seems possible. Is there any reason why Erlang isn't distributed this way already? If there was a public API for extending the loader, or adding preloaded modules via sections, I imagine that solving (1) would be easy as well. Is anyone open to accepting this upstream if the work is done? From erlang@REDACTED Sun Sep 24 12:27:03 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 24 Sep 2017 12:27:03 +0200 Subject: [erlang-questions] Customizing Beam Build In-Reply-To: References: Message-ID: This is great (I think) - if you've done what I think you might have done Have you seen following this thread? http://erlang.org/pipermail/erlang-questions/2017-September/093468.html I posted a comment here: http://erlang.org/pipermail/erlang-questions/2017-September/093496.html If we can link all the DLL's to a single together with a small number of precompiled erlang modules then I think making stand-alone applications is doable. I experimented with self-modifying code - you basically take the executable and append some beam code at the end of the executable. The precompiled part checks to see if any code has been added to the end of the executable and if so loads and runs it. The nice part about this is you can write the self-modifying part in Erlang. What is needed is a script that can make a stand-alone program with a number of precompiled erlang modules, say something like this: > make_stand_alone_executable x.erl y.erl z.erl -start x -out prog This would turn x.erl y.erl and z.erl into C (this is easy) then build a stand alone executable called prog. The command "prog args" would start execution of x:main(args) I thought to myself this must be possible since all the C code necessary is available - the fact that erlexec is small and does not statically link the emulator must have to do with the makefiles - Unfortunately I could not easily figure out how the makefiles were constructed. I tried to do this a while back - and remember there was one other problem - the code in erlexec presupposes a bin directory (or something like that) There would be another problem with NIFs - since these are dynamically loaded they would have to be statically linked instead. If you can make a statically linked binary containing a small number of pre-compiled beam files the rest should be easy. The only thing to watch for is to make sure that that static binary does not require and funny environment variables or directory paths to be set. To me "stand alone" means "stand alone" (ie the program should not break if you relocate it because you forgot to set some environment variable. If you have the basis for a 'make_stand_alone_executable' script it would be great - how to integrate this with all the autoconf stuff is a mystery to me. Cheers /Joe On Sun, Sep 24, 2017 at 10:25 AM, Sargun Dhillon wrote: > I've customized my OTP build to have a custom Erlang build. The reason > I've done this to begin the process of building a fully-self contained > Erlang program. One of the biggest benefits of the infrastructure in > Go is that they're fully self-contained statically linked binaries. In > Erlang, although you can statically link beam itself, unfortunately, > the releases themselves are messy, in that you have a lot of files and > getting the execution environment right isn't always trivial. > > Eventually, the purpose is to build an init system -- something that > will be primarily used by sysadmin-types. These need to be statically > linked, and often times, the file system isn't fully-functioning when > they're starting. > > I've done two things: > > 1) I've modified the set of preloaded files in the VM. It would be > really nice if there was a way to do this without forking OTP. I > didn't add any existing beam files, but I just changed around some of > the file, and code loading code to my need. > > 2) I've modified the final, linked product to include custom objects > in the final ELF binary. > > This is kind of terrible, as it requires I maintain a fork of OTP. > Does anyone have any recommendations for solving this problem another > way? > > My proposal would be to output a beam.a file with the requisite > objects to build a BEAM installation. It would be valuable if I could > statically link against this binary, and own main, if it had a > separate entrypoint that I could then call into. I've toyed around > with this, and for the unix build, it seems possible. Is there any > reason why Erlang isn't distributed this way already? > > If there was a public API for extending the loader, or adding > preloaded modules via sections, I imagine that solving (1) would be > easy as well. > > Is anyone open to accepting this upstream if the work is done? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Sun Sep 24 17:29:23 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 24 Sep 2017 15:29:23 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: Consider an ETS table and insert [{Key, Id}, {Id, Key}] This ensures fast lookup in both directions. Alternatively, run this in a map. I often do this when mapping monitor references in code. On Sat, Sep 23, 2017 at 3:50 PM Grzegorz Junka wrote: > Zipper looks interesting but I don't think it would suit here. Keeping > paths to leafs could work but since the tree is dynamic those paths > would often change and it would be very difficult to update paths that > have already been stored. > > In short, I have keys, which may be any Erlang terms, and numerical Ids > assigned to those terms. Keys must be sorted. Numerical Ids are > increasing monotonically. Then the following lookups should be efficient: > > 1. Having a key quickly search for its numeric Id > 2. Having a numeric Id quickly get back the key > > Also the following conditions should be met: > > 3. The same key should be always assigned the same numeric Id > 4. One key has always one numeric Id (and vice versa), so they are > always added or removed together > > Since keys can be of any length I don't want to store them more than > once. And since they must be sorted I was thinking about using a B+Tree. > Then another structure to map the numeric Id back to their keys. > > An obvious solution would be to make the B+Tree bidirectional - the key > would be recreated by traversing the tree back from a leaf to the root. > The other structure would then need to keep a mapping between the > numeric Id and the reference to the leaf for the key corresponding to > that numeric Id. > > But an equally valid solution would be to tag all nodes of the B+Tree > with numbers and store tags to parents in their children. Then in > another structure, e.g. an array, the mapping between the tag and its > node. That would also allow to traverse the tree backward from leafs to > the root. > > GrzegorzJ > > > On 22/09/2017 00:37, Richard A. O'Keefe wrote: > > > > > > On 22/09/17 2:23 AM, Grzegorz Junka wrote: > >> To answer your question Richard, I don't actually need a cyclical > >> structure. As I mentioned, this is a simplification of a more specific > >> problem. I didn't want to describe all the specifics to ask a question. > >> I need to design a tree, a variant of a HAMT tree, but which can be > >> traversed in both directions, from the root to leafs and from leafs to > >> the root (independently, i.e. starting from a reference to a leaf rather > >> than recursively). > > > > Of course this pushes back the question a further step. > > Why do you want to traverse the tree in both directions? > > > > Might the "zipper" approach be any use? > > > > If you want references to leaves, such references could be paths. > > I used that technique once in a programming language that used > > reference counting, so that link cycles meant no collection. > > (Yes, I know modern reference-based collectors have solved that > > problem.) I've also used that technique for processing XML in > > a pure functional language. > > > > In an imperative context, that doesn't work very well because the > > tree structure might change. The DOM suffers from trying to make > > this "work" in some sense, but trying to iterate over a graph > > while something else is changing it is never going to be easy. > > > > It is possible that the Haskell "Functional Graph Library" > > may be relevant to your needs. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Sun Sep 24 18:02:05 2017 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 24 Sep 2017 16:02:05 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: Thanks but that won't work. As mentioned keys must be sorted so that I can quickly query for range from min/max key. I could add an index but in that case the key would be stored twice, once in the index and once in the value, which I wanted to avoid. GrzegorzJ On 24/09/2017 15:29, Jesper Louis Andersen wrote: > Consider an ETS table and insert > > [{Key, Id}, {Id, Key}] > > This ensures fast lookup in both directions. Alternatively, run this > in a map. I often do this when mapping monitor references in code. > > On Sat, Sep 23, 2017 at 3:50 PM Grzegorz Junka > wrote: > > Zipper looks interesting but I don't think it would suit here. Keeping > paths to leafs could work but since the tree is dynamic those paths > would often change and it would be very difficult to update paths that > have already been stored. > > In short, I have keys, which may be any Erlang terms, and > numerical Ids > assigned to those terms. Keys must be sorted. Numerical Ids are > increasing monotonically. Then the following lookups should be > efficient: > > 1. Having a key quickly search for its numeric Id > 2. Having a numeric Id quickly get back the key > > Also the following conditions should be met: > > 3. The same key should be always assigned the same numeric Id > 4. One key has always one numeric Id (and vice versa), so they are > always added or removed together > > Since keys can be of any length I don't want to store them more than > once. And since they must be sorted I was thinking about using a > B+Tree. > Then another structure to map the numeric Id back to their keys. > > An obvious solution would be to make the B+Tree bidirectional - > the key > would be recreated by traversing the tree back from a leaf to the > root. > The other structure would then need to keep a mapping between the > numeric Id and the reference to the leaf for the key corresponding to > that numeric Id. > > But an equally valid solution would be to tag all nodes of the B+Tree > with numbers and store tags to parents in their children. Then in > another structure, e.g. an array, the mapping between the tag and its > node. That would also allow to traverse the tree backward from > leafs to > the root. > > GrzegorzJ > > > On 22/09/2017 00:37, Richard A. O'Keefe wrote: > > > > > > On 22/09/17 2:23 AM, Grzegorz Junka wrote: > >> To answer your question Richard, I don't actually need a cyclical > >> structure. As I mentioned, this is a simplification of a more > specific > >> problem. I didn't want to describe all the specifics to ask a > question. > >> I need to design a tree, a variant of a HAMT tree, but which can be > >> traversed in both directions, from the root to leafs and from > leafs to > >> the root (independently, i.e. starting from a reference to a > leaf rather > >> than recursively). > > > > Of course this pushes back the question a further step. > > Why do you want to traverse the tree in both directions? > > > > Might the "zipper" approach be any use? > > > > If you want references to leaves, such references could be paths. > > I used that technique once in a programming language that used > > reference counting, so that link cycles meant no collection. > > (Yes, I know modern reference-based collectors have solved that > > problem.) I've also used that technique for processing XML in > > a pure functional language. > > > > In an imperative context, that doesn't work very well because the > > tree structure might change. The DOM suffers from trying to make > > this "work" in some sense, but trying to iterate over a graph > > while something else is changing it is never going to be easy. > > > > It is possible that the Haskell "Functional Graph Library" > > may be relevant to your needs. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Sep 24 18:43:28 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 24 Sep 2017 16:43:28 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: Of course it will work: the ETS table is an ordered_set. This ensures that you have Term order. Now, in Erlang, numbers and thus integers have order less than everything else. In short, min/1 is given by a call to ets:first/1. To get max, we observe that the atom 'x' is not part of your keys, nor part of the integers. So it is larger than any integer. If we ask ets:prev/2 on that 'x', we get max/1. This covers the range operations on IDs. For the range operations on keys, it is even easier because they already have sorted order. You can also get rather fast traversal by using ets:select/1,2 and using the continuations to walk the data set in larger chunks. On Sun, Sep 24, 2017 at 6:02 PM Grzegorz Junka wrote: > Thanks but that won't work. As mentioned keys must be sorted so that I can > quickly query for range from min/max key. I could add an index but in that > case the key would be stored twice, once in the index and once in the > value, which I wanted to avoid. > > GrzegorzJ > > On 24/09/2017 15:29, Jesper Louis Andersen wrote: > > Consider an ETS table and insert > > [{Key, Id}, {Id, Key}] > > This ensures fast lookup in both directions. Alternatively, run this in a > map. I often do this when mapping monitor references in code. > > On Sat, Sep 23, 2017 at 3:50 PM Grzegorz Junka wrote: > >> Zipper looks interesting but I don't think it would suit here. Keeping >> paths to leafs could work but since the tree is dynamic those paths >> would often change and it would be very difficult to update paths that >> have already been stored. >> >> In short, I have keys, which may be any Erlang terms, and numerical Ids >> assigned to those terms. Keys must be sorted. Numerical Ids are >> increasing monotonically. Then the following lookups should be efficient: >> >> 1. Having a key quickly search for its numeric Id >> 2. Having a numeric Id quickly get back the key >> >> Also the following conditions should be met: >> >> 3. The same key should be always assigned the same numeric Id >> 4. One key has always one numeric Id (and vice versa), so they are >> always added or removed together >> >> Since keys can be of any length I don't want to store them more than >> once. And since they must be sorted I was thinking about using a B+Tree. >> Then another structure to map the numeric Id back to their keys. >> >> An obvious solution would be to make the B+Tree bidirectional - the key >> would be recreated by traversing the tree back from a leaf to the root. >> The other structure would then need to keep a mapping between the >> numeric Id and the reference to the leaf for the key corresponding to >> that numeric Id. >> >> But an equally valid solution would be to tag all nodes of the B+Tree >> with numbers and store tags to parents in their children. Then in >> another structure, e.g. an array, the mapping between the tag and its >> node. That would also allow to traverse the tree backward from leafs to >> the root. >> >> GrzegorzJ >> >> >> On 22/09/2017 00:37, Richard A. O'Keefe wrote: >> > >> > >> > On 22/09/17 2:23 AM, Grzegorz Junka wrote: >> >> To answer your question Richard, I don't actually need a cyclical >> >> structure. As I mentioned, this is a simplification of a more specific >> >> problem. I didn't want to describe all the specifics to ask a question. >> >> I need to design a tree, a variant of a HAMT tree, but which can be >> >> traversed in both directions, from the root to leafs and from leafs to >> >> the root (independently, i.e. starting from a reference to a leaf >> rather >> >> than recursively). >> > >> > Of course this pushes back the question a further step. >> > Why do you want to traverse the tree in both directions? >> > >> > Might the "zipper" approach be any use? >> > >> > If you want references to leaves, such references could be paths. >> > I used that technique once in a programming language that used >> > reference counting, so that link cycles meant no collection. >> > (Yes, I know modern reference-based collectors have solved that >> > problem.) I've also used that technique for processing XML in >> > a pure functional language. >> > >> > In an imperative context, that doesn't work very well because the >> > tree structure might change. The DOM suffers from trying to make >> > this "work" in some sense, but trying to iterate over a graph >> > while something else is changing it is never going to be easy. >> > >> > It is possible that the Haskell "Functional Graph Library" >> > may be relevant to your needs. >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Mon Sep 25 01:50:45 2017 From: mfidelman@REDACTED (Miles Fidelman) Date: Sun, 24 Sep 2017 16:50:45 -0700 Subject: [erlang-questions] question re. message delivery Message-ID: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> Folks, I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a question that's been nagging me. As I understand it, message delivery is not guaranteed, but message order IS. So how, exactly does that work?? What's the underlying mechanism that imposes sequencing, but allows messages to get lost?? (Particularly across a network.)? What are the various scenarios at play? Inquiring minds want to know! Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From zxq9@REDACTED Mon Sep 25 03:10:31 2017 From: zxq9@REDACTED (zxq9) Date: Mon, 25 Sep 2017 10:10:31 +0900 Subject: [erlang-questions] question re. message delivery In-Reply-To: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> References: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> Message-ID: <2921926.TZIjoQhpI3@changa> On 2017?09?24? ??? 16:50:45 Miles Fidelman wrote: > Folks, > > I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a > question that's been nagging me. > > As I understand it, message delivery is not guaranteed, but message > order IS. So how, exactly does that work? What's the underlying > mechanism that imposes sequencing, but allows messages to get lost? > (Particularly across a network.) What are the various scenarios at play? This is sort of backwards. Message delivery is guaranteed, assuming the process you are sending a message to exists and is available, BUT from the perspective of the sender there is no way to tell whether the receiver actually got it, has crashed, disappeared, fell into a network blackhole, or whatever. Monitoring can tell you whether the process you are trying to reach is available right at that moment, but that's it. The point is, though, that whether the receiver is unreachable, has crashed, got the message and did its work but was unable to report back about it, or whatever -- its all the same reality from the perspective of the sender. "Unavailable" means "unavailable", not matter what the cause -- because the cause cannot be determined from the perspective of the sender. You can only know this with an out of context check of some sort, and that is basically the role the runtime plays for you with regard to monitors and links. The OTP synchronous "call" mechanism is actually a complex procedure built from asynchronous messages, unique reference tags, and monitors. What IS guaranteed is the ordering of messages *relative to two processes*. If A sends B the messages 1, 2 and 3 in that order, they will certainly arrive in that order (assuming they arrive at all -- meaning that B is available from the perspective of A). If C sends B the messages 4, 5, 6 in that order those will also certainly arrive in that order for B. If A sends B and C the messages 1, 2 and 3, and as a reaction C starts sending B the messages 4, 5, 6 -- we can never know what order of interleaving these will have. It could be [1,2,3,4,5,6], or [1,2,4,5,3,6] or [1,4,5,6,2,3] or whatever, but only the relative ordering between a pair of processes can be known. A digression about design implications... One magical side effect of these strict guarantees AND strict ambiguities is that right from the start of a project in Erlang, even one running on a local system, you wind up staring the CAP theorem straight in the face. This tends to result in a better understanding of the constraints introduced by concurrency and distribution because they are present in the mind of every developer right from the start. The general outcome I've noticed (but don't know how to quantify with a metric of any sort) is that consideration of design tradeoffs rules architecture, even on a subconscious level, and this really bears itself out as a project matures. -Craig From mfidelman@REDACTED Mon Sep 25 08:24:31 2017 From: mfidelman@REDACTED (Miles Fidelman) Date: Sun, 24 Sep 2017 23:24:31 -0700 Subject: [erlang-questions] question re. message delivery In-Reply-To: <2921926.TZIjoQhpI3@changa> References: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> <2921926.TZIjoQhpI3@changa> Message-ID: <3d05c826-4f76-6ae1-1ca1-a0762148c0c2@meetinghouse.net> See below.... On 9/24/17 6:10 PM, zxq9 wrote: > On 2017?09?24? ??? 16:50:45 Miles Fidelman wrote: >> Folks, >> >> I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a >> question that's been nagging me. >> >> As I understand it, message delivery is not guaranteed, but message >> order IS. So how, exactly does that work? What's the underlying >> mechanism that imposes sequencing, but allows messages to get lost? >> (Particularly across a network.) What are the various scenarios at play? > This is sort of backwards. > > Message delivery is guaranteed, assuming the process you are sending a > message to exists and is available, BUT from the perspective of the > sender there is no way to tell whether the receiver actually got it, > has crashed, disappeared, fell into a network blackhole, or whatever. > Monitoring can tell you whether the process you are trying to reach > is available right at that moment, but that's it. > > The point is, though, that whether the receiver is unreachable, has > crashed, got the message and did its work but was unable to report > back about it, or whatever -- its all the same reality from the > perspective of the sender. "Unavailable" means "unavailable", not matter > what the cause -- because the cause cannot be determined from the > perspective of the sender. You can only know this with an out of > context check of some sort, and that is basically the role the runtime > plays for you with regard to monitors and links. > > The OTP synchronous "call" mechanism is actually a complex procedure > built from asynchronous messages, unique reference tags, and monitors. > Note that I didn't ask about the synchronous calls, I asked about raw interprocess messages. > What IS guaranteed is the ordering of messages *relative to two processes* > > If A sends B the messages 1, 2 and 3 in that order, they will certainly > arrive in that order (assuming they arrive at all -- meaning that B is > available from the perspective of A). But that's the question.? Particularly when sent via network, 1, 2, 3 may be sent in that order, but, at the protocol level, they may not arrive in that order. With a reliable transport protocol - say TCP - if the message-containing packets arrived as 1, 3, 2, the protocol engine would wait for 2 to arrive and deliver 1,2,3 in that order.? If It received 1 & 3, but 2 got lost, it would request a re-transmit, wait for it to arrive, and again, deliver in that order. But the implication of Erlang's stated rules is that an unreliable transport protocol is being used, if you send 1, 2, 3, and what arrives is 1, 3, 2 - then what would be delivered to the receiving PiD is 1 & 3 and 2 would be discarded.? Is that a correct assumption about the underlying transport mechanism?? Is that guaranteed behavior? Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From list1@REDACTED Mon Sep 25 08:35:43 2017 From: list1@REDACTED (Grzegorz Junka) Date: Mon, 25 Sep 2017 06:35:43 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: Fair enough, I missed that you can set ordered_set in ETS, but nevertheless it won't work with maps. However, even if I use ETS the Key is still stored twice, isn't? (once as the key for ordered_set, once as the value when the Id is the key). On 24/09/2017 16:43, Jesper Louis Andersen wrote: > Of course it will work: the ETS table is an ordered_set. This ensures > that you have Term order. Now, in Erlang, numbers and thus integers > have order less than everything else. In short, min/1 is given by a > call to ets:first/1. To get max, we observe that the atom 'x' is not > part of your keys, nor part of the integers. So it is larger than any > integer. If we ask ets:prev/2 on that 'x', we get max/1. > > This covers the range operations on IDs. For the range operations on > keys, it is even easier because they already have sorted order. You > can also get rather fast traversal by using ets:select/1,2 and using > the continuations to walk the data set in larger chunks. > > On Sun, Sep 24, 2017 at 6:02 PM Grzegorz Junka > wrote: > > Thanks but that won't work. As mentioned keys must be sorted so > that I can quickly query for range from min/max key. I could add > an index but in that case the key would be stored twice, once in > the index and once in the value, which I wanted to avoid. > > GrzegorzJ > > > On 24/09/2017 15:29, Jesper Louis Andersen wrote: >> Consider an ETS table and insert >> >> [{Key, Id}, {Id, Key}] >> >> This ensures fast lookup in both directions. Alternatively, run >> this in a map. I often do this when mapping monitor references in >> code. >> >> On Sat, Sep 23, 2017 at 3:50 PM Grzegorz Junka > > wrote: >> >> Zipper looks interesting but I don't think it would suit >> here. Keeping >> paths to leafs could work but since the tree is dynamic those >> paths >> would often change and it would be very difficult to update >> paths that >> have already been stored. >> >> In short, I have keys, which may be any Erlang terms, and >> numerical Ids >> assigned to those terms. Keys must be sorted. Numerical Ids are >> increasing monotonically. Then the following lookups should >> be efficient: >> >> 1. Having a key quickly search for its numeric Id >> 2. Having a numeric Id quickly get back the key >> >> Also the following conditions should be met: >> >> 3. The same key should be always assigned the same numeric Id >> 4. One key has always one numeric Id (and vice versa), so >> they are >> always added or removed together >> >> Since keys can be of any length I don't want to store them >> more than >> once. And since they must be sorted I was thinking about >> using a B+Tree. >> Then another structure to map the numeric Id back to their keys. >> >> An obvious solution would be to make the B+Tree bidirectional >> - the key >> would be recreated by traversing the tree back from a leaf to >> the root. >> The other structure would then need to keep a mapping between the >> numeric Id and the reference to the leaf for the key >> corresponding to >> that numeric Id. >> >> But an equally valid solution would be to tag all nodes of >> the B+Tree >> with numbers and store tags to parents in their children. Then in >> another structure, e.g. an array, the mapping between the tag >> and its >> node. That would also allow to traverse the tree backward >> from leafs to >> the root. >> >> GrzegorzJ >> >> >> On 22/09/2017 00:37, Richard A. O'Keefe wrote: >> > >> > >> > On 22/09/17 2:23 AM, Grzegorz Junka wrote: >> >> To answer your question Richard, I don't actually need a >> cyclical >> >> structure. As I mentioned, this is a simplification of a >> more specific >> >> problem. I didn't want to describe all the specifics to >> ask a question. >> >> I need to design a tree, a variant of a HAMT tree, but >> which can be >> >> traversed in both directions, from the root to leafs and >> from leafs to >> >> the root (independently, i.e. starting from a reference to >> a leaf rather >> >> than recursively). >> > >> > Of course this pushes back the question a further step. >> > Why do you want to traverse the tree in both directions? >> > >> > Might the "zipper" approach be any use? >> > >> > If you want references to leaves, such references could be >> paths. >> > I used that technique once in a programming language that used >> > reference counting, so that link cycles meant no collection. >> > (Yes, I know modern reference-based collectors have solved that >> > problem.) I've also used that technique for processing XML in >> > a pure functional language. >> > >> > In an imperative context, that doesn't work very well >> because the >> > tree structure might change. The DOM suffers from trying >> to make >> > this "work" in some sense, but trying to iterate over a graph >> > while something else is changing it is never going to be easy. >> > >> > It is possible that the Haskell "Functional Graph Library" >> > may be relevant to your needs. >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Sep 25 08:53:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 25 Sep 2017 08:53:09 +0200 Subject: [erlang-questions] question re. message delivery In-Reply-To: <3d05c826-4f76-6ae1-1ca1-a0762148c0c2@meetinghouse.net> References: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> <2921926.TZIjoQhpI3@changa> <3d05c826-4f76-6ae1-1ca1-a0762148c0c2@meetinghouse.net> Message-ID: <20170925065309.GA41893@erix.ericsson.se> On Sun, Sep 24, 2017 at 11:24:31PM -0700, Miles Fidelman wrote: > See below.... > > > On 9/24/17 6:10 PM, zxq9 wrote: > > On 2017?09?24? ??? 16:50:45 Miles Fidelman wrote: > >> Folks, > >> > >> I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a > >> question that's been nagging me. > >> > >> As I understand it, message delivery is not guaranteed, but message > >> order IS. So how, exactly does that work? What's the underlying > >> mechanism that imposes sequencing, but allows messages to get lost? > >> (Particularly across a network.) What are the various scenarios at play? > > This is sort of backwards. > > > > Message delivery is guaranteed, assuming the process you are sending a > > message to exists and is available, BUT from the perspective of the > > sender there is no way to tell whether the receiver actually got it, > > has crashed, disappeared, fell into a network blackhole, or whatever. > > Monitoring can tell you whether the process you are trying to reach > > is available right at that moment, but that's it. > > > > The point is, though, that whether the receiver is unreachable, has > > crashed, got the message and did its work but was unable to report > > back about it, or whatever -- its all the same reality from the > > perspective of the sender. "Unavailable" means "unavailable", not matter > > what the cause -- because the cause cannot be determined from the > > perspective of the sender. You can only know this with an out of > > context check of some sort, and that is basically the role the runtime > > plays for you with regard to monitors and links. > > > > The OTP synchronous "call" mechanism is actually a complex procedure > > built from asynchronous messages, unique reference tags, and monitors. > > > > Note that I didn't ask about the synchronous calls, I asked about raw > interprocess messages. > > > What IS guaranteed is the ordering of messages *relative to two processes* > > > > If A sends B the messages 1, 2 and 3 in that order, they will certainly > > arrive in that order (assuming they arrive at all -- meaning that B is > > available from the perspective of A). > > But that's the question.? Particularly when sent via network, 1, 2, 3 > may be sent in that order, but, at the protocol level, they may not > arrive in that order. What protocol level? Erlang distribution has to use or implement a reliable protocol. Today TCP, but anything is possible. Note that this protocol is between two nodes, both containing many processes. But the emulator relies on the transport protocol being reliable. > > With a reliable transport protocol - say TCP - if the message-containing > packets arrived as 1, 3, 2, the protocol engine would wait for 2 to > arrive and deliver 1,2,3 in that order.? If It received 1 & 3, but 2 got > lost, it would request a re-transmit, wait for it to arrive, and again, > deliver in that order. > > But the implication of Erlang's stated rules is that an unreliable > transport protocol is being used, if you send 1, 2, 3, and what arrives What? What is stated? > is 1, 3, 2 - then what would be delivered to the receiving PiD is 1 & 3 > and 2 would be discarded.? Is that a correct assumption about the > underlying transport mechanism?? Is that guaranteed behavior? Aah. That would not be possible, 1 might be delivered to a Pid1, and 3 to Pid2, if you send to a registered name. But if you send to a Pid; 1 might be delivered and 2 & 3 discarded, 1 & 2 delivered and 3 discarded, or 1 & 2 & 3 delivered. The order is guaranteed while the destination is available, and if it is not, the messages are discarded. But if you send to a registered name the Pid behind that name may change any time. Still order is guaranteed - up to a point all messages are delivered in order, then some may be discarded, then when the new pid is up and registered all messages are delivered in order again. > > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Mon Sep 25 11:05:13 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 25 Sep 2017 11:05:13 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170904084815.GA43354@erix.ericsson.se> References: <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <59A8F03E.9040400@gmail.com> <20170901085439.GD87875@erix.ericsson.se> <59A93DEB.6000205@gmail.com> <20170901121346.GB11300@erix.ericsson.se> <59AB404B.4040408@gmail.com> <20170904084815.GA43354@erix.ericsson.se> Message-ID: <20170925090513.GB51084@erix.ericsson.se> On Mon, Sep 04, 2017 at 10:48:15AM +0200, Raimo Niskanen wrote: : > We should add a 'rand' plugin to the 'crypto' module that does this > buffered crypto:strong_random_bytes/1 trick. There is something like that > in rand_SUITE, but we should really have an official one. > > I also wonder where the sweet spot is? 64 KB seems like a lot of buffer. > I have created a pull request for such an extension to the crypto module: https://github.com/erlang/otp/pull/1573 -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Mon Sep 25 11:07:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 25 Sep 2017 11:07:09 +0200 Subject: [erlang-questions] On OTP rand module difference between OTP 19 and OTP 20 In-Reply-To: <20170908122628.GA77024@erix.ericsson.se> References: <5aead032-c932-48db-3453-329e043780aa@cs.otago.ac.nz> <20170831123008.GA73645@erix.ericsson.se> <59A856EC.1030000@gmail.com> <2d198a4b-7963-d578-696a-2725f3eee573@cs.otago.ac.nz> <20170901084958.GC87875@erix.ericsson.se> <9ea82731-b43f-1d0f-93f4-29c6d7822438@cs.otago.ac.nz> <20170904094905.GB43354@erix.ericsson.se> <0efe7c07-91f4-f8f3-4cfb-a968d7f2f2c0@cs.otago.ac.nz> <20170908122628.GA77024@erix.ericsson.se> Message-ID: <20170925090709.GC51084@erix.ericsson.se> On Fri, Sep 08, 2017 at 02:26:28PM +0200, Raimo Niskanen wrote: > To conclude > =========== > > It would be convenient to have functions in the rand module that > generates on the interval (0.0, 1.0] e.g rand:uniform_nonzero/0 > and rand:uniform_nonzero_s/1 or maybe rand:uniform_nz/0 > and rand:uniform_nz_s/1. > > But since it is very easy to use e.g (1.0 - rand:uniform()) I see > little value in adding them. Adding hints in the documentation could > suffice. > > However, I think we could add functions with the same names and > interval that also has got a different uniformity i.e still uniform, > but not equidistant, instead increasing precison towards 0. This would > have a greater value. Such a uniformity would work better for some > suggested algorithms such as Box-Muller. > > Ironically, the implementation by Kenji that I changed was a few bits in > that direction i.e used the generators' extra bits over 53 (5 or 11) > for increased precision, but I want to have at least 53 extra bits which I > hope is close enough to infinity. > > I have implemented such functions in my own GitHub repo, but ran into > problems with the number distribution that I suspect is caused by > rounding errors in the current implementation of erlang:float/1. > > So I will try to find the time to investigate that further... I have created a pull request with such an extension to the rand module: https://github.com/erlang/otp/pull/1574 -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From attila.r.nohl@REDACTED Mon Sep 25 11:41:52 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 25 Sep 2017 11:41:52 +0200 Subject: [erlang-questions] Erlang VM in Rust In-Reply-To: References: <83a85eee-fdce-a7fa-b8c0-0feb6d3eeb33@cs.otago.ac.nz> <4dbdf4551bdc4f97c5c2db3110221ab9.squirrel@chasm.otago.ac.nz> Message-ID: 2017-09-23 9:28 GMT+02:00 Oliver Korpilla : [...] > Having said that I see no immense security risk in writing code for a remote sensor in C. It avoids one of the prime security risks: human user interactions and all the buffer overrun problems and string processing stuff that is so hard to do safely that there are still books being written about. If it is connected to the internet (the first letter in IoT), then it at least needs to handle IPv4 - and it involves parsing of potentially untrusted data. From sverker.eriksson@REDACTED Mon Sep 25 11:52:30 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 25 Sep 2017 11:52:30 +0200 Subject: [erlang-questions] nif: how to check if a resource is given as an arg In-Reply-To: References: Message-ID: <9b8b55b8-59ca-c20c-f5bc-44cb2d59bcd5@ericsson.com> In OTP-20 you can compare one resource term to another and get a match if they refer to the same resource instance. Beyond that, enif_get_resource is the *only* way to check if a term is a resource. /Sverker On 09/23/2017 09:52 PM, Benoit Chesneau wrote: > Is there any better method than first trying enif_get_resource to check if a resourcce is given as an argument? What would be the impact of doing it? Should I use an explicit function instead of trying to match it? > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From codewiget95@REDACTED Mon Sep 25 15:25:48 2017 From: codewiget95@REDACTED (code wiget) Date: Mon, 25 Sep 2017 09:25:48 -0400 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions Message-ID: Hello everyone, As I get further into Erlang, I am starting to realize that some of my functions have been getting pretty ugly with nested case statements. For example, I had a nested case statement that looked something like this: Send_update(Arg1) -> case do this(Arg1) of {ok, [Val1, Val2]} -> case do_that(Val1, Val2) of {ok, [Val3, Val4]} -> case do_this2(?) of ?. It continued into this for another few functions, you get the picture - its ugly, and it is hard to read. So I went and I converted it to a top level function that would then call lower level functions like so: send_update(Arg1) -> case ... of {ok, [Val1, Val2]} -> send_update(check_indices, {Arg1, Val1, Val2}); Else -> lager:error("ERROR: ..") end. send_update(check_indices, {Arg1, Arg2, Arg3}) -> case check_indices(Arg2, Arg3)of true -> send_update(get_values, {Arg1, Arg3}); false -> lager:error("EMERGENCY: ?.") end; send_update(get_values, {Arg1, Arg2}) -> ... case ... of {ok, [Val1, Val2, VAl3]} -> send_update(send_value, {Arg1, Val1, Val2, Val3}); Error -> lager:error("ERROR: ?") end; send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> ? Do_something(Args), ok. Now that I look at it though, both don?t look right. They don?t look like something I would write in any other language where I would just have if?s and else?s. Is this the proper way to write Erlang? I know everyone has their own style, but I assume there is some accepted form of writing functional programs with deep nests. Thank you for your advice! From daniel.goertzen@REDACTED Mon Sep 25 16:15:21 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 25 Sep 2017 14:15:21 +0000 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: References: Message-ID: You are headed in the right direction. Two comments on your refactor: 1. Ditch the atom selector. Just make this part of the function name. 2. You can flatten out the remaining case expressions if you like. Whether this is better or not is a matter of opinion, but it definitely makes tracing easier. This is where I got with your code... send_update(Arg1) -> send_update_check_indices(...). send_update_check_indices({ok, [Val1, Val2]}) -> send_update_check_values(...); send_update_check_indices(Else) -> lager:error("ERROR: .."). % and so on On Mon, Sep 25, 2017 at 8:26 AM code wiget wrote: > Hello everyone, > > As I get further into Erlang, I am starting to realize that some of my > functions have been getting pretty ugly with nested case statements. For > example, I had a nested case statement that looked something like this: > > Send_update(Arg1) -> > case do this(Arg1) of > {ok, [Val1, Val2]} -> > case do_that(Val1, Val2) of > {ok, [Val3, Val4]} -> > case do_this2(?) of > ?. > > It continued into this for another few functions, you get the picture - > its ugly, and it is hard to read. > > So I went and I converted it to a top level function that would then call > lower level functions like so: > > > > send_update(Arg1) -> > case ... of > {ok, [Val1, Val2]} -> > send_update(check_indices, {Arg1, Val1, Val2}); > Else -> > lager:error("ERROR: ..") > end. > send_update(check_indices, {Arg1, Arg2, Arg3}) -> > case check_indices(Arg2, Arg3)of > true -> > send_update(get_values, {Arg1, Arg3}); > false -> > lager:error("EMERGENCY: ?.") > end; > send_update(get_values, {Arg1, Arg2}) -> > ... > case ... of > {ok, [Val1, Val2, VAl3]} -> > send_update(send_value, {Arg1, Val1, Val2, Val3}); > Error -> > lager:error("ERROR: ?") > end; > send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> > ? > Do_something(Args), > ok. > > > Now that I look at it though, both don?t look right. They don?t look like > something I would write in any other language where I would just have if?s > and else?s. > > Is this the proper way to write Erlang? I know everyone has their own > style, but I assume there is some accepted form of writing functional > programs with deep nests. > > Thank you for your advice! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Mon Sep 25 16:20:03 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Mon, 25 Sep 2017 10:20:03 -0400 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: References: Message-ID: <31D72E6F-8034-4871-B57D-81AAD641F615@waisbrot.net> More functions is almost always better. You are writing in a functional language. :) I lean towards naming the functions for what they'll do, rather than having an initial arg that tells them what to do: `do_something(Arg)` rather than `do(something, Arg)`. In your code, it looks like you have a success-path and anything that deviates is an error (a serious one?). I'd begin by writing it: ``` send_update(Request) -> {ok, IndexArgs} = check_request(Request), {ok, IndexResult} = check_indices(IndexArgs) {ok, ValueResult} = get_values(IndexResult) ok = transmit(ValueResult). ``` Here, we have a clear description of the happy-path and anything else will crash. If this is a server handling single-shot client requests (a REST server, for example) then you can either silently drop bad clients or have a top-level exception handler that just returns a 4xx or 5xx error. That may be enough right there, but even if you want richer responses to your clients I think the happy-path is a great way to start. The other thing it will do is help you start to think about overall resilience: it's cool if you drop one TCP connection because of an unexpected request, but all your other live connections and the rest of the system ought to be unaffected. > On Sep 25, 2017, at 9:25 AM, code wiget wrote: > > Hello everyone, > > As I get further into Erlang, I am starting to realize that some of my functions have been getting pretty ugly with nested case statements. For example, I had a nested case statement that looked something like this: > > Send_update(Arg1) -> > case do this(Arg1) of > {ok, [Val1, Val2]} -> > case do_that(Val1, Val2) of > {ok, [Val3, Val4]} -> > case do_this2(?) of > ?. > > It continued into this for another few functions, you get the picture - its ugly, and it is hard to read. > > So I went and I converted it to a top level function that would then call lower level functions like so: > > > > send_update(Arg1) -> > case ... of > {ok, [Val1, Val2]} -> > send_update(check_indices, {Arg1, Val1, Val2}); > Else -> > lager:error("ERROR: ..") > end. > send_update(check_indices, {Arg1, Arg2, Arg3}) -> > case check_indices(Arg2, Arg3)of > true -> > send_update(get_values, {Arg1, Arg3}); > false -> > lager:error("EMERGENCY: ?.") > end; > send_update(get_values, {Arg1, Arg2}) -> > ... > case ... of > {ok, [Val1, Val2, VAl3]} -> > send_update(send_value, {Arg1, Val1, Val2, Val3}); > Error -> > lager:error("ERROR: ?") > end; > send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> > ? > Do_something(Args), > ok. > > > Now that I look at it though, both don?t look right. They don?t look like something I would write in any other language where I would just have if?s and else?s. > > Is this the proper way to write Erlang? I know everyone has their own style, but I assume there is some accepted form of writing functional programs with deep nests. > > Thank you for your advice! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Mon Sep 25 17:11:00 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 26 Sep 2017 00:11 +0900 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: References: Message-ID: <5752590.zR9ZgV7JUD@changa> On 2017?09?25? ??? 09:25:48 code wiget wrote: > Hello everyone, > > As I get further into Erlang, I am starting to realize that some of my functions have been getting pretty ugly with nested case statements. For example, I had a nested case statement that looked something like this: > > Send_update(Arg1) -> > case do this(Arg1) of > {ok, [Val1, Val2]} -> > case do_that(Val1, Val2) of > {ok, [Val3, Val4]} -> > case do_this2(?) of > ?. > > It continued into this for another few functions, you get the picture - its ugly, and it is hard to read. This happens a lot when you're first starting out. Don't worry, it gets better. And you're headed in the right direction, asking the right questions, clearly sensitive to the right smells. > So I went and I converted it to a top level function that would then call lower level functions like so: > > > > send_update(Arg1) -> > case ... of > {ok, [Val1, Val2]} -> > send_update(check_indices, {Arg1, Val1, Val2}); > Else -> > lager:error("ERROR: ..") > end. > send_update(check_indices, {Arg1, Arg2, Arg3}) -> > case check_indices(Arg2, Arg3)of > true -> > send_update(get_values, {Arg1, Arg3}); > false -> > lager:error("EMERGENCY: ?.") > end; > send_update(get_values, {Arg1, Arg2}) -> > ... > case ... of > {ok, [Val1, Val2, VAl3]} -> > send_update(send_value, {Arg1, Val1, Val2, Val3}); > Error -> > lager:error("ERROR: ?") > end; > send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> > ? > Do_something(Args), > ok. > > > Now that I look at it though, both don?t look right. They don?t look like something I would write in any other language where I would just have if?s and else?s. > > Is this the proper way to write Erlang? I know everyone has their own style, but I assume there is some accepted form of writing functional programs with deep nests. This is the universe sending you a message: 1- Some of your messages are supposed indicate dispatch to specific functions. 2- VERY often having a bunch of nested cases means that you have awkward data structures and need to rethink what your program is actually doing. 3- Occasionally you will have a series of checks you need to perform in a specific order -- and there we can use a special form of a fold somtimes called a "pipeline" to normalize this. Case 1: Cutting your example down: send_update(Arg1) -> ... . send_update(check_indices, {Arg1, Arg2, Arg3}) -> ... ; send_update(get_values, {Arg1, Arg2}) -> ... ; send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> ... . Why are these all called `send_update/1,2`? They aren't SENDING any UPDATES as far as I can tell. They are DISPATCHING based on the shape of the received input, and that's a totally different thing. In fact, this particular example is recursing over itself in a pretty weird way that confuses the call graph in my mind. Imagine if it had, say, 20 more clauses! It is quite common to find yourself in need of something analogous to gen_server:handle_cast/2 or handle_call/3. Quite a lot of the time when you need to reinvent this on your own, however, you are dealing with either unsanitized data, or data with lots of variable arguments. If you have, say, raw messages coming over the wire to a pure Erlang process (a process hand-written using proc_lib instead of being a gen_server or whatever) you'll often have something like: handle(Incoming, State = #s{buffer = Buffer}) -> case interpret(Incoming) of {ok, Message, Rest} -> dispatch(Message, State#s{buffer = add_to(Buffer, Rest)}); {error, Error} -> ok = log(warning, "Got bad things: ~tp", [Error]), terminate() end. dispatch(Message, State = #s{socket = Socket}) -> Result = case Message of {check_indices, {A1, A2, A3}} -> check_indices(A1, A2, A3); {get_values, {A1, A2}} -> get_values(A1, A2); {send_value, Values} -> send_value(Values, Socket); Unknown -> ok = log(error, "Probably should avoid this clause because we should die here."), terminate() end, NewState = some_state_update_or_whatever(State), {Result, NewState}. You might need the above procedure to be larger or smaller, have a return send back through the socket inserted somewhere, enter a discreet receive loop as a reaction (which defines a special state of the process as a state machine), have another function do more pre-processing or more post-processing, or whatever. That's not the point. The point is that dispatching and recursively calling different clauses of the same function that have SEMANTICALLY DIFFERENT MEANING is just asking for code that is really hard to read. Now consider the origin of that convolution -- you were putting way too much in a single clause of a function, and then you resorted to putting way too much into several clauses of the same function, which is a different form of the same confusion. Break things down, dispatch to specific handlers, pull the result to be returned back to the dispatcher, handle the result in a common way. This isn't always possible, but it nearly always is, which is the reason it is possible to write event loops at all. Case 2: When you've never really thought much about what your program is doing it is pretty easy to get rather cavalier with data and wind up with a lot of nested data that is hard to get at, and then you suddenly find that you want to get at the inner data only some of the time and you can shortcut excessive checks with `case` and start turning your nested data problem into a nested `case` problem as well. The biggest culprit there that I've seen are nested maps and nested records -- and nested maps of nested records and vice-versa. Whenever you find yourself nesting a lot of data and the data structures themselves start feeling arbitrary and awkward to manipulate -- JUST STOP WHAT YOU'RE DOING. Before you write another universally shared record inside some .hrl file STOP. Once this point has been reached the universe is telling you that you really need to be writing a module that is an abstract data structure. Write a module that contains an inner state structure of some kind, and (basically) setter/getter/updater functions for it. That's how quite a bit of the stdlib is written, and it is a major headache saver. Make the exported data type be opaque to external callers so that Dialyzer can catch you when you start getting sneaky and trying to be too intimate with the hidden structure. The reason for this is simple to understand: you didn't understand your data initially so it is likely you still don't quite get it, so you're going to eventually modify the shape of things later. If you are too intimate with the data internals instead of hiding them behind functions then you'll wind up blowing up a lot of code (as you proabably already know). This will leave you with a choice: in the interest of time do you just write another clause next to the existing one, add another `case` layer somewhere, or actually rewrite everything to be "correct"? You'll almost never get the time or motivation to do it write later -- so another level of cases or clause cancer it will usually be. Ugh. Just evil. Case 3: If you have a need to check whether some input is good, the permissions of a user, the context of the request, and the status of some external element before proceeding with an action, how could we do this without 4 levels of `case`? Especially if we want to return a properly detailed error message? We could trickle-down through a series of functions that essentially curry out the stuff we've dealth with: handle_request(Request, User, Context, ExternalThingy) -> case interpret(Request) of {ok, Action} -> handle_request2(Action, User, Context, ExternalThingy); Error -> Error end. handle_request2(Action, User, Context, ExternalThingy) -> case check_permission(User, Action) of ok -> handle_request3(Action, User, Context, ExternalThingy); Error -> Error end. handle_request3(Action, User, Context, ExternalThingy) -> case dispatch_target(Action, Context) of {ok, Target} -> handle_request4(Target, User, Context, ExternalThingy); Error -> Error end. handle_request(Target, User, Context, ExternalThingy) -> case check_condition(ExternalThingy) of ok -> dispatch(Target, User, Context); Error -> Error end. % ...and so on. Now that's not really so bad, and in some cases it is really the right sort of solution because you might have a lot of variance in what is going on in each step. The case above, however, is so boringly vanilla that WE DON'T EVEN HAVE SPECIAL FUNCTION NAMES. That's a sign of something dumb going on. It is pretty rare that you should ever be writing functions named the same thing plus some incrementing number at the end, or find yourself misunderstanding single assignment to such a degree that you write a long line of updating variables that look like State1, State2, State3, State4. That sort of thing is one more hint to yourself that you're both not understanding the problem and probably trying to write, say, Python in Erlang. But consider how little variance we had above. Maybe we could do something more like... pre_check(Action, User, Context, ExternalThingy) -> Checks = [fun check_request/2, fun check_permission/2, fun check_dispatch_target/2, fun check_condition/2], Args = {Action, User, Context, ExternalThingy}, Harness = fun (Check, ok) -> Check(Args); (_, Error) -> Error end, case lists:foldl(Harness, ok, Checks) of ok -> dispatch(Action, User, Context); Error -> Error end. Now the above example is pretty short, only having four checks to perform, but you can probably see quite quickly how we could have eliminated the Args variable entirely there and written a more general form of this function as a list operation (specifically, a non-backtracking, non-transforming pipeline). In this case we have a static thingy (your collection of initial arguments) and a list of thingies we want to run through (your checks). A normal fold would be a list of things to run a single function over, so in this case we have just swapped that around and have a static set of values we want to run a list of operations over. But values are just values, whether they are functions or not, so this is STILL a fold, if you choose to structure it that way. Not everything lends itself to this kind of structure, of course, but quite often when you see a big chain of `case` statements pushing out past the right side of the screen you've either got a strong indication a pipeline is needed (or that you're looking at machine generated code). There are several kinds of pipelines you can craft, depending on the situation, and the harness doesn't always have to be a closure (transforming pipelines don't have any need for that, for example). Anyway.... While I've been typing this out, way past my bedtime and with copious mistakes, bad grammar and typos, I see that some others have already homed in on the first thing I mentioned: those initial atom-based-clause switches should be their own functions. Once you begin to refactor in this direction you will begin to see which sort of situation you are in as far as determining execution flow, and you will also subconsciously begin to EVOLVE A BETTER MODEL of your actual program, the problem it handles, and will probably stumble on a way to make most of the code either move somewhere else or collapse tremendously. Simplicity tends to follow complexity. Whatever you write, get something working, then refactor it! -Craig From zxq9@REDACTED Mon Sep 25 17:26:05 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 26 Sep 2017 00:26:05 +0900 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: <5752590.zR9ZgV7JUD@changa> References: <5752590.zR9ZgV7JUD@changa> Message-ID: <2327773.3Xg827ZKDe@changa> Dhoh! On 2017?09?26? ??? 00:11:00 you wrote: > handle_request(Target, User, Context, ExternalThingy) -> Should be handle_request4/4. > pre_check(Action, User, Context, ExternalThingy) -> > Checks = > [fun check_request/2, > fun check_permission/2, > fun check_dispatch_target/2, > fun check_condition/2], These should all be arity of one, obviously. I knew something like this was bound to happen at 00:25... ;-) Anyway, hopefully I explained more than I confused. The main point remains: you are on the right track and your code will become much nicer to read in short order. -Craig From jesper.louis.andersen@REDACTED Mon Sep 25 17:48:54 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 25 Sep 2017 15:48:54 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: On Mon, Sep 25, 2017 at 8:35 AM Grzegorz Junka wrote: > Fair enough, I missed that you can set ordered_set in ETS, but > nevertheless it won't work with maps. However, even if I use ETS the Key is > still stored twice, isn't? (once as the key for ordered_set, once as the > value when the Id is the key). > > It depends. If it is a large binary (larger than 64 characters) it will go on the binary heap once unless you form it again and again in your code. Otherwise it will take up the double amount of space. We still don't know what the underlying problem statement is. This makes it harder to solve because whenever we come up with a solution, a new constraint is added and we have to adapt. One advantage of using something like a gb_trees for the above is that you only have the key once in the process heap and everything else will be pointers. It is also possible to use the above scheme with gb_trees. But then again, with ETS you can do key lookup from any process, whereas it has to factor through the tree owner with gb_trees. There is also the risk your problem is entirely too large to fit in memory at all, but we don't yet know the size of your N and how large of a machine you are willing to scale to, so it is hard to do any napkin math on the size here. In short, we need more info if we are to come up with a better solution :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Sep 25 17:57:37 2017 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 25 Sep 2017 17:57:37 +0200 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: References: Message-ID: Good question. This is example of abnormal termination. The Erlang 'way' is to write the happy path and, not write twisty little passages full of error correcting code. I'll assume send_update has a single argument 'a' or 'b' - and is as follows: send_update(Arg1) -> {Val1, Val2} = do_this(Arg1), {Val3, Val4} = do_that(Arg1), Val2+Val3. do_this(a) -> {1,2}; do_this(b) -> {3,4}. do_that(a) ->{10,20}; do_that(b) ->{15,12}. This is version 1 - no decent errors. Here's what we see in the shell: 1> 2> demo:send_update(a). 12 3> demo:send_update(c). ** exception error: no function clause matching demo:do_this(c) (demo.erl, line 9) in function demo:send_update/1 (demo.erl, line 5) The error message is not very good - but is enough to get you started in finding the error. We can get some information like this: 4> (catch demo:send_update(c)). {'EXIT',{function_clause,[{demo,do_this, [c], [{file,"demo.erl"},{line,9}]}, {demo,send_update,1,[{file,"demo.erl"},{line,5}]}, ... So you can see a stack trace. But we want better than this. So now I'll make a few small changes. send_update_v1(Arg1) -> {Val1, Val2} = do_this_v1(Arg1), {Val3, Val4} = do_that_v1(Arg1), Val2+Val3. do_this_v1(a) -> {1,2}; do_this_v1(b) -> {3,4}; do_this_v1(X) -> exit({do_this_v1,bad_arg,X}). do_that_v1(a) ->{10,20}; do_that_v1(b) ->{15,12}; do_that_v1(X) -> exit({do_that_v1,bad_arg,X}). We can now call this as follows: 1> demo:send_update_v1(a). 12 8> demo:send_update_v1(z). ** exception exit: {do_this_v1,bad_arg,z} in function demo:do_this_v1/1 (demo.erl, line 25) in call from demo:send_update_v1/1 (demo.erl, line 19) 9> (catch demo:send_update_v1(z)). {'EXIT',{do_this_v1,bad_arg,z}} Now let's get creative: send_update_v2(Arg1) -> try send_update_v2_happy(Arg1) of X -> X catch throw:{Type,Val} -> lager_error(Type, Val) end. send_update_v2_happy(Arg1) -> {Val1, Val2} = do_this_v2(Arg1), {Val3, Val4} = do_that_v2(Arg1), Val2+Val3. do_this_v2(a) -> {1,2}; do_this_v2(b) -> {3,4}; do_this_v2(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). do_that_v2(a) ->{10,20}; do_that_v2(b) ->{15,12}; do_that_v2(X) -> throw({"ERROR",{v2,bad_arg,X}}). lager_error(Type, Val) -> io:format("Error:~p ~p~n",[Type,Val]). Here we retain the happy path code - but add a wrapper that catches the error and reports it. 1> demo:send_update_v2(a). 12 2> demo:send_update_v2(b). 19 3> demo:send_update_v2(c). Error:"EMERGENCY" {v1,bad_arg,c} This code is beginning to look nice - but we're not their yet ... Lets' stop and think. We've refactored the code into three parts: + the happy path + the subroutines that report errors if their arguments are wrong + a wrapper that decides what to do with the errors But there's another problem - in a sense we've provisioned the program to detect a specific class of errors, and we call lager to report these if they happen. What happens if a totally unexpected error creeps in one where we do not throw an error?? send_update_v3(Arg1) -> try send_update_v3_happy(Arg1) of X -> X catch throw:{Type,Val} -> lager_error(Type, Val); error:Why -> io:format("Unprovisioned error :: ~p~n",[Why]) end. send_update_v3_happy(Arg1) -> {Val1, Val2} = do_this_v3(Arg1), {Val3, Val4} = do_that_v3(Arg1), Val2+Val3. do_this_v3(a) -> {1,2}; do_this_v3(b) -> {X,Y,Z} = do_that_v3(b), {X+Y,Z}; do_this_v3(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). do_that_v3(a) ->{10,20}; do_that_v3(b) ->{15,12}; do_that_v3(X) -> throw({"ERROR",{v3,bad_arg,X}}). Let's run this code: 1> demo:send_update_v3(b). Unprovisioned error :: {badmatch,{15,12}} ok 2> demo:send_update_v3(a). 12 3> demo:send_update_v3(c). Error:"EMERGENCY" {v1,bad_arg,c} Now our program has two type of errors - errors we write code to handle (they go the error logger) and - totally unexpected errors BUT in the wrapper that calls the happy case we can distinguish the two. Again this is good practice. BUT we can do even better :-) Errors occur in processes - if we spawn_link a process then the exception that is caught in the 'try catch end' is propagated to the link set of the process. This means we can handle the error in a *remote* process. As you can see, you can start with a very simple program (the happy case, no error handling) and refine it by adding exit(Why) or throw(Why) statements to the code. How far you go down this path depends upon the program and the target audience. You can read more about error handling here: http://learnyousomeerlang.com/errors-and-exceptions The complete program is here: -module(demo). -compile(export_all). send_update(Arg1) -> {Val1, Val2} = do_this(Arg1), {Val3, Val4} = do_that(Arg1), Val2+Val3. do_this(a) -> {1,2}; do_this(b) -> {3,4}. do_that(a) ->{10,20}; do_that(b) ->{15,12}. %% vsn with exit send_update_v1(Arg1) -> {Val1, Val2} = do_this_v1(Arg1), {Val3, Val4} = do_that_v1(Arg1), Val2+Val3. do_this_v1(a) -> {1,2}; do_this_v1(b) -> {3,4}; do_this_v1(X) -> exit({do_this_v1,bad_arg,X}). do_that_v1(a) ->{10,20}; do_that_v1(b) ->{15,12}; do_that_v1(X) -> exit({do_that_v1,bad_arg,X}). %% version 2 send_update_v2(Arg1) -> try send_update_v2_happy(Arg1) of X -> X catch throw:{Type,Val} -> lager_error(Type, Val) end. send_update_v2_happy(Arg1) -> {Val1, Val2} = do_this_v2(Arg1), {Val3, Val4} = do_that_v2(Arg1), Val2+Val3. do_this_v2(a) -> {1,2}; do_this_v2(b) -> {3,4}; do_this_v2(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). do_that_v2(a) ->{10,20}; do_that_v2(b) ->{15,12}; do_that_v2(X) -> throw({"ERROR",{v2,bad_arg,X}}). lager_error(Type, Val) -> io:format("Error:~p ~p~n",[Type,Val]). %% version 3 send_update_v3(Arg1) -> try send_update_v3_happy(Arg1) of X -> X catch throw:{Type,Val} -> lager_error(Type, Val); error:Why -> io:format("Unprovisioned error :: ~p~n",[Why]) end. send_update_v3_happy(Arg1) -> {Val1, Val2} = do_this_v3(Arg1), {Val3, Val4} = do_that_v3(Arg1), Val2+Val3. do_this_v3(a) -> {1,2}; do_this_v3(b) -> {X,Y,Z} = do_that_v3(b), {X+Y,Z}; do_this_v3(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). do_that_v3(a) ->{10,20}; do_that_v3(b) ->{15,12}; do_that_v3(X) -> throw({"ERROR",{v3,bad_arg,X}}). Cheers /Joe On Mon, Sep 25, 2017 at 3:25 PM, code wiget wrote: > Hello everyone, > > As I get further into Erlang, I am starting to realize that some of my functions have been getting pretty ugly with nested case statements. For example, I had a nested case statement that looked something like this: > > Send_update(Arg1) -> > case do this(Arg1) of > {ok, [Val1, Val2]} -> > case do_that(Val1, Val2) of > {ok, [Val3, Val4]} -> > case do_this2(?) of > ?. > > It continued into this for another few functions, you get the picture - its ugly, and it is hard to read. > > So I went and I converted it to a top level function that would then call lower level functions like so: > > > > send_update(Arg1) -> > case ... of > {ok, [Val1, Val2]} -> > send_update(check_indices, {Arg1, Val1, Val2}); > Else -> > lager:error("ERROR: ..") > end. > send_update(check_indices, {Arg1, Arg2, Arg3}) -> > case check_indices(Arg2, Arg3)of > true -> > send_update(get_values, {Arg1, Arg3}); > false -> > lager:error("EMERGENCY: ?.") > end; > send_update(get_values, {Arg1, Arg2}) -> > ... > case ... of > {ok, [Val1, Val2, VAl3]} -> > send_update(send_value, {Arg1, Val1, Val2, Val3}); > Error -> > lager:error("ERROR: ?") > end; > send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> > ? > Do_something(Args), > ok. > > > Now that I look at it though, both don?t look right. They don?t look like something I would write in any other language where I would just have if?s and else?s. > > Is this the proper way to write Erlang? I know everyone has their own style, but I assume there is some accepted form of writing functional programs with deep nests. > > Thank you for your advice! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Mon Sep 25 18:02:39 2017 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 25 Sep 2017 09:02:39 -0700 Subject: [erlang-questions] question re. message delivery In-Reply-To: <20170925065309.GA41893@erix.ericsson.se> References: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> <2921926.TZIjoQhpI3@changa> <3d05c826-4f76-6ae1-1ca1-a0762148c0c2@meetinghouse.net> <20170925065309.GA41893@erix.ericsson.se> Message-ID: <7ed9c6fa-8346-71b1-bff8-d232e0a9c1cf@meetinghouse.net> On 9/24/17 11:53 PM, Raimo Niskanen wrote: > On Sun, Sep 24, 2017 at 11:24:31PM -0700, Miles Fidelman wrote: >> See below.... >> >> >> On 9/24/17 6:10 PM, zxq9 wrote: >>> On 2017?09?24? ??? 16:50:45 Miles Fidelman wrote: >>>> Folks, >>>> >>>> I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a >>>> question that's been nagging me. >>>> >>>> As I understand it, message delivery is not guaranteed, but message >>>> order IS. So how, exactly does that work? What's the underlying >>>> mechanism that imposes sequencing, but allows messages to get lost? >>>> (Particularly across a network.) What are the various scenarios at play? >>> This is sort of backwards. >>> >>> Message delivery is guaranteed, assuming the process you are sending a >>> message to exists and is available, BUT from the perspective of the >>> sender there is no way to tell whether the receiver actually got it, >>> has crashed, disappeared, fell into a network blackhole, or whatever. >>> Monitoring can tell you whether the process you are trying to reach >>> is available right at that moment, but that's it. >>> >>> The point is, though, that whether the receiver is unreachable, has >>> crashed, got the message and did its work but was unable to report >>> back about it, or whatever -- its all the same reality from the >>> perspective of the sender. "Unavailable" means "unavailable", not matter >>> what the cause -- because the cause cannot be determined from the >>> perspective of the sender. You can only know this with an out of >>> context check of some sort, and that is basically the role the runtime >>> plays for you with regard to monitors and links. >>> >>> The OTP synchronous "call" mechanism is actually a complex procedure >>> built from asynchronous messages, unique reference tags, and monitors. >>> >> Note that I didn't ask about the synchronous calls, I asked about raw >> interprocess messages. >> >>> What IS guaranteed is the ordering of messages *relative to two processes* >>> >>> If A sends B the messages 1, 2 and 3 in that order, they will certainly >>> arrive in that order (assuming they arrive at all -- meaning that B is >>> available from the perspective of A). >> But that's the question.? Particularly when sent via network, 1, 2, 3 >> may be sent in that order, but, at the protocol level, they may not >> arrive in that order. > What protocol level? > > Erlang distribution has to use or implement a reliable protocol. Today > TCP, but anything is possible. Note that this protocol is between two > nodes, both containing many processes. But the emulator relies on the > transport protocol being reliable. No.? It doesn't.? It could simply send UDP packets.? I'm asking about implementation details.? In Joe's thesis, he says that the behavior is a "design choice."? I'm asking about the implementation details.? How does BEAM actually handle message delivery - locally, via network? >> With a reliable transport protocol - say TCP - if the message-containing >> packets arrived as 1, 3, 2, the protocol engine would wait for 2 to >> arrive and deliver 1,2,3 in that order.? If It received 1 & 3, but 2 got >> lost, it would request a re-transmit, wait for it to arrive, and again, >> deliver in that order. >> >> But the implication of Erlang's stated rules is that an unreliable >> transport protocol is being used, if you send 1, 2, 3, and what arrives > What? What is stated? From Joe Armstrong's Thesis: "Message passing is assumed to be unreliable with no guarantee of delivery." "Since we made no assumptions about reliable message passing, and must write our application so that it works in the presence of unreliable message passing it should indeed work in the presence of message passing errors. The initial ecort involved will reward us when we try to scale up our systems." "2. Message passing between a pair of processes is assumed to be ordered meaning that if a sequence of messages is sent and received between any pair of processes then the messages will be received in the same order they were sent." "Note that point two is a design decision, and does not reflect any under- lying semantics in the network used to transmit messages. The underlying network might reorder the messages, but between any pair of processes these messages can be buffered, and re-assembled into the correct order before delivery. This assumption makes programming message passing applications much easier than if we had to always allow for out of order messages." --- I read this as saying, messages will be delivered in order, but some may be missing. I'm really interested in this design decision, and how it's implemented.? (I'm also interested in the logic of why it's easier to program around missing messages than out-of-order messages.) Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Sep 25 18:05:29 2017 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 25 Sep 2017 18:05:29 +0200 Subject: [erlang-questions] Nested Case Statements v.s. multiple functions In-Reply-To: References: Message-ID: This thread has had some pretty long replies. To summarize: - code the happy path (ie no nested cases, just what you expect to happen) - add exit(Why) code at points in your code where you realise something has gone wrong This is what I call "let it crash" Then - if you need it - processes the error *somewhere else* ie NOT at the place where it occurred. Erlang is a concurrent language - we have many processes running it's not a biggie if a few of them crash. In a sequential language with one thread it's a big deal if your one process crashes - so sequential programs spend all their effort checking things and trying not to crash. We do the opposite :-) /Joe On Mon, Sep 25, 2017 at 5:57 PM, Joe Armstrong wrote: > Good question. > > This is example of abnormal termination. > > The Erlang 'way' is to write the happy path and, not write twisty little > passages full of error correcting code. > > I'll assume send_update has a single argument 'a' or 'b' - and is as follows: > > send_update(Arg1) -> > {Val1, Val2} = do_this(Arg1), > {Val3, Val4} = do_that(Arg1), > Val2+Val3. > > do_this(a) -> {1,2}; > do_this(b) -> {3,4}. > > do_that(a) ->{10,20}; > do_that(b) ->{15,12}. > > This is version 1 - no decent errors. Here's what we see in the shell: > > 1> 2> demo:send_update(a). > 12 > 3> demo:send_update(c). > ** exception error: no function clause matching demo:do_this(c) > (demo.erl, line 9) > in function demo:send_update/1 (demo.erl, line 5) > > The error message is not very good - but is enough to get you started in finding > the error. We can get some information like this: > > 4> (catch demo:send_update(c)). > {'EXIT',{function_clause,[{demo,do_this, > [c], > [{file,"demo.erl"},{line,9}]}, > {demo,send_update,1,[{file,"demo.erl"},{line,5}]}, > ... > > So you can see a stack trace. > > But we want better than this. So now I'll make a few small changes. > > > send_update_v1(Arg1) -> > {Val1, Val2} = do_this_v1(Arg1), > {Val3, Val4} = do_that_v1(Arg1), > Val2+Val3. > > do_this_v1(a) -> {1,2}; > do_this_v1(b) -> {3,4}; > do_this_v1(X) -> exit({do_this_v1,bad_arg,X}). > > do_that_v1(a) ->{10,20}; > do_that_v1(b) ->{15,12}; > do_that_v1(X) -> exit({do_that_v1,bad_arg,X}). > > > We can now call this as follows: > > 1> demo:send_update_v1(a). > 12 > > 8> demo:send_update_v1(z). > ** exception exit: {do_this_v1,bad_arg,z} > in function demo:do_this_v1/1 (demo.erl, line 25) > in call from demo:send_update_v1/1 (demo.erl, line 19) > > 9> (catch demo:send_update_v1(z)). > {'EXIT',{do_this_v1,bad_arg,z}} > > Now let's get creative: > > send_update_v2(Arg1) -> > try send_update_v2_happy(Arg1) of > X -> X > catch > throw:{Type,Val} -> > lager_error(Type, Val) > end. > > send_update_v2_happy(Arg1) -> > {Val1, Val2} = do_this_v2(Arg1), > {Val3, Val4} = do_that_v2(Arg1), > Val2+Val3. > > do_this_v2(a) -> {1,2}; > do_this_v2(b) -> {3,4}; > do_this_v2(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). > > do_that_v2(a) ->{10,20}; > do_that_v2(b) ->{15,12}; > do_that_v2(X) -> throw({"ERROR",{v2,bad_arg,X}}). > > lager_error(Type, Val) -> > io:format("Error:~p ~p~n",[Type,Val]). > > Here we retain the happy path code - but add a wrapper that catches > the error and reports it. > > 1> demo:send_update_v2(a). > 12 > 2> demo:send_update_v2(b). > 19 > 3> demo:send_update_v2(c). > Error:"EMERGENCY" {v1,bad_arg,c} > > This code is beginning to look nice - but we're not their yet ... > > Lets' stop and think. We've refactored the code into three > parts: > > + the happy path > + the subroutines that report errors if their arguments are wrong > + a wrapper that decides what to do with the errors > > But there's another problem - in a sense we've provisioned the program > to detect a specific class of errors, and we call lager to report these if > they happen. > > What happens if a totally unexpected error creeps in one where we do not > throw an error?? > > send_update_v3(Arg1) -> > try send_update_v3_happy(Arg1) of > X -> X > catch > throw:{Type,Val} -> > lager_error(Type, Val); > error:Why -> > io:format("Unprovisioned error :: ~p~n",[Why]) > end. > > send_update_v3_happy(Arg1) -> > {Val1, Val2} = do_this_v3(Arg1), > {Val3, Val4} = do_that_v3(Arg1), > Val2+Val3. > > do_this_v3(a) -> {1,2}; > do_this_v3(b) -> {X,Y,Z} = do_that_v3(b), > {X+Y,Z}; > do_this_v3(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). > > do_that_v3(a) ->{10,20}; > do_that_v3(b) ->{15,12}; > do_that_v3(X) -> throw({"ERROR",{v3,bad_arg,X}}). > > Let's run this code: > > 1> demo:send_update_v3(b). > Unprovisioned error :: {badmatch,{15,12}} > ok > 2> demo:send_update_v3(a). > 12 > 3> demo:send_update_v3(c). > Error:"EMERGENCY" {v1,bad_arg,c} > > Now our program has two type of errors > > - errors we write code to handle (they go the error logger) and > - totally unexpected errors > > BUT in the wrapper that calls the happy case we can distinguish the two. > > Again this is good practice. > > BUT we can do even better :-) > > Errors occur in processes - if we spawn_link a process then the > exception that is caught in the 'try catch end' is propagated to > the link set of the process. > > This means we can handle the error in a *remote* process. > > As you can see, you can start with a very simple program > (the happy case, no error handling) and refine it by adding > exit(Why) or throw(Why) statements to the code. How > far you go down this path depends upon the program and the > target audience. > > You can read more about error handling here: > > http://learnyousomeerlang.com/errors-and-exceptions > > The complete program is here: > > -module(demo). > -compile(export_all). > > send_update(Arg1) -> > {Val1, Val2} = do_this(Arg1), > {Val3, Val4} = do_that(Arg1), > Val2+Val3. > > do_this(a) -> {1,2}; > do_this(b) -> {3,4}. > > do_that(a) ->{10,20}; > do_that(b) ->{15,12}. > > > %% vsn with exit > > send_update_v1(Arg1) -> > {Val1, Val2} = do_this_v1(Arg1), > {Val3, Val4} = do_that_v1(Arg1), > Val2+Val3. > > do_this_v1(a) -> {1,2}; > do_this_v1(b) -> {3,4}; > do_this_v1(X) -> exit({do_this_v1,bad_arg,X}). > > do_that_v1(a) ->{10,20}; > do_that_v1(b) ->{15,12}; > do_that_v1(X) -> exit({do_that_v1,bad_arg,X}). > > %% version 2 > > send_update_v2(Arg1) -> > try send_update_v2_happy(Arg1) of > X -> X > catch > throw:{Type,Val} -> > lager_error(Type, Val) > end. > > send_update_v2_happy(Arg1) -> > {Val1, Val2} = do_this_v2(Arg1), > {Val3, Val4} = do_that_v2(Arg1), > Val2+Val3. > > do_this_v2(a) -> {1,2}; > do_this_v2(b) -> {3,4}; > do_this_v2(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). > > do_that_v2(a) ->{10,20}; > do_that_v2(b) ->{15,12}; > do_that_v2(X) -> throw({"ERROR",{v2,bad_arg,X}}). > > lager_error(Type, Val) -> > io:format("Error:~p ~p~n",[Type,Val]). > > %% version 3 > > send_update_v3(Arg1) -> > try send_update_v3_happy(Arg1) of > X -> X > catch > throw:{Type,Val} -> > lager_error(Type, Val); > error:Why -> > io:format("Unprovisioned error :: ~p~n",[Why]) > end. > > send_update_v3_happy(Arg1) -> > {Val1, Val2} = do_this_v3(Arg1), > {Val3, Val4} = do_that_v3(Arg1), > Val2+Val3. > > do_this_v3(a) -> {1,2}; > do_this_v3(b) -> {X,Y,Z} = do_that_v3(b), > {X+Y,Z}; > do_this_v3(X) -> throw({"EMERGENCY",{v1,bad_arg,X}}). > > do_that_v3(a) ->{10,20}; > do_that_v3(b) ->{15,12}; > do_that_v3(X) -> throw({"ERROR",{v3,bad_arg,X}}). > > Cheers > > /Joe > > On Mon, Sep 25, 2017 at 3:25 PM, code wiget wrote: >> Hello everyone, >> >> As I get further into Erlang, I am starting to realize that some of my functions have been getting pretty ugly with nested case statements. For example, I had a nested case statement that looked something like this: >> >> Send_update(Arg1) -> >> case do this(Arg1) of >> {ok, [Val1, Val2]} -> >> case do_that(Val1, Val2) of >> {ok, [Val3, Val4]} -> >> case do_this2(?) of >> ?. >> >> It continued into this for another few functions, you get the picture - its ugly, and it is hard to read. >> >> So I went and I converted it to a top level function that would then call lower level functions like so: >> >> >> >> send_update(Arg1) -> >> case ... of >> {ok, [Val1, Val2]} -> >> send_update(check_indices, {Arg1, Val1, Val2}); >> Else -> >> lager:error("ERROR: ..") >> end. >> send_update(check_indices, {Arg1, Arg2, Arg3}) -> >> case check_indices(Arg2, Arg3)of >> true -> >> send_update(get_values, {Arg1, Arg3}); >> false -> >> lager:error("EMERGENCY: ?.") >> end; >> send_update(get_values, {Arg1, Arg2}) -> >> ... >> case ... of >> {ok, [Val1, Val2, VAl3]} -> >> send_update(send_value, {Arg1, Val1, Val2, Val3}); >> Error -> >> lager:error("ERROR: ?") >> end; >> send_update(send_value, {Arg1, Arg2, Arg3, Arg4}) -> >> ? >> Do_something(Args), >> ok. >> >> >> Now that I look at it though, both don?t look right. They don?t look like something I would write in any other language where I would just have if?s and else?s. >> >> Is this the proper way to write Erlang? I know everyone has their own style, but I assume there is some accepted form of writing functional programs with deep nests. >> >> Thank you for your advice! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Mon Sep 25 18:29:42 2017 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 25 Sep 2017 18:29:42 +0200 Subject: [erlang-questions] question re. message delivery In-Reply-To: <7ed9c6fa-8346-71b1-bff8-d232e0a9c1cf@meetinghouse.net> References: <78543d28-5246-000a-e41a-eaebb2545ebf@meetinghouse.net> <2921926.TZIjoQhpI3@changa> <3d05c826-4f76-6ae1-1ca1-a0762148c0c2@meetinghouse.net> <20170925065309.GA41893@erix.ericsson.se> <7ed9c6fa-8346-71b1-bff8-d232e0a9c1cf@meetinghouse.net> Message-ID: I think I should add - that "in the absence of errors message passing is ordered" I think we should deliberately separate errors which are detected by sockets closing, pings timing out etc. from the message passing behaviour. Imagine a client sends numbered messages 1,2,3,4,5 in order to a server. If no errors have been observed and the server receives message 5 we can assume that messages 1..4 have also been received in order. The client will never know how many messages the server has received unless the server tells it. This is what the system is supposed to do - how it does it will depend on time (ie how it does it today will differ from how it was done 10 years ago) In the old days there was just a single socket between a client and server - so we assumed that what you wrote to the socket got reead in the order it was send. This true at the application level packets might get fragmented but they are not reordered. (In lower levels of the TCP stack, out of order packages are reordered and missing packets resent). So In the absence of errors (meaning the TCP socket had not closed) what comes out of a socket has the same ordering as what went in. (Note: not true for UDP) I guess we also make assumptions that the underlying layers are also reliable - So Erlang messaging should be reliable if TCP is reliable. The subject is complicated by a load of theorems saying that various things are mathematically impossible (distributed consensus, exactly once delivery if messages) - Add multicore processors and multiple sockets between nodes and the situation because a lot more complicated. Cheers /Joe On Mon, Sep 25, 2017 at 6:02 PM, Miles Fidelman wrote: > On 9/24/17 11:53 PM, Raimo Niskanen wrote: > > On Sun, Sep 24, 2017 at 11:24:31PM -0700, Miles Fidelman wrote: > > See below.... > > > On 9/24/17 6:10 PM, zxq9 wrote: > > On 2017?09?24? ??? 16:50:45 Miles Fidelman wrote: > > Folks, > > I've just been re-reading Joe Armstrong's thesis, and I'm reminded of a > question that's been nagging me. > > As I understand it, message delivery is not guaranteed, but message > order IS. So how, exactly does that work? What's the underlying > mechanism that imposes sequencing, but allows messages to get lost? > (Particularly across a network.) What are the various scenarios at play? > > This is sort of backwards. > > Message delivery is guaranteed, assuming the process you are sending a > message to exists and is available, BUT from the perspective of the > sender there is no way to tell whether the receiver actually got it, > has crashed, disappeared, fell into a network blackhole, or whatever. > Monitoring can tell you whether the process you are trying to reach > is available right at that moment, but that's it. > > The point is, though, that whether the receiver is unreachable, has > crashed, got the message and did its work but was unable to report > back about it, or whatever -- its all the same reality from the > perspective of the sender. "Unavailable" means "unavailable", not matter > what the cause -- because the cause cannot be determined from the > perspective of the sender. You can only know this with an out of > context check of some sort, and that is basically the role the runtime > plays for you with regard to monitors and links. > > The OTP synchronous "call" mechanism is actually a complex procedure > built from asynchronous messages, unique reference tags, and monitors. > > Note that I didn't ask about the synchronous calls, I asked about raw > interprocess messages. > > What IS guaranteed is the ordering of messages *relative to two processes* > > If A sends B the messages 1, 2 and 3 in that order, they will certainly > arrive in that order (assuming they arrive at all -- meaning that B is > available from the perspective of A). > > But that's the question. Particularly when sent via network, 1, 2, 3 > may be sent in that order, but, at the protocol level, they may not > arrive in that order. > > What protocol level? > > Erlang distribution has to use or implement a reliable protocol. Today > TCP, but anything is possible. Note that this protocol is between two > nodes, both containing many processes. But the emulator relies on the > transport protocol being reliable. > > > No. It doesn't. It could simply send UDP packets. I'm asking about > implementation details. In Joe's thesis, he says that the behavior is a > "design choice." I'm asking about the implementation details. How does > BEAM actually handle message delivery - locally, via network? > > With a reliable transport protocol - say TCP - if the message-containing > packets arrived as 1, 3, 2, the protocol engine would wait for 2 to > arrive and deliver 1,2,3 in that order. If It received 1 & 3, but 2 got > lost, it would request a re-transmit, wait for it to arrive, and again, > deliver in that order. > > But the implication of Erlang's stated rules is that an unreliable > transport protocol is being used, if you send 1, 2, 3, and what arrives > > What? What is stated? > > From Joe Armstrong's Thesis: > > "Message passing is assumed to be unreliable with no guarantee of delivery." > > "Since we made no assumptions about reliable message passing, and must write > our application so that it works in the presence of unreliable message > passing it should indeed work in the presence of message passing errors. The > initial ecort involved will reward us when we try to scale up our systems." > > "2. Message passing between a pair of processes is assumed to be ordered > meaning that if a sequence of messages is sent and received between any pair > of processes then the messages will be received in the same order they were > sent." > > "Note that point two is a design decision, and does not reflect any under- > lying semantics in the network used to transmit messages. The underlying > network might reorder the messages, but between any pair of processes these > messages can be buffered, and re-assembled into the correct order before > delivery. This assumption makes programming message passing applications > much easier than if we had to always allow for out of order messages." > > --- > I read this as saying, messages will be delivered in order, but some may be > missing. > > I'm really interested in this design decision, and how it's implemented. > (I'm also interested in the logic of why it's easier to program around > missing messages than out-of-order messages.) > > Miles > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From davidnwelton@REDACTED Mon Sep 25 19:43:22 2017 From: davidnwelton@REDACTED (David Welton) Date: Mon, 25 Sep 2017 10:43:22 -0700 Subject: [erlang-questions] relx relup, appups and dependencies In-Reply-To: References: Message-ID: Hi, So I started looking at rebar and relx to see if there was anything to be done, and it looks like rebar3 has a variety of dependencies that can be hooked into. The trick would be to * Have a local directory with appups for the dependencies, so that they can be managed as part of my project. * Hook that copies those into the relevant directories in _rel when it's time to make a relup. This seems kind of hacky, and I feel like maybe I'm doing something wrong, but not sure what. Maybe no one actually uses relups? -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From zxq9@REDACTED Mon Sep 25 20:02:25 2017 From: zxq9@REDACTED (zxq9) Date: Tue, 26 Sep 2017 03:02:25 +0900 Subject: [erlang-questions] relx relup, appups and dependencies In-Reply-To: References: Message-ID: <1557707.SVULAB17Ky@changa> On 2017?09?25? ??? 10:43:22 David Welton wrote: > This seems kind of hacky, and I feel like maybe I'm doing something > wrong, but not sure what. Maybe no one actually uses relups? Tristan and Fred would have to verify this, but from what I've seen no, people who use rebar and erlang.mk do not do relups. In my experience (having only seen three projects that were serious about using relups instead of restarts) projects that do relups use systools and relup. And the tooling isn't actually bad, it is just more complex because very few conveniences have been built into them (partly because "reasonable default" is a lot harder to pin down, and partly because, I think, most people who figure out reltool once have solved it for the life of their project, and forget that it was a PITA to straighten out initially). -Craig From bchesneau@REDACTED Mon Sep 25 21:13:47 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 25 Sep 2017 21:13:47 +0200 Subject: [erlang-questions] Customizing Beam Build In-Reply-To: References: Message-ID: <2A2952F5-FC7D-4A95-87A8-39A53B70A26B@gmail.com> > On 24 Sep 2017, at 10:25, Sargun Dhillon wrote: > > I've customized my OTP build to have a custom Erlang build. The reason > I've done this to begin the process of building a fully-self contained > Erlang program. One of the biggest benefits of the infrastructure in > Go is that they're fully self-contained statically linked binaries. In > Erlang, although you can statically link beam itself, unfortunately, > the releases themselves are messy, in that you have a lot of files and > getting the execution environment right isn't always trivial. > > Eventually, the purpose is to build an init system -- something that > will be primarily used by sysadmin-types. These need to be statically > linked, and often times, the file system isn't fully-functioning when > they're starting. > > I've done two things: > > 1) I've modified the set of preloaded files in the VM. It would be > really nice if there was a way to do this without forking OTP. I > didn't add any existing beam files, but I just changed around some of > the file, and code loading code to my need. +1 > > 2) I've modified the final, linked product to include custom objects > in the final ELF binary. What files did you changed for it? > > This is kind of terrible, as it requires I maintain a fork of OTP. Is this something public? > Does anyone have any recommendations for solving this problem another > way? > > My proposal would be to output a beam.a file with the requisite > objects to build a BEAM installation. It would be valuable if I could > statically link against this binary, and own main, if it had a > separate entrypoint that I could then call into. I've toyed around > with this, and for the unix build, it seems possible. Is there any > reason why Erlang isn't distributed this way already? > > If there was a public API for extending the loader, or adding > preloaded modules via sections, I imagine that solving (1) would be > easy as well. > > Is anyone open to accepting this upstream if the work is done? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Mon Sep 25 21:14:56 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 25 Sep 2017 21:14:56 +0200 Subject: [erlang-questions] slave feature In-Reply-To: References: <59C547BF.80709@gmail.com> Message-ID: <12D4F6E0-9628-4296-B4DC-57196C0C2DAC@gmail.com> > On 22 Sep 2017, at 19:35, Fisher, Paul wrote: > > I should have clarified our use... we have a version of the slave module that is exactly the same except for that feature. We want to have each node handle its own IO directly to the system. > I would be interrested by such code! > -- > paul > Fellow, Engineering > From: erlang-questions-bounces@REDACTED > on behalf of Michael Truog > > Sent: Friday, September 22, 2017 12:26:23 PM > To: Benoit Chesneau; erlang-questions@REDACTED > Subject: Re: [erlang-questions] slave feature > > On 09/22/2017 08:00 AM, Benoit Chesneau wrote: > > i?m looking for some usage examples of the slave feature apart for its use in tests? what was the original purpose? > > > > > > You should be aware that nodes started with slave try to make the slave node use the master's filesystem automatically, though this doesn't work in all places (as shown in the bug https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.erlang.org_browse_ERL-2D446&d=DwIFEA&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=ZYaInvQKkF5t9wYb-bJCCzIq2vg5EMJ-yPQV5oGSzhE&s=6hwbTzX8VVRetMB-wwsC4dH_o0qsG_TWFYjhbCw7Hbs&e= which affects tsung). This behavior doesn't seem to be well documented (nor the -master command line argument used on the erl executable) but it appears to be the main deciding factor for whether slave is usable for a given situation (in addition to the linking that occurs between the master node, making the nodes not exist independently). > > Best Regards, > Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwIFEA&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=ZYaInvQKkF5t9wYb-bJCCzIq2vg5EMJ-yPQV5oGSzhE&s=vNTGFf8Z6SLM093TNOTb5lcV2cYPKArsfp5_0u6JpuQ&e= > Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Mon Sep 25 21:16:14 2017 From: list1@REDACTED (Grzegorz Junka) Date: Mon, 25 Sep 2017 19:16:14 +0000 Subject: [erlang-questions] Tuples referencing each other problem In-Reply-To: References: <95dab94a-b5e9-b783-6648-209f490c731c@gjunka.com> <3f5ae83a-20cd-be32-7d33-fc720c3f7166@cs.otago.ac.nz> <0bc4e2f2-b1df-b753-6b20-d1144768a89a@cs.otago.ac.nz> <035f6141-8d95-0398-cc80-2f3941fb6ada@gjunka.com> <90c29bdb-d069-b4f2-f83d-634047a8946b@cs.otago.ac.nz> Message-ID: On 25/09/2017 15:48, Jesper Louis Andersen wrote: > On Mon, Sep 25, 2017 at 8:35 AM Grzegorz Junka > wrote: > > Fair enough, I missed that you can set ordered_set in ETS, but > nevertheless it won't work with maps. However, even if I use ETS > the Key is still stored twice, isn't? (once as the key for > ordered_set, once as the value when the Id is the key). > > > > It depends. If it is a large binary (larger than 64 characters) it > will go on the binary heap once unless you form it again and again in > your code. Otherwise it will take up the double amount of space. I don't think it's that easy. If ordered_set is using B+Tree then the key would be split into segments to annotate nodes of the tree. But the value would be left untouched. Binaries could potentially be reused by referencing segments of the same binary. So if binaries would be reused or not depends on the actual implementation of ordered_set. Also, with ETS the data must be copied between the database and the process. It's possible that the VM will not actually copy the binary but instead will create another reference to it in the process. But this is all valid only for binaries. For any other Erlang term what I wrote earlier would hold. I am not sure I want to rely on a solution with so many unknowns. > > We still don't know what the underlying problem statement is. This > makes it harder to solve because whenever we come up with a solution, > a new constraint is added and we have to adapt. I did state all the constraints I have: In short, I have keys, which may be any Erlang terms, and numerical Ids assigned to those terms. Keys must be sorted. Numerical Ids are increasing monotonically. Then the following lookups should be efficient: 1. Having a key quickly search for its numeric Id 2. Having a numeric Id quickly get back the key Also the following conditions should be met: 3. The same key should be always assigned the same numeric Id 4. One key has always one numeric Id (and vice versa), so they are always added or removed together Since keys can be of any length I don't want to store them more than once. > > One advantage of using something like a gb_trees for the above is that > you only have the key once in the process heap and everything else > will be pointers. It is also possible to use the above scheme with > gb_trees. But then again, with ETS you can do key lookup from any > process, whereas it has to factor through the tree owner with gb_trees. gb_tree would not prevent from having to store the key twice (once as the key for gb_tree and once as the value). Not sure why you mention gb_tree here? > > There is also the risk your problem is entirely too large to fit in > memory at all, but we don't yet know the size of your N and how large > of a machine you are willing to scale to, so it is hard to do any > napkin math on the size here. It's part of a bigger problem and I don't want to be getting into describing it in its entirety. Essentially I want to design a sorted index of terms that can hold billions of entries. For that I want it to be distributed among multiple processes, each process holding a part of the whole index. For now I am just looking into a data structure suitable for implementing one of those processes. ETS would be a valid solution if not that it's opaque - I don't have control over how the data is stored. Let's say that I am investigating alternatives. > > In short, we need more info if we are to come up with a better solution :) That's the whole point. How would you implement an RDF/triple database in Erlang? :) GrzegorzJ -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Mon Sep 25 21:32:53 2017 From: t@REDACTED (Tristan Sloughter) Date: Mon, 25 Sep 2017 12:32:53 -0700 Subject: [erlang-questions] Customizing Beam Build In-Reply-To: References: Message-ID: <1506367973.3716795.1117827296.450FD0F1@webmail.messagingengine.com> I like the idea of easily built static builds but for dealing with releases being messy (if I'm understanding what you mean) is something that may be solved simply with existing tools like makeself: http://makeself.io/ -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Sun, Sep 24, 2017, at 01:25 AM, Sargun Dhillon wrote: > I've customized my OTP build to have a custom Erlang build. The reason > I've done this to begin the process of building a fully-self contained > Erlang program. One of the biggest benefits of the infrastructure in > Go is that they're fully self-contained statically linked binaries. In > Erlang, although you can statically link beam itself, unfortunately, > the releases themselves are messy, in that you have a lot of files and > getting the execution environment right isn't always trivial. > > Eventually, the purpose is to build an init system -- something that > will be primarily used by sysadmin-types. These need to be statically > linked, and often times, the file system isn't fully-functioning when > they're starting. > > I've done two things: > > 1) I've modified the set of preloaded files in the VM. It would be > really nice if there was a way to do this without forking OTP. I > didn't add any existing beam files, but I just changed around some of > the file, and code loading code to my need. > > 2) I've modified the final, linked product to include custom objects > in the final ELF binary. > > This is kind of terrible, as it requires I maintain a fork of OTP. > Does anyone have any recommendations for solving this problem another > way? > > My proposal would be to output a beam.a file with the requisite > objects to build a BEAM installation. It would be valuable if I could > statically link against this binary, and own main, if it had a > separate entrypoint that I could then call into. I've toyed around > with this, and for the unix build, it seems possible. Is there any > reason why Erlang isn't distributed this way already? > > If there was a public API for extending the loader, or adding > preloaded modules via sections, I imagine that solving (1) would be > easy as well. > > Is anyone open to accepting this upstream if the work is done? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Mon Sep 25 21:38:30 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 25 Sep 2017 22:38:30 +0300 Subject: [erlang-questions] Customizing Beam Build In-Reply-To: <1506367973.3716795.1117827296.450FD0F1@webmail.messagingengine.com> References: <1506367973.3716795.1117827296.450FD0F1@webmail.messagingengine.com> Message-ID: > On 25 Sep 2017, at 22.32, Tristan Sloughter wrote: > > may be solved simply with existing tools like makeself +1. There are not issue to package Erlang release to tarball and then to self-extractable archive. This is what I am doing on daily basic. I am trying to understand why community rejects this approach and targets some complex tooling? Is this because of Windows platform support or escripts? Or something else? - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Mon Sep 25 21:59:51 2017 From: t@REDACTED (Tristan Sloughter) Date: Mon, 25 Sep 2017 12:59:51 -0700 Subject: [erlang-questions] Customizing Beam Build In-Reply-To: References: <1506367973.3716795.1117827296.450FD0F1@webmail.messagingengine.com> Message-ID: <1506369591.3723073.1117860376.64BC7C1B@webmail.messagingengine.com> > There are not issue to package Erlang release to tarball and then to > self-extractable archive. This is what I am doing on daily basic. I am > trying to understand why community rejects this approach and targets > some complex tooling? Is this because of Windows platform support or > escripts? Or something else? escripts are a separate use case. And not working on Windows is certainly limiting but I doubt it is why the pattern isn't used much. Personally I just use containers these days, so no real need for building self-extractable archives, and this is probably the case of many now. Plus the fact that unpacking a tarball and running `bin/