From technion@REDACTED Wed Jun 1 00:18:40 2016 From: technion@REDACTED (Technion) Date: Tue, 31 May 2016 22:18:40 +0000 Subject: [erlang-questions] bcrypt message queues In-Reply-To: <20160531093125.GA28976@jarowit.net> References: <574D51A5.8000106@ericsson.com>,<20160531093125.GA28976@jarowit.net> Message-ID: Is there any particularly great documentation on how to ensure a NIF doesn't fall into these traps? I'm going to have a need for Argon2 at some point, at the moment making such a NIF looks well over my head. ________________________________________ From: erlang-questions-bounces@REDACTED on behalf of Stanislaw Klekot Sent: Tuesday, 31 May 2016 7:31:25 PM To: Sverker Eriksson Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] bcrypt message queues On Tue, May 31, 2016 at 10:56:05AM +0200, Sverker Eriksson wrote: > How heavy duty is this bcrypt? > Does it take milliseconds to hash one little password? Password hashes are *designed* to take long time to compute (e.g. a semi-traditional MD5-based crypt() is MD5 hash applied 1000 times, each time to the result of previous computation). This is to make precomputation attacks so much more costly. Having that said, https://github.com/chef/erlang-bcrypt has its internals sub-par, as it runs two processes (one for NIF and one for port driver; why port driver? there's no state to maintain between calls as far as I'm aware), and NIF is always called in one. This makes a great example of an unnecessary bottleneck. -- Stanislaw Klekot _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From technion@REDACTED Wed Jun 1 12:49:09 2016 From: technion@REDACTED (Technion) Date: Wed, 1 Jun 2016 10:49:09 +0000 Subject: [erlang-questions] Erlang offensive paper Message-ID: Hey guys, I stumbled on this on reddit and found it to be a great read: https://drive.google.com/file/d/0Bz8Lmg2kodQiRXYwWVpGNXQtdG5hNG5GaDFFNF9UNXp4UXo4/view [https://lh6.googleusercontent.com/DjEvzXdgdUZQq0MJ7xUmtpzZ1vb5_yiehoviMLZyCJmchFy2C5rOWaBMXXLQXRHWf4wUsL7Y46g=w1200-h630-p] In Defense of Erlang.pdf drive.google.com I can't find a date on this so apologies if you've all seen it or it's years old. Does anyone know if there is anything in the works or proposed around the "If someone gets inside the network, the cookie is the only protection left" situation? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 1 13:19:14 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 1 Jun 2016 13:19:14 +0200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: Message-ID: <574EC4B2.7040009@ninenines.eu> On 06/01/2016 12:49 PM, Technion wrote: > Does anyone know if there is anything in the works or proposed around > the "If someone gets inside the network, the cookie is the only > protection left" situation? If someone gets inside the network, they can read the cookie easily. It's not a security protection, it's there to allow separating clusters. Erlang itself does not come with any safeguards; someone gets in, you're dead. Not trying to raise an alarm there, it's not necessarily a bad thing, it depends entirely on your use case. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From nathaniel@REDACTED Wed Jun 1 13:32:07 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Wed, 1 Jun 2016 07:32:07 -0400 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: Message-ID: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> > Does anyone know if there is anything in the works or proposed around the "If someone gets inside the network, the cookie is the only protection left" situation? Yes: use SSL for distribution and to talk to other services. http://erlang.org/doc/apps/ssl/ssl_distribution.html This assumes that by "inside the network" you mean past the firewall/gateway/NAT. But you could also view this as using encryption to build an inner network that just contains your Erlang nodes. Once you're inside *that* network things are still open. The author suggests that since the BEAM is an OS you might want all the access controls that a full OS offers. This would (e.g.) allow some people to launch processes and kill the process that they'd launched, but only some root user could terminate the Cowboy application. This would take an enormous amount of work and there are other ways of getting the same effect, so I can't imagine this happening. What you should do is understand that a network of Erlang nodes behaves (as much as possible) like a single node. If you don't trust a remote node, do not link with it under any circumstances. If you want to allow trusted and untrusted code to interoperate, you need to write your own communication layer for them. Finally, to get the security model of all the other languages that I'm aware of, you can disable distribution. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Eric.desCourtis@REDACTED Wed Jun 1 15:51:43 2016 From: Eric.desCourtis@REDACTED (Eric des Courtis) Date: Wed, 1 Jun 2016 09:51:43 -0400 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> References: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> Message-ID: I would be nice if BEAM could address these issues (not Erlang) so that new more secure languages could be implemented on the BEAM. I think it will be done sooner or later. The sooner the better IMO if BEAM is to remain relevant in the long term. On Wed, Jun 1, 2016 at 7:32 AM, Nathaniel Waisbrot wrote: > Does anyone know if there is anything in the works or proposed around the > "If someone gets inside the network, the cookie is the only protection > left" situation? > > > > Yes: use SSL for distribution and to talk to other services. > http://erlang.org/doc/apps/ssl/ssl_distribution.html > > This assumes that by "inside the network" you mean past the > firewall/gateway/NAT. But you could also view this as using encryption to > build an inner network that just contains your Erlang nodes. Once you're > inside *that* network things are still open. > > The author suggests that since the BEAM is an OS you might want all the > access controls that a full OS offers. This would (e.g.) allow some people > to launch processes and kill the process that they'd launched, but only > some root user could terminate the Cowboy application. This would take an > enormous amount of work and there are other ways of getting the same > effect, so I can't imagine this happening. > > What you should do is understand that a network of Erlang nodes behaves > (as much as possible) like a single node. If you don't trust a remote node, > do not link with it under any circumstances. If you want to allow trusted > and untrusted code to interoperate, you need to write your own > communication layer for them. > > > Finally, to get the security model of all the other languages that I'm > aware of, you can disable distribution. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Jun 1 16:06:56 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 01 Jun 2016 23:06:56 +0900 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> Message-ID: <5388109.NjTHRdhrIS@changa> On 2016?6?1? ??? 09:51:43 Eric des Courtis wrote: > I would be nice if BEAM could address these issues (not Erlang) so that new > more secure languages could be implemented on the BEAM. > > I think it will be done sooner or later. The sooner the better IMO if BEAM > is to remain relevant in the long term. At what point would this become like desiring to have, say, Akka actors live within a MAC scheme, or Twisted threads adhere to an access control policy? I think this is the wrong place to start looking to employ a security scheme. There are other issues raised by the paper that have nothing to do with disterl, and these are valid. But disterl cookies being a security thing instead of an identity and segregation token within a trusted environment is, imo, wrong. If the docs say "security" (I didn't check, but the author of the referenced slides says they do) that should be changed because it is in conflict with everything else written about disterl, the capabilities of the mechanism itself and the way it is used in practice. -Craig From garazdawi@REDACTED Wed Jun 1 16:10:01 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Wed, 1 Jun 2016 16:10:01 +0200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> Message-ID: On Wed, Jun 1, 2016 at 3:51 PM, Eric des Courtis < Eric.desCourtis@REDACTED> wrote: > I would be nice if BEAM could address these issues (not Erlang) so that > new more secure languages could be implemented on the BEAM. > What makes you think that you cannot do that today? The only thing I see is missing from the vm is the possibility to run different distribution drivers at the same time, but if you use the same driver for all of them, then you can definitely implement what you describe today, especially if you are willing to rewrite parts of net_kernel. It's not a small task to do, but very possible without changing the VM. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pragtob@REDACTED Wed Jun 1 08:47:09 2016 From: pragtob@REDACTED (Tobias Pfeiffer) Date: Wed, 1 Jun 2016 08:47:09 +0200 Subject: [erlang-questions] Turn off Garbage Collection for a time Message-ID: <574E84ED.7040400@gmail.com> Hi everyone, is there a way in Elixir/Erlang to turn off the Garbage collection? I've searched and what I found so far is :erlang.garbage_collect to force garbage collection.http://erlang.org/doc/man/erlang.html#garbage_collect-0 I was also made aware of :erlang.spawn_opt/2 http://erlang.org/doc/man/erlang.html#spawn_opt-2 and setting a high value for min_heap_size + setting fullsweep_after. Is there any way to turn it off completely? Why would I want to do that? I'm working on a benchmarking tool and I don't want garbage collection to mess with my measured execution times. I run them multiple times and standard deviation goes way up because usually GC triggers during one run of n. Any hints welcome + thanks Tobi -- http://www.pragtob.info/ From uniaika@REDACTED Wed Jun 1 16:00:26 2016 From: uniaika@REDACTED (Uniaika) Date: Wed, 1 Jun 2016 16:00:26 +0200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: <6A7229F3-AFD4-4FE1-9E25-0BC0B5D03631@waisbrot.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 One of the first things I did when starting to play with distributed Erlang was to set a Tinc VPN between all my nodes. It's a mesh VPN so the setup is fairly easy (and scalable), and it provides good guarantees in terms of crypto (that can be enforced when following the instructions at https://bettercrypto.org). That's basically all the security I have for inter-node communications and I'm satisfied enough. Anne-Gwenn. -----BEGIN PGP SIGNATURE----- Version: Keybase OpenPGP v2.0.53 Comment: https://keybase.io/crypto wsFcBAABCgAGBQJXTupTAAoJEAaxffTdQxbs5sUQAKyT3OaThA6PsrjUOj/G7rrQ c5++u04H5dxMPzndnm6XZv41nbAZWHdMf1tg0b82Hvy5kYvDa3a1/6nTIA8WwLO8 +Xuo9oUWsBG4CF8qJ4rX37IKt8/ZDgOvAag0if2hv3K7XLn8bhsPQdn4KzuiuzAS d8wM6dtTic9343o+nUyrQJc6qov1xmoKQSlitCc9ZggPeyi5B8KsGe/kNykdEZuD IpfXKCtBBOrbXb3nJ+JPNwm7OZ6q0eyQ/ccH8a3WliRbPVkaY7DVpVpQme4jzSs6 do/tvYraSw/lUIXirsHOp3J9kXQ8GxklsVmJwkfqguytLV9U5WFdqLAHHuPIR/FA E8GKmu/+kv1KRjyh5ttzYoa4wb/xQ6Z3T3NdPTnNJnCBMPDkPAKFW3RMZvWzrIjx 89VG47Om8olFUnpIa+Re2YzgmgRcWzhwK42vVPxich6rd1IPOF5ODj8McmZ7Gdwv jx75a7kP2+nL44/oTxNSirmu3rMYoFqFLgczPSR7UZ8YO9/OLgofzRG1/KQMegIZ eSwLe/OcYkuES07TEYAtreJuhinXuRrFykGDdOdqgrhoSuzf79D5hZDLxf2VXbL7 9hQwY60rLRAxayfzGinfmDTQh5bOa4W5Zk/ogGo9Kok9GG3uEeqDTKPkIIOYAgzG f+3x2xl0qEKUKjyPbhd0 =cr0u -----END PGP SIGNATURE----- On 06/01/2016 03:51 PM, Eric des Courtis wrote: > I would be nice if BEAM could address these issues (not Erlang) so that > new more secure languages could be implemented on the BEAM. > > I think it will be done sooner or later. The sooner the better IMO if > BEAM is to remain relevant in the long term. > > On Wed, Jun 1, 2016 at 7:32 AM, Nathaniel Waisbrot > > wrote: > >> Does anyone know if there is anything in the works or proposed >> around the "If someone gets inside the network, the cookie is the >> only protection left" situation? > > > Yes: use SSL for distribution and to talk to other services. > http://erlang.org/doc/apps/ssl/ssl_distribution.html > > This assumes that by "inside the network" you mean past the > firewall/gateway/NAT. But you could also view this as using > encryption to build an inner network that just contains your Erlang > nodes. Once you're inside *that* network things are still open. > > The author suggests that since the BEAM is an OS you might want all > the access controls that a full OS offers. This would (e.g.) allow > some people to launch processes and kill the process that they'd > launched, but only some root user could terminate the Cowboy > application. This would take an enormous amount of work and there > are other ways of getting the same effect, so I can't imagine this > happening. > > What you should do is understand that a network of Erlang nodes > behaves (as much as possible) like a single node. If you don't trust > a remote node, do not link with it under any circumstances. If you > want to allow trusted and untrusted code to interoperate, you need > to write your own communication layer for them. > > > Finally, to get the security model of all the other languages that > I'm aware of, you can disable distribution. > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0xDD4316EC.asc Type: application/pgp-keys Size: 3090 bytes Desc: not available URL: From mikpelinux@REDACTED Wed Jun 1 19:11:06 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Wed, 1 Jun 2016 19:11:06 +0200 Subject: [erlang-questions] Turn off Garbage Collection for a time In-Reply-To: <574E84ED.7040400@gmail.com> References: <574E84ED.7040400@gmail.com> Message-ID: <22351.5930.388982.466466@gargle.gargle.HOWL> Tobias Pfeiffer writes: > Hi everyone, > > is there a way in Elixir/Erlang to turn off the Garbage collection? I've > searched and what I found so far is :erlang.garbage_collect to force > garbage collection.http://erlang.org/doc/man/erlang.html#garbage_collect-0 > > I was also made aware of :erlang.spawn_opt/2 http://erlang.org/doc/man/erlang.html#spawn_opt-2 and setting a high value for min_heap_size + setting fullsweep_after. > > Is there any way to turn it off completely? No. A VM using copying garbage collection usually has no way of extending stack and/or heap without identifying live data, and only a garbage collection cycle does that. There is an approach called "mostly copying" garbage collection by Joel Bartlett (at DEC), but to the best of my knowledge no current system implements it. I implemented it in a Scheme VM ages ago. > Why would I want to do that? I'm working on a benchmarking tool and I > don't want garbage collection to mess with my measured execution times. I run them multiple times and standard deviation goes way up because usually GC triggers during one run of n. > > Any hints welcome + thanks Just use spawn_opt to up the initial stack and heap size so that your test run process doesn't need to GC. From ok@REDACTED Thu Jun 2 00:05:02 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 2 Jun 2016 10:05:02 +1200 Subject: [erlang-questions] Turn off Garbage Collection for a time In-Reply-To: <574E84ED.7040400@gmail.com> References: <574E84ED.7040400@gmail.com> Message-ID: On 1/06/16 6:47 PM, Tobias Pfeiffer wrote: > > Why would I want to do that? I'm working on a benchmarking tool and I > don't want garbage collection to mess with my measured execution > times. I run them multiple times and standard deviation goes way up > because usually GC triggers during one run of n. I know that trying to avoid GC during benchmarking is a common practice, but I invite you to think very carefully about the point of doing a benchmark. If the thing you are measuring allocates memory, then that memory will have to be reclaimed somehow, and that's GC. So if you *don't* include GC time in your measurements, you will be (unintentionally) deceiving anyone who relies on your results about the true costs of using whatever it is. So why not go in the other direction? Run the benchmark LOTS of times so that you get lots of GCs? If you run the benchmark 100 times and get 7 GCs, then each run incurred 7% of the GC time, even if a GC didn't happen to occur during that run. (I've been looking at some papers recently that basically did this, but then, the point of those papers was to evaluate GC algorithms...) From ok@REDACTED Thu Jun 2 00:18:19 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 2 Jun 2016 10:18:19 +1200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: Message-ID: <5c6f7e61-4abd-7415-5347-681fbe6803d5@cs.otago.ac.nz> A rough summary: - The underlying C code can be attacked through Erlang. * Avoid NIFs if you can. - The default distribution machinery has weak security. * Search the archives for alternative distribution methods, e.g., TLS - Secrets can leak out through the OS and attacks can leak in. * Can dumps be routed to another machine, through TLS? * Limit use of external commands. Whatever happened to Laurie Brown's work on "Safe Erlang"? From lee.sylvester@REDACTED Thu Jun 2 00:26:56 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Thu, 2 Jun 2016 10:26:56 +1200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: <5c6f7e61-4abd-7415-5347-681fbe6803d5@cs.otago.ac.nz> References: <5c6f7e61-4abd-7415-5347-681fbe6803d5@cs.otago.ac.nz> Message-ID: In fairness; should we ever rely on the underlying virtual machine to be secure for any platform? If you were coding a *ahem* NodeJS app, would you rely on its security? Personally, I implement security for the messaging, be it HTTP or sockets etc., and implement safeguards around that VM through other technologies. In fact, I even proxy my HTTP / sockets. On Thu, Jun 2, 2016 at 10:18 AM, Richard A. O'Keefe wrote: > A rough summary: > - The underlying C code can be attacked through Erlang. > * Avoid NIFs if you can. > - The default distribution machinery has weak security. > * Search the archives for alternative distribution methods, > e.g., TLS > - Secrets can leak out through the OS and attacks can leak in. > * Can dumps be routed to another machine, through TLS? > * Limit use of external commands. > > Whatever happened to Laurie Brown's work on "Safe Erlang"? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jun 2 00:46:10 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 2 Jun 2016 10:46:10 +1200 Subject: [erlang-questions] Erlang offensive paper In-Reply-To: References: <5c6f7e61-4abd-7415-5347-681fbe6803d5@cs.otago.ac.nz> Message-ID: <971956e5-bb9f-4800-dcff-af1047c9f8ce@cs.otago.ac.nz> On 2/06/16 10:26 AM, Lee Sylvester wrote: > In fairness; should we ever rely on the underlying virtual machine to > be secure for any platform? Can we? Maybe, if there is a formal security model and the VM has been verified against that model. Should we? No. As a long-time reader of comp.risks, I keep on being amazed/amused/horrified/driven to despair by the weird and wonderful ways people find to break things, that I could never begin to imagine, let alone prevent. > If you were coding a *ahem* NodeJS app, would you rely on its security? If I were coding a node.js app, I would need my university's Employee Assistance Program's 3-free-sessions-with-a-counsellor to deal with depression. > Personally, I implement security for the messaging, be it HTTP or > sockets etc., and implement safeguards around that VM through other > technologies. In fact, I even proxy my HTTP / sockets. You know the joke about how to secure a computer. It takes a man and a dog. The man is there to feed the dog. The dog is there to bite the man if he tries to turn the computer on. You can do everything you said and your system will still leak information like crazy, see TEMPEST. And of course you have to trust your OS, hypervisor, and HAL, which I don't. The practical question is how MUCH vulnerability Erlang adds compared with alternatives, and how much it costs to mitigate those vulnerabilities compared with alternatives. The paper that triggered this thread (or at least the slides) didn't present any numbers. I haven't any myself. From be.dmitry@REDACTED Thu Jun 2 01:24:48 2016 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Thu, 02 Jun 2016 09:24:48 +1000 Subject: [erlang-questions] Turn off Garbage Collection for a time In-Reply-To: References: <574E84ED.7040400@gmail.com> Message-ID: <9FDDA559-2D8A-42C8-93AB-9A3EED9CE39D@gmail.com> As we know in Beam GC might not run at all for processes that have enough heap to operate until their end. And some optimisation techniques suggest using that possibility. The tests that hit GC only eventually give us average estimates for that particular setup that have those particular GC properties like consumed and available heap sizes. They are not that much helpful for any real application where those GC parameters will be different. That's why we always receive replies to measure in the real system. On the other hand if we force GC during a benchmark we measure high limit that includes possible GC pressure. This way we can predict that system won't consume more resources than in the benchmark. But that doesn't give us clues how system would behave if we optimise it so it doesn't GC at all. On 2 June 2016 8:05:02 AM AEST, "Richard A. O'Keefe" wrote: > > >On 1/06/16 6:47 PM, Tobias Pfeiffer wrote: >> >> Why would I want to do that? I'm working on a benchmarking tool and I >> don't want garbage collection to mess with my measured execution >> times. I run them multiple times and standard deviation goes way up >> because usually GC triggers during one run of n. >I know that trying to avoid GC during benchmarking is a common >practice, >but I invite you to think very carefully about the point of doing a >benchmark. >If the thing you are measuring allocates memory, then that memory will >have >to be reclaimed somehow, and that's GC. So if you *don't* include GC >time >in your measurements, you will be (unintentionally) deceiving anyone >who >relies on your results about the true costs of using whatever it is. > >So why not go in the other direction? Run the benchmark LOTS of times >so that you get lots of GCs? If you run the benchmark 100 times and >get >7 GCs, then each run incurred 7% of the GC time, even if a GC didn't >happen to occur during that run. > >(I've been looking at some papers recently that basically did this, but > >then, >the point of those papers was to evaluate GC algorithms...) > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Best wishes, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jun 2 02:29:23 2016 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 01 Jun 2016 17:29:23 -0700 Subject: [erlang-questions] Fourth draft of EEP 44 - Additional preprocessor directives In-Reply-To: References: Message-ID: <574F7DE3.6010801@gmail.com> On 10/29/2015 06:08 AM, Bj?rn Gustavsson wrote: > Here is the fourth draft with more clarifications. > > There is also a minor correction in the reference implementation. > > http://www.erlang.org/eeps/eep-0044.html > https://github.com/erlang/eep/blob/master/eeps/eep-0044.md > > /Bj?rn > It is unclear if these changes will make it into 19.0. Will any of EEP44 be available? The is_deprecated/3 described by EEP44 seems unusable due to being focused on Erlang/OTP deprecated functions. Could we instead implement it with usage of the module_info function so that anyone can easily use is_deprecated/3? I don't think it requires checking the exports list (if someone is checking an unexported function, that is their problem that they should discover later in their source code). I have an example implementation below: is_deprecated(Module, Function, Arity) -> Attributes = Module:module_info(attributes), lists:any(fun(Attribute) -> case Attribute of {deprecated, [_ | _] = Deprecated} -> lists:any(fun(Deprecate) -> case Deprecate of {Function, FunctionArity, _} when FunctionArity == Arity; FunctionArity == '_' -> true; {Function, FunctionArity} when FunctionArity == Arity; FunctionArity == '_' -> true; _ -> false end end, Deprecated); _ -> false end end, Attributes). From erlangworkshop@REDACTED Wed Jun 1 20:52:29 2016 From: erlangworkshop@REDACTED (Erlang Workshop) Date: Wed, 1 Jun 2016 20:52:29 +0200 Subject: [erlang-questions] [ANN] Deadline extended! Erlang Workshop 2016 In-Reply-To: <573DA35E.5000709@gmail.com> References: <573DA35E.5000709@gmail.com> Message-ID: <574F2EED.5020906@gmail.com> **** Paper submission deadline extended: 10 June, 2016 **** Apologies for any duplicates you may receive. CALL FOR PAPERS =============== Fifteenth ACM SIGPLAN Erlang Workshop ------------------------------ ----------------------------- Nara, Japan, September 23, 2016 Satellite event of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016) September 18-24, 2016 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing interesting contributions either in theoretical work or real world applications. Submission related to Erlang, Elixir, Akka, CloudHaskell, Occam, and functional programming are welcome and encouraged. Topics of interest include (but are not limited to): - virtual machine extensions and compilation techniques - implementations and interfaces of Erlang in/with other languages - new tools (profilers, tracers, debuggers, testing frameworks etc.) - language extensions - formal semantics, correctness and verification - testing Erlang programs - program analysis and transformation - Erlang-like languages and technologies - functional languages and multi-processing - concurrency in functional languages - functional languages and distributed computing - parallel programming - pattern based programming - Erlang in education The maximum length for technical papers is restricted to 12 pages. 2. Experience reports describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the experience report is restricted to 2 pages. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Melinda T?th, E?tv?s Lor?nd University, Hungary Scott Lystig Fritchie, Basho Japan KK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Jamie Allen, Typesafe Laura M. Castro, University of A Coru?a, Spain Natalia Chechina, University of Glasgow Viktoria F?rd?s, Erlang Solutions Yosuke Hara, Rakuten, Inc. Kenji Rikitake, KRPEO Bruce Tate, iCanMakeItBetter Simon Thompson, University of Kent, UK Important Dates ----------------------- Submissions due (extended): Friday, 10 June, 2016 Author notification: Friday, 8 July, 2016 Final copy due: Sunday, 31 July, 2016 Workshop date: September 23, 2016 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2016" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2016 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2016 web site at: http://conf.researchr.org/home/icfp-2016 Related Links -------------------- CFP: http://conf.researchr.org/track/icfp-2016/erlang-2016-papers ICFP 2016 web site: http://conf.researchr.org/home/icfp-2016 Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2016 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Atendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/Anti-harassment -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Thu Jun 2 10:51:04 2016 From: kenneth@REDACTED (Kenneth Lundin) Date: Thu, 2 Jun 2016 10:51:04 +0200 Subject: [erlang-questions] Decisions regarding EEP 44 and EEP 45 Message-ID: The following decisions regarding EEPs (Erlang Enhancement Proposals) has been taken byt the OTP technical board during the spring: EEP-45 Function_Name and Function_Arity as macros *Approved* Already implemented and available in OTP 19.0-rc1. EEP-44 Additional preprocessor directives *Partly Approved* Only the -warning and -error directives was approved and they are already implemented and available in OTP 19.0-rc1. Some of the reasons for not approving the other directives in EEP-44 where: - Rather unclear what an OTP_RELEASE is and it will not be clearer as there are plans to split OTP in several parts. - Could lead to strange situations if these directives are combined with a package manager which also deals with dependencies. - Could lead to strange situations if testing for a version or existence of functions in non OTP modules. In summary we where not convinced that the new directives will solve more problems than they might introduce. It is also already rather easy to create a preprocessor variable in the build-procesess that can be used in combination with -ifdef to achieve similar effects. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Thu Jun 2 10:52:39 2016 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 2 Jun 2016 10:52:39 +0200 Subject: [erlang-questions] Fourth draft of EEP 44 - Additional preprocessor directives In-Reply-To: <574F7DE3.6010801@gmail.com> References: <574F7DE3.6010801@gmail.com> Message-ID: See a separate mail about EEP 44 and 45 that I just sent. The information is also on www.erlang.org /Kenneth On Thu, Jun 2, 2016 at 2:29 AM, Michael Truog wrote: > On 10/29/2015 06:08 AM, Bj?rn Gustavsson wrote: > >> Here is the fourth draft with more clarifications. >> >> There is also a minor correction in the reference implementation. >> >> http://www.erlang.org/eeps/eep-0044.html >> https://github.com/erlang/eep/blob/master/eeps/eep-0044.md >> >> /Bj?rn >> >> It is unclear if these changes will make it into 19.0. Will any of EEP44 > be available? > > The is_deprecated/3 described by EEP44 seems unusable due to being focused > on Erlang/OTP deprecated functions. Could we instead implement it with > usage of the module_info function so that anyone can easily use > is_deprecated/3? I don't think it requires checking the exports list (if > someone is checking an unexported function, that is their problem that they > should discover later in their source code). I have an example > implementation below: > > is_deprecated(Module, Function, Arity) -> > Attributes = Module:module_info(attributes), > lists:any(fun(Attribute) -> > case Attribute of > {deprecated, [_ | _] = Deprecated} -> > lists:any(fun(Deprecate) -> > case Deprecate of > {Function, FunctionArity, _} > when FunctionArity == Arity; > FunctionArity == '_' -> > true; > {Function, FunctionArity} > when FunctionArity == Arity; > FunctionArity == '_' -> > true; > _ -> > false > end > end, Deprecated); > _ -> > false > end > end, Attributes). > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Thu Jun 2 11:21:00 2016 From: sdl.web@REDACTED (Leo Liu) Date: Thu, 02 Jun 2016 17:21:00 +0800 Subject: [erlang-questions] Master failed to compile on OSX 10.11.5 Message-ID: CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_os.o CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_lists.o beam/beam_emu.c:5344:51: error: use of undeclared identifier 'next' "", next); ^ beam/beam_emu.c:5414:3: error: indirect goto in function with no address-of-label expressions Goto(*I); ^ beam/beam_emu.c:58:21: note: expanded from macro 'Goto' # define Goto(Rel) goto *((void *)Rel) ^ 2 errors generated. make[3]: *** [obj/x86_64-apple-darwin15.5.0/opt/smp/beam_emu.o] Error 1 make[3]: *** Waiting for unfinished jobs.... make[2]: *** [opt] Error 2 make[1]: *** [smp] Error 2 make: *** [emulator] Error 2 From dangud@REDACTED Thu Jun 2 11:56:32 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 02 Jun 2016 09:56:32 +0000 Subject: [erlang-questions] math ceil floor Message-ID: Hi I made pull request https://github.com/erlang/otp/pull/1084 which implements ceil and floor in the math module. The question is, should they return integer or floats? The implementation in the PR currently returns integers and also implements ceilf and floorf which returns floats. Opinions? /Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Thu Jun 2 12:05:52 2016 From: rickard@REDACTED (Rickard Green) Date: Thu, 2 Jun 2016 12:05:52 +0200 Subject: [erlang-questions] Master failed to compile on OSX 10.11.5 In-Reply-To: References: Message-ID: On Thu, Jun 2, 2016 at 11:21 AM, Leo Liu wrote: > CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_os.o > CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_lists.o > beam/beam_emu.c:5344:51: error: use of undeclared identifier 'next' > "", next); > ^ > beam/beam_emu.c:5414:3: error: indirect goto in function with no > address-of-label expressions > Goto(*I); > ^ > beam/beam_emu.c:58:21: note: expanded from macro 'Goto' > # define Goto(Rel) goto *((void *)Rel) > ^ > 2 errors generated. > make[3]: *** [obj/x86_64-apple-darwin15.5.0/opt/smp/beam_emu.o] Error 1 > make[3]: *** Waiting for unfinished jobs.... > make[2]: *** [opt] Error 2 > make[1]: *** [smp] Error 2 > make: *** [emulator] Error 2 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Thanks, it will soon be fixed, however not in 19.0-rc2. Note that it will build if you don't enable dirty schedulers. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From mikpelinux@REDACTED Thu Jun 2 12:11:09 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Thu, 2 Jun 2016 12:11:09 +0200 Subject: [erlang-questions] [SPAM] Master failed to compile on OSX 10.11.5 In-Reply-To: References: Message-ID: <22352.1597.811490.465116@gargle.gargle.HOWL> Leo Liu writes: > CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_os.o > CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_lists.o > beam/beam_emu.c:5344:51: error: use of undeclared identifier 'next' > "", next); > ^ Unbound variable inside an #ifdef USE_VM_PROBES block. I notice a similar block much earlier in the file, and there 'next' _is_ bound. Could be a misapplied patch, a broken refactoring, or something like that. > beam/beam_emu.c:5414:3: error: indirect goto in function with no > address-of-label expressions > Goto(*I); > ^ Inside #ifdef ERTS_DIRTY_SCHEDULERS. Looks like a blatant bug. Like the compiler said, there are no &&label expressions in that function, so the indirect goto would have nowhere legitimate to go to, if it ever was to execute. And despite some claims, you cannot goto from one function to another and except things to work reliably. From henrik.x.nord@REDACTED Thu Jun 2 15:28:35 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Thu, 2 Jun 2016 15:28:35 +0200 Subject: [erlang-questions] Erlang/OTP 19.0-rc2 is available for testing Message-ID: <57503483.9050407@ericsson.com> This is the second and last release candidate before the final OTP 19.0 product release in June 2016. Among other changes you might want to consider: *+hmqd off_heap|on_heap* Sets the default value for the process flag message_queue_data. If +hmqd is not passed, on_heap will be the default. For more information, see the documentation of process_flag(message_queue_data, MQD) . Previous default value "mixed" used in OTP-19.0-rc1 has been removed. The new default value is "on_heap", which is similar to the previous releases. You can find the tag on github. Documentation on erlang.org: http://erlang.org/documentation/doc-8.0-rc2/doc/ and full release notes here: http://erlang.org/download/OTP-19.0-rc2.README -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Thu Jun 2 15:34:17 2016 From: rickard@REDACTED (Rickard Green) Date: Thu, 2 Jun 2016 15:34:17 +0200 Subject: [erlang-questions] Master failed to compile on OSX 10.11.5 In-Reply-To: References: Message-ID: On Thu, Jun 2, 2016 at 12:05 PM, Rickard Green wrote: > On Thu, Jun 2, 2016 at 11:21 AM, Leo Liu wrote: >> CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_os.o >> CC obj/x86_64-apple-darwin15.5.0/opt/smp/erl_bif_lists.o >> beam/beam_emu.c:5344:51: error: use of undeclared identifier 'next' >> "", next); >> ^ >> beam/beam_emu.c:5414:3: error: indirect goto in function with no >> address-of-label expressions >> Goto(*I); >> ^ >> beam/beam_emu.c:58:21: note: expanded from macro 'Goto' >> # define Goto(Rel) goto *((void *)Rel) >> ^ >> 2 errors generated. >> make[3]: *** [obj/x86_64-apple-darwin15.5.0/opt/smp/beam_emu.o] Error 1 >> make[3]: *** Waiting for unfinished jobs.... >> make[2]: *** [opt] Error 2 >> make[1]: *** [smp] Error 2 >> make: *** [emulator] Error 2 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > Thanks, it will soon be fixed, however not in 19.0-rc2. > > Note that it will build if you don't enable dirty schedulers. > > Regards, > Rickard > -- > Rickard Green, Erlang/OTP, Ericsson AB Fixed in master now. Commit e25df0378738ba17fb66ae0bd947c439ac925800 equals OTP-19.0-rc2 + the fix for this issue. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From technion@REDACTED Fri Jun 3 00:42:03 2016 From: technion@REDACTED (Technion) Date: Thu, 2 Jun 2016 22:42:03 +0000 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: Following from other languages, C's ceil returns floats, but Ruby's ceil returns an Integer, PHP returns a float. I'm seeing here there isn't a particularly accepted standard. ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Dan Gudmundsson Sent: Thursday, 2 June 2016 7:56:32 PM To: Erlang Questions Subject: [erlang-questions] math ceil floor Hi I made pull request https://github.com/erlang/otp/pull/1084 which implements ceil and floor in the math module. The question is, should they return integer or floats? The implementation in the PR currently returns integers and also implements ceilf and floorf which returns floats. Opinions? /Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Jun 3 03:39:18 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 03 Jun 2016 01:39:18 +0000 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: I would think the output type should be the same as the input type. So float in, float out. FYI, Java takes a double and returns a double. On Thu, Jun 2, 2016 at 3:42 PM Technion wrote: > Following from other languages, C's ceil returns floats, but Ruby's ceil > returns an Integer, PHP returns a float. > > > I'm seeing here there isn't a particularly accepted standard. > ------------------------------ > *From:* erlang-questions-bounces@REDACTED < > erlang-questions-bounces@REDACTED> on behalf of Dan Gudmundsson < > dangud@REDACTED> > *Sent:* Thursday, 2 June 2016 7:56:32 PM > *To:* Erlang Questions > *Subject:* [erlang-questions] math ceil floor > > Hi > I made pull request https://github.com/erlang/otp/pull/1084 which > implements ceil and floor in the math module. > > The question is, should they return integer or floats? > > The implementation in the PR currently returns integers and also > implements ceilf and floorf which returns floats. > > Opinions? > > /Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Fri Jun 3 04:37:23 2016 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 03 Jun 2016 10:37:23 +0800 Subject: [erlang-questions] Master failed to compile on OSX 10.11.5 References: Message-ID: On 2016-06-02 15:34 +0200, Rickard Green wrote: > Fixed in master now. Commit e25df0378738ba17fb66ae0bd947c439ac925800 > equals OTP-19.0-rc2 + the fix for this issue. Thanks. Build succeeded. Leo From ok@REDACTED Fri Jun 3 04:59:12 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 3 Jun 2016 14:59:12 +1200 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: On 2/06/16 9:56 PM, Dan Gudmundsson wrote: > Hi > I made pull request https://github.com/erlang/otp/pull/1084which > implements ceil and floor in the math module. > > The question is, should they return integer or floats? Yes, they should. For each of floor, ceiling, round, truncate there are two sensible functions: one returns the appropriate integer as an integer, and the other returns it as a floating-point number. > The implementation in the PR currently returns integers and also > implements ceilf and floorf which returns floats. Just what we needed, yet another name for ffloor. Let me draw your attention to http://www.lispworks.com/documentation/HyperSpec/Body/f_floorc.htm which defines floor ffloor ceiling fceiling round fround truncate ftruncate and to ANSI INCITS 319-1998, where #floor, #ceiling, #rounded, and #truncated return Integers, whereas x integerPart (truncation), and x // 1 (flooring) answer the same type as x. Since your 'ceil' function doesn't have the semantics of C's, it's not clear why you want to use the C name. For what it's worth, my Smalltalk library includes several versions of rounding differing in what they do with a value exactly half-way between two integers. floor(X+0.5) is only one alternative... From ok@REDACTED Fri Jun 3 05:36:16 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 3 Jun 2016 15:36:16 +1200 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: On 3/06/16 10:42 AM, Technion wrote: > > Following from other languages, C's ceil returns floats, but Ruby's > ceil returns an Integer, PHP returns a float. > > > I'm seeing here there isn't a particularly accepted standard. > Let's put that another way. C's library is defective in only offering truncation to integer (via (int)) and rounding (via l[l]rint{f,,l}()), no int-valued floor or ceiling. Ruby's Float class is defective in only offering conversion to integer, not float-valued ceiling &c (Float.round can be persuaded to return a float, but not when you want it to be integral, which is both absurd and painfully typical of Ruby). PHP is, well, PHP. If, on the other hand, we turn to other languages, we learn that Lisp has {f,}(round,floor,ceiling,truncate) and ML has realFloor, realCeil, realTrunc, realRound and floor, ceil, trunc, round, 8 functions, not 4. (The REAL signature, section 11.50 of The Standard ML Basis Library.) So there is prior art for providing *both*, with the simple name delivering an integer, and the prefixed name returning float. From sdl.web@REDACTED Fri Jun 3 06:21:58 2016 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 03 Jun 2016 12:21:58 +0800 Subject: [erlang-questions] Erlang/OTP 19.0-rc2 is available for testing References: <57503483.9050407@ericsson.com> Message-ID: On 2016-06-02 15:28 +0200, Henrik Nord X wrote: > http://erlang.org/download/OTP-19.0-rc2.README Are these OTP-XXXXX's internal i.e. no public access? thanks. Leo From foldl@REDACTED Fri Jun 3 06:05:07 2016 From: foldl@REDACTED (Zhengji Li) Date: Fri, 3 Jun 2016 12:05:07 +0800 Subject: [erlang-questions] =?utf-8?b?562U5aSNOiAgbWF0aCBjZWlsIGZsb29y?= Message-ID: IMHO, ceil/floor should return integers when we have big-integers. In C, to avoid overflow, they have to return floating point numbers. IMHO, I don?t think we need ffloor/fround. Mathematica can be taken as an example. Best regards, Li Zhengji ???: Richard A. O'Keefe ????: 2016?6?3? 11:36 ???: erlang-questions@REDACTED ??: Re: [erlang-questions] math ceil floor On 3/06/16 10:42 AM, Technion wrote: > > Following from other languages, C's ceil returns floats, but Ruby's > ceil returns an Integer, PHP returns a float. > > > I'm seeing here there isn't a particularly accepted standard. > Let's put that another way. C's library is defective in only offering truncation to integer (via (int)) and rounding (via l[l]rint{f,,l}()), no int-valued floor or ceiling. Ruby's Float class is defective in only offering conversion to integer, not float-valued ceiling &c (Float.round can be persuaded to return a float, but not when you want it to be integral, which is both absurd and painfully typical of Ruby). PHP is, well, PHP. If, on the other hand, we turn to other languages, we learn that Lisp has {f,}(round,floor,ceiling,truncate) and ML has realFloor, realCeil, realTrunc, realRound and floor, ceil, trunc, round, 8 functions, not 4. (The REAL signature, section 11.50 of The Standard ML Basis Library.) So there is prior art for providing *both*, with the simple name delivering an integer, and the prefixed name returning float. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri Jun 3 09:09:38 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 3 Jun 2016 09:09:38 +0200 Subject: [erlang-questions] Erlang/OTP 19.0-rc2 is available for testing In-Reply-To: References: <57503483.9050407@ericsson.com> Message-ID: On Fri, Jun 3, 2016 at 6:21 AM, Leo Liu wrote: > On 2016-06-02 15:28 +0200, Henrik Nord X wrote: > > http://erlang.org/download/OTP-19.0-rc2.README > > Are these OTP-XXXXX's internal i.e. no public access? thanks. > Yes they are internal, though they contain very little more information that what is needed to generate the readme+release notes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rizkhan@REDACTED Fri Jun 3 09:18:19 2016 From: rizkhan@REDACTED (Rizwan Khan) Date: Fri, 3 Jun 2016 12:18:19 +0500 Subject: [erlang-questions] Walkie Talkie app with Erlang backend Message-ID: I am thinking of creating a clone for Voxer which is a walkie talkie app which uses data for transferring the messages. It uses nodejs at the backend and I think erlang would be a better fit for this. What do you guys think? And rather than writing a new backend, cant I do the same thing with and XMPP server? Suggestions needed. Thanks, Rizwan Khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 3 13:25:16 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 3 Jun 2016 14:25:16 +0300 Subject: [erlang-questions] monitor long_schedule and strange timeouts In-Reply-To: <22335.8155.361923.881447@gargle.gargle.HOWL> References: <22335.8155.361923.881447@gargle.gargle.HOWL> Message-ID: Can you give a hint how to make a perf snapshot of what is happening with erlang? We are getting these problems and they are becoming annoying =( -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Jun 3 15:49:50 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 03 Jun 2016 13:49:50 +0000 Subject: [erlang-questions] [Meetup] Erlang Paris Message-ID: Just a quick mail to inform french people and others that our next Erlang Meetup in Paris will be on 2016/06/14 at 6:30PM in the Murex Offices. We will have 2 talks and then discussions this time. Everybody is welcome, this is a free event :) More details and registration are here: http://erlang.paris See you there! Beno?t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Fri Jun 3 19:08:46 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Fri, 3 Jun 2016 17:08:46 +0000 Subject: [erlang-questions] Efficient way to store message lists in mnesia Message-ID: Hi, I need to store a list of unread messages for every user id. The order of the messages needs to be preserved. Currently I store the messages in a list as the value of the user id key in a table of type set: {user_id, [message]} The table is a disc_copies table. Whenever someone sends a message to a user, the message is prepended to the head of the message list of the user. Now my concern is, adding a new message to the list requires reading and writing the whole list, which can be pretty slow if the list is very long, i.e. thousands of messages. There is another way to do this, which uses a table of type bag and composite keys: {user_id, timestamp1, message1} {user_id, timestamp2, message2} {user_id, timestamp3, message3} ... But I don't think this is any better because I heard that mnesia bag tables performance is very bad: http://erlang.org/pipermail/erlang-questions/2011-March/057044.html Are there other ways to store lists of messages in mnesia where reading/updating the message lists can be very efficient? -Khitai From mattevans123@REDACTED Fri Jun 3 19:24:12 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Fri, 3 Jun 2016 13:24:12 -0400 Subject: [erlang-questions] [erlang-bugs] beam core file R17 In-Reply-To: References: , , <22347.4371.901213.18449@gargle.gargle.HOWL>, , , , , Message-ID: For what it's worth here's a gdb with symbols: (gdb) bt #0 sweep_off_heap (p=0x7faeb8fc5488, fullsweep=-1295963664) at beam/erl_gc.c:2353 #1 0x00000000004b77d1 in do_minor (p=0x7faeb8fc5488, new_sz=, objv=, nobj=) at beam/erl_gc.c:1166 #2 0x00000000004b8479 in minor_collection (recl=, nobj=, objv=, need=, p=) at beam/erl_gc.c:876 #3 erts_garbage_collect (p=0x7faeb8fc5488, need=, objv=, nobj=) at beam/erl_gc.c:450 #4 0x00000000004e1374 in process_main () at beam/beam_emu.c:1858 #5 0x000000000048071d in sched_thread_func (vesdp=) at beam/erl_process.c:7719 #6 0x0000000000549f89 in thr_wrapper (vtwd=) at pthread/ethread.c:106 #7 0x00007faeba32ba30 in ?? () #8 0x0000000000000000 in ?? () From: mattevans123@REDACTED To: garazdawi@REDACTED Date: Sun, 29 May 2016 21:32:58 -0400 CC: erlang-questions@REDACTED; erlang-bugs@REDACTED Subject: Re: [erlang-questions] [erlang-bugs] beam core file R17 Thanks, Fortunately our latest release has moved all the NIF logic to a separate CNODE based process. I am leaning towards the NIF as the cause since the VM itself has proven to be very stable. Date: Sun, 29 May 2016 21:23:15 +0200 Subject: Re: [erlang-bugs] beam core file R17 From: garazdawi@REDACTED To: mattevans123@REDACTED CC: erlang-questions@REDACTED sweep_off_heap is run when collecting refc binaries (among other things), so if you by mistake have decremented the reference count of a binary in a nif too much, this error will happen when the GC is inspecting the binary. I'd recommend looking for something like that in any nifs you have. On Sun, May 29, 2016 at 6:04 PM, Matthew Evans wrote: Thanks, this software does use nifs Sent from my iPhone > On May 29, 2016, at 11:56 AM, Mikael Pettersson wrote: > > Matthew Evans writes: >> Hi, >> This core was found on a live system (R17): >> >> >> >> >> >> >> >> >> 10:16:38:# erl >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] >> >> >> Eshell V6.2 (abort with ^G) >> 1> >> >> ........ >> >> >> >> >> >> >> >> >> May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1ac000] >> warning: Can't read pathname for load map: Input/output error.[Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc.so.6 > > It looks like your beam.smp binary lacks debugging information, so we only know the general > area where it crashed (sweep_off_heap() as called from do_minor()). Crashes here would usually > be due to memory corruption, which could be caused by: > - a bug in the VM > - a bug in a NIF > - a bug in HiPE > - a bug in the C compiler used to compile the VM (I've seen that happen at least 3 times) > - a HW error (though you'd then also find e.g. machine check events logged) > > If you want to debug this, you should first ensure that your beam.smp gets built and installed > with full debugging information (just attach gdb, bt, and list to verify). You should also try > without NIFs or native code, if those are used and you can configure them not to be. _______________________________________________ erlang-bugs mailing list erlang-bugs@REDACTED http://erlang.org/mailman/listinfo/erlang-bugs _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From samuelrivas@REDACTED Fri Jun 3 23:37:48 2016 From: samuelrivas@REDACTED (Samuel) Date: Fri, 3 Jun 2016 23:37:48 +0200 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: >> I made pull request https://github.com/erlang/otp/pull/1084which >> implements ceil and floor in the math module. >> >> The question is, should they return integer or floats? > > > Yes, they should. For each of floor, ceiling, round, truncate there are > two sensible functions: one returns the appropriate integer as an integer, > and the other returns it as a floating-point number. What is the use case when you want floor/ceil to return a float and not an integer in Erlang? Since Erlang has bignums, which I understand can represent the integer part of any float, and those operations are supposed to return an integer I can't see a reason for a float result (other than, maybe, performance). Best -- Samuel From sdl.web@REDACTED Sat Jun 4 04:54:47 2016 From: sdl.web@REDACTED (Leo Liu) Date: Sat, 04 Jun 2016 10:54:47 +0800 Subject: [erlang-questions] Erlang/OTP 19.0-rc2 is available for testing References: <57503483.9050407@ericsson.com> Message-ID: On 2016-06-03 09:09 +0200, Lukas Larsson wrote: > Yes they are internal, though they contain very little more information > that what is needed to generate the readme+release notes. Thanks for the info. Leo From t6sn7gt@REDACTED Sat Jun 4 05:11:16 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Fri, 3 Jun 2016 23:11:16 -0400 Subject: [erlang-questions] Strange float / tuple problem Message-ID: <575246D4.3080005@aim.com> Hi all, I'm having a problem I don't understand. I'm creating some X and Y coordinates in a program. When I print them via: io:format("~nX: ~f Y: ~f ", [X,Y]), I get: X: 0.000000 Y: -5.000000 which is correct. ====== When I make a tuple of them, add them to a list and print them via: io:format("~nL: ~p~n", [L]), I get: {4.440892098500626e-16,-5.0}, which is very strange. There are many other {X,Y} tuples in the list and all of them are correct, including earlier values of X which were also 0.0 and print correctly as 0.0. ===== Your thoughts would be very welcome. Thanks. Don From zxq9@REDACTED Sat Jun 4 05:58:14 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 04 Jun 2016 12:58:14 +0900 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <575246D4.3080005@aim.com> References: <575246D4.3080005@aim.com> Message-ID: <2976609.FPqHkKGNq5@changa> On 2016?6?3? ??? 23:11:16 Donald Steven wrote: > Hi all, > > I'm having a problem I don't understand. > > I'm creating some X and Y coordinates in a program. When I print them > via: io:format("~nX: ~f Y: ~f ", [X,Y]), > > I get: X: 0.000000 Y: -5.000000 > > which is correct. > > ====== > > When I make a tuple of them, add them to a list and print them via: > io:format("~nL: ~p~n", [L]), > > I get: {4.440892098500626e-16,-5.0}, > > which is very strange. There are many other {X,Y} tuples in the list > and all of them are correct, including earlier values of X which were > also 0.0 and print correctly as 0.0. > > ===== > > Your thoughts would be very welcome. Life with floats can be a bit ?(???;)? 4.440892098500626e-16 == 0.000000000000000444089209850 Pretty darn close to zero. This 0 is almost certainly the result of some computation and is a float -- so it is not really a 0, it is some fractional value that probably cannot be represented in binary directly. So you have something *very close* to, but not quite, 0.0 left over. -Craig From zxq9@REDACTED Sat Jun 4 06:11:29 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 04 Jun 2016 13:11:29 +0900 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <2976609.FPqHkKGNq5@changa> References: <575246D4.3080005@aim.com> <2976609.FPqHkKGNq5@changa> Message-ID: <4166951.SFMmkHvP3l@changa> On 2016?6?4? ??? 12:58:14 zxq9 wrote: > On 2016?6?3? ??? 23:11:16 Donald Steven wrote: > > > > I get: X: 0.000000 Y: -5.000000 > > > > which is correct. > > > > ====== > > > > When I make a tuple of them, add them to a list and print them via: > > io:format("~nL: ~p~n", [L]), > > > > I get: {4.440892098500626e-16,-5.0}, > > > > > > Your thoughts would be very welcome. > > 4.440892098500626e-16 == 0.000000000000000444089209850 Sorry, I used the wrong symbol. It really should be written as: 4.440892098500626e-16 ? 0.000000000000000444089209850 These two are not actually equal -- and actually doing strict comparisons on float values after computation is a recipe for intense frustration, so avoid it by deriving an approximation that is "good enough" first, and then comparing that. From bjorn@REDACTED Sat Jun 4 08:13:07 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 4 Jun 2016 08:13:07 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance Message-ID: We think that it is time to update the Myths of Erlang Performance in the Efficiency Guide. Some myths have probably died by now and some new may have arisen. I am open to suggestions for myths to retire (move to a separate section at very end of the guide) and new myths to add. Currently, I think that the following myths definitely should be retired. 2.1 Myth: Funs are Slow 2.2 Myth: List Comprehensions are Slow Are any other myths dead? I am considering adding a new myth, something like this: "I thought binaries were always faster than lists, so why is binary matching slower than list matching?" Then there should be an example for both binaries and lists, something like this: f(<>) -> %% Do something with B. f(T); f(<<>>) -> %% Return something. The answer should be that for more complicated binary patterns, binary matching wins, and also that the cost of using binaries vs. lists in the entire application must be considered. Long lists or many short lists in many processes will cost more in GC costs; then there are different costs for sending binaries vs. lists as messages. Are there any more myths I should consider? /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Sat Jun 4 08:32:41 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Jun 2016 09:32:41 +0300 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: Maybe myth about speed of nif? We in erlyvideo have tried many kind of nif functions and have removed most of them. For example making mmap was one of the worsest ideas On Sat, Jun 4, 2016 at 9:13 AM, Bj?rn Gustavsson wrote: > We think that it is time to update the Myths of Erlang Performance in > the Efficiency Guide. Some myths have probably died by now and some > new may have arisen. > > I am open to suggestions for myths to retire (move to a separate > section at very end of the guide) and new myths to add. > > Currently, I think that the following myths definitely should be retired. > > 2.1 Myth: Funs are Slow > > 2.2 Myth: List Comprehensions are Slow > > Are any other myths dead? > > I am considering adding a new myth, something like this: > > "I thought binaries were always faster than lists, so why is binary > matching slower than list matching?" > > Then there should be an example for both binaries and lists, something > like this: > > f(<>) -> > %% Do something with B. > f(T); > f(<<>>) -> > %% Return something. > > The answer should be that for more complicated binary patterns, binary > matching wins, and also that the cost of using binaries vs. lists in > the entire application must be considered. Long lists or many short > lists in many processes will cost more in GC costs; then there are > different costs for sending binaries vs. lists as messages. > > > Are there any more myths I should consider? > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Sat Jun 4 08:33:28 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Sat, 04 Jun 2016 06:33:28 +0000 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: On Fri, Jun 3, 2016 at 11:38 PM Samuel wrote: > >> I made pull request https://github.com/erlang/otp/pull/1084which > >> implements ceil and floor in the math module. > >> > >> The question is, should they return integer or floats? > > > > > > Yes, they should. For each of floor, ceiling, round, truncate there are > > two sensible functions: one returns the appropriate integer as an > integer, > > and the other returns it as a floating-point number. > > What is the use case when you want floor/ceil to return a float and > not an integer in Erlang? > > Since Erlang has bignums, which I understand can represent the integer > part of any float, and those operations are supposed to return an > integer I can't see a reason for a float result (other than, maybe, > performance). > > My reason for a variant that returns a float is purely performance the beam compiler is good at avoiding conversion/type checking floats in code blocks that handle floats, and sometimes you want to continue to work with the result as float, e.g. multiply the result with another float. > Best > -- > Samuel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sat Jun 4 08:55:04 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sat, 04 Jun 2016 08:55:04 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: Message-ID: <3b639ac6-aabe-4994-8aea-84c5e402e1fd@email.android.com> An HTML attachment was scrubbed... URL: From valentin@REDACTED Sat Jun 4 09:07:53 2016 From: valentin@REDACTED (Valentin Micic) Date: Sat, 4 Jun 2016 09:07:53 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> On 04 Jun 2016, at 8:13 AM, Bj?rn Gustavsson wrote: > "I thought binaries were always faster than lists, so why is binary > matching slower than list matching?" > Stated this way, the myth may be confusing to some people (well, it is for me). So, what is the myth? Is it that binaries are faster than lists, or that binary matching is slower then list matching? Kind regards V/ From bjorn@REDACTED Sat Jun 4 09:41:17 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 4 Jun 2016 09:41:17 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> Message-ID: On Sat, Jun 4, 2016 at 9:07 AM, Valentin Micic wrote: > > On 04 Jun 2016, at 8:13 AM, Bj?rn Gustavsson wrote: > >> "I thought binaries were always faster than lists, so why is binary >> matching slower than list matching?" >> > > Stated this way, the myth may be confusing to some people (well, it is for me). > So, what is the myth? Is it that binaries are faster than lists, or that binary matching is slower then list matching? > Yes, the myth needs some polishing. Here are some more details of what I had in mind. It is often recommended to use binaries (for example in this mailing) instead of lists. Someone read this recommendation, tries it, and is then surprised that this code: f(<>) -> %% Do something with B. f(T); f(<<>>) -> %% Return something. is slower than: f([B|T]) -> %% Do something with B. f(T); f([]) -> %% Return something. Fact is, it is hard to beat the speed of lists, and that can be surprising. When you do more complicated matching such as: f(<>) -> %% Do something with B. f(T); f(<<>>) -> %% Return something. instead of: f([A,B,C,D|T]) -> X = (A bsl 24) bor (B bsl 16) bor (C bsl 8) bor D, %% Do something with X. f(T); f([]) -> %% Return something. bit syntax matching wins. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Sat Jun 4 09:51:12 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 4 Jun 2016 09:51:12 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <3b639ac6-aabe-4994-8aea-84c5e402e1fd@email.android.com> References: <3b639ac6-aabe-4994-8aea-84c5e402e1fd@email.android.com> Message-ID: On Sat, Jun 4, 2016 at 8:55 AM, ?ric Pailleau wrote: > Hi, > I think it would be better to move fun myths/reality in fun paragraph, and > so on. > So that newcomers are aware of what they should / shouldn't do by reading > related paragraph. Erlang documentation is big, and sometimes difficult to > find some informations unless reading the whole regularly... > Myths of Erlang should be kept only for general myth regarding Beam or other > languages comparison, Mho. We intentionally wanted to keep most information about efficiency in one place, in the Efficiency Guide, not sprinkled throughout the doumentation. The idea is not to worry about performance until you have measured your correct solution and found it to be too slow. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Sat Jun 4 10:07:22 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 4 Jun 2016 10:07:22 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On Sat, Jun 4, 2016 at 8:32 AM, Max Lapshin wrote: > Maybe myth about speed of nif? > > We in erlyvideo have tried many kind of nif functions and have removed most > of them. For example making mmap was one of the worsest ideas > Yes, that is a good candidate. Perhaps: "Myth: Replacing Erlang code with a NIF will always speed up your application"? Does anyone have any other examples of failed conversions to NIFs? /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From list1@REDACTED Sat Jun 4 10:08:02 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 4 Jun 2016 08:08:02 +0000 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: On 04/06/2016 06:33, Dan Gudmundsson wrote: > On Fri, Jun 3, 2016 at 11:38 PM Samuel > wrote: > > >> I made pull request https://github.com/erlang/otp/pull/1084which > >> implements ceil and floor in the math module. > >> > >> The question is, should they return integer or floats? > > > > > > Yes, they should. For each of floor, ceiling, round, truncate > there are > > two sensible functions: one returns the appropriate integer as > an integer, > > and the other returns it as a floating-point number. > > What is the use case when you want floor/ceil to return a float and > not an integer in Erlang? > > Since Erlang has bignums, which I understand can represent the integer > part of any float, and those operations are supposed to return an > integer I can't see a reason for a float result (other than, maybe, > performance). > > > My reason for a variant that returns a float is purely performance the > beam compiler is good at avoiding conversion/type checking floats in > code blocks that handle floats, and sometimes you want to continue to > work with the result as float, e.g. multiply the result with another > float. > Is there any reason why there may not be two functions, one returning float and one returning integer? One function won't cover all scenarios and trying to discuss which one is better in that context will never end. If only one type of function is possible then it would be best to verify which usage is more common out there - in real applications - so that the decision isn't speculative but backed by some data. Grzegorz -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Sat Jun 4 10:31:21 2016 From: matthias@REDACTED (Matthias Lang) Date: Sat, 4 Jun 2016 10:31:21 +0200 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <575246D4.3080005@aim.com> References: <575246D4.3080005@aim.com> Message-ID: <20160604083121.GA11334@corelatus.se> > When I print them via io:format("~nX: ~f Y: ~f ", [X,Y]), > > I get: X: 0.000000 Y: -5.000000 > > which is correct. > > ====== > > When I make a tuple of them, add them to a list and print them via: > io:format("~nL: ~p~n", [L]), > > I get: {4.440892098500626e-16,-5.0}, > > which is very strange. Does this clear things up? 1> X = 4.44E-16. 4.44e-16 2> io:fwrite("With twiddle-f: ~f With twiddle-p: ~p\n", [X, X]). With twiddle-f: 0.000000 With twiddle-p: 4.44e-16 With the ~f format string, the default precision is 6, which is why you see 0.000000 in that case. Here's a site which talks about the general problem without getting unnecessarily abstract: http://floating-point-gui.de/ Matt From eric.pailleau@REDACTED Sat Jun 4 10:47:41 2016 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 4 Jun 2016 10:47:41 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: <3b639ac6-aabe-4994-8aea-84c5e402e1fd@email.android.com> Message-ID: <575295AD.1040201@wanadoo.fr> Hi Bj?rn, make sens, but in such case a "trouble-shooting" / "common caveat" / "performance issue" link in 'fun' paragraph to Myths paragraph should be done for people wanted to go further. I know it is another thing to maintain, but as far documentation is bigger and bigger, links are really usefull. I sometime remember having readen something interresting in documentation and do not remember where, and find it again weeks later fortuitously, while do not need it anymore. but I suppose it is a general problem in all big documentation. Le 04/06/2016 09:51, Bj?rn Gustavsson a ?crit : > On Sat, Jun 4, 2016 at 8:55 AM, ?ric Pailleau wrote: >> Hi, >> I think it would be better to move fun myths/reality in fun paragraph, and >> so on. >> So that newcomers are aware of what they should / shouldn't do by reading >> related paragraph. Erlang documentation is big, and sometimes difficult to >> find some informations unless reading the whole regularly... >> Myths of Erlang should be kept only for general myth regarding Beam or other >> languages comparison, Mho. > > We intentionally wanted to keep most information about efficiency in > one place, in the Efficiency Guide, not sprinkled throughout the > doumentation. The idea is not to worry about performance until you > have measured your correct solution and found it to be too slow. > > /Bj?rn > From erlang@REDACTED Sat Jun 4 11:43:44 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 4 Jun 2016 11:43:44 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: This is an excellent suggestion: Some other myths.For starters: 1) That using cases/if/multiple entry points is somehow more efficient I've see several posts suggesting that rewriting functions with a single entry point conta multiple entry points and so on will be magically more efficient - when in fact after compilation the code is pretty much identical. 2) That you can predict performance without doing measurements. A simple guide to measuring performance might be good here. 3) That inter-process message passing time is constant (on a big multi-core sending a message across cores can be way less efficient than sending across the same core) 4) That thow-away light weight processes should be avoided. 5) That string processing is inefficient (there are even Blog and rants about Erlangs "string problem" - please tell people to avoid tail-appending long lists A ++ LongList is a killer when I/O lists are perfect. 6) That Erlang has a string problem - no it doesn't - Erlang has no strings. It has string literals and lists of integers - and libraries to serialise lists of integers into UTF8 byte streams and so on. Cheers /Joe On Sat, Jun 4, 2016 at 8:13 AM, Bj?rn Gustavsson wrote: > We think that it is time to update the Myths of Erlang Performance in > the Efficiency Guide. Some myths have probably died by now and some > new may have arisen. > > I am open to suggestions for myths to retire (move to a separate > section at very end of the guide) and new myths to add. > > Currently, I think that the following myths definitely should be retired. > > 2.1 Myth: Funs are Slow > > 2.2 Myth: List Comprehensions are Slow > > Are any other myths dead? > > I am considering adding a new myth, something like this: > > "I thought binaries were always faster than lists, so why is binary > matching slower than list matching?" > > Then there should be an example for both binaries and lists, something > like this: > > f(<>) -> > %% Do something with B. > f(T); > f(<<>>) -> > %% Return something. > > The answer should be that for more complicated binary patterns, binary > matching wins, and also that the cost of using binaries vs. lists in > the entire application must be considered. Long lists or many short > lists in many processes will cost more in GC costs; then there are > different costs for sending binaries vs. lists as messages. > > > Are there any more myths I should consider? > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Sat Jun 4 12:52:32 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 4 Jun 2016 12:52:32 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> Message-ID: <5752B2F0.8020209@ninenines.eu> On 06/04/2016 09:41 AM, Bj?rn Gustavsson wrote: > On Sat, Jun 4, 2016 at 9:07 AM, Valentin Micic wrote: >> >> On 04 Jun 2016, at 8:13 AM, Bj?rn Gustavsson wrote: >> >>> "I thought binaries were always faster than lists, so why is binary >>> matching slower than list matching?" >>> >> >> Stated this way, the myth may be confusing to some people (well, it is for me). >> So, what is the myth? Is it that binaries are faster than lists, or that binary matching is slower then list matching? >> > > Yes, the myth needs some polishing. > > Here are some more details of what I had in mind. > > It is often recommended to use binaries (for example in this mailing) > instead of lists. > > Someone read this recommendation, tries it, and is then surprised that > this code: > > f(<>) -> > %% Do something with B. > f(T); > f(<<>>) -> > %% Return something. > > is slower than: > > f([B|T]) -> > %% Do something with B. > f(T); > f([]) -> > %% Return something. > > Fact is, it is hard to beat the speed of lists, and that can be surprising. On the other hand, if your input is a binary, converting + looping might be slower than just looping through the binary. Not to mention the extra garbage created. This section should point this out, otherwise we'll end up with users converting binaries to list "because lists are faster for this". Also I think there are too few cases where lists win, especially in that kind of comparison. You are basically comparing binaries-as-string vs strings, and when using those it's frequent to match against <<"abcdef", T/binary>> forms (vs [$a, $b, $c, $d, $e, $f|T]) and as you pointed out, in that case binaries win. IMO the myth is that list strings are efficient at all when in fact parsing them or even just having them in memory is inefficient. They have their use cases (Unicode related mostly). Of course if the section is about people storing lists of integers in a binary then it makes perfect sense, but I haven't seen that one yet. :-) -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From valentin@REDACTED Sat Jun 4 13:42:18 2016 From: valentin@REDACTED (Valentin Micic) Date: Sat, 4 Jun 2016 13:42:18 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> Message-ID: On 04 Jun 2016, at 9:41 AM, Bj?rn Gustavsson wrote: > On Sat, Jun 4, 2016 at 9:07 AM, Valentin Micic wrote: >> >> On 04 Jun 2016, at 8:13 AM, Bj?rn Gustavsson wrote: >> >>> "I thought binaries were always faster than lists, so why is binary >>> matching slower than list matching?" >>> >> >> Stated this way, the myth may be confusing to some people (well, it is for me). >> So, what is the myth? Is it that binaries are faster than lists, or that binary matching is slower then list matching? >> > > Yes, the myth needs some polishing. > > Here are some more details of what I had in mind. > > It is often recommended to use binaries (for example in this mailing) > instead of lists. It is, indeed, my experience that operations on binaries are considerably slower than functional equivalent operation on lists. I think that things get even worse when one utilizes bit-syntax instead of using bitwise operations on an individual element of the list. However, if the speed of execution is not essential for a success, the binary syntax makes the code far easier to follow compared to the functional equivalent implemented via lists. Or is this another myth? V/ From valentin@REDACTED Sat Jun 4 14:10:35 2016 From: valentin@REDACTED (Valentin Micic) Date: Sat, 4 Jun 2016 14:10:35 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <5752B2F0.8020209@ninenines.eu> References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> <5752B2F0.8020209@ninenines.eu> Message-ID: <0562714D-7D3F-442C-8E37-DD9257AF17ED@pixie.co.za> On 04 Jun 2016, at 12:52 PM, Lo?c Hoguin wrote: > On the other hand, if your input is a binary, converting + looping might be slower than just looping through the binary. Not to mention the extra garbage created. > > This section should point this out, otherwise we'll end up with users converting binaries to list "because lists are faster for this". > > Also I think there are too few cases where lists win, especially in that kind of comparison. You are basically comparing binaries-as-string vs strings, and when using those it's frequent to match against <<"abcdef", T/binary>> forms (vs [$a, $b, $c, $d, $e, $f|T]) and as you pointed out, in that case binaries win. > > IMO the myth is that list strings are efficient at all when in fact parsing them or even just having them in memory is inefficient. They have their use cases (Unicode related mostly). > > Of course if the section is about people storing lists of integers in a binary then it makes perfect sense, but I haven't seen that one yet. :-) > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang Consider the following functional equivalents (A) and (B): (A) twist_tripod( <<>>, <<>>, <<>>, <> ) -> Result; twist_tripod( <>, <>, <>, <> ) -> twist_tripod( RemA, RemB, RemC, <> ) . (B) twist_tripod( [], [], [], Result ) -> Result; twist_tripod( [A|LA], [B|LB], [C|LC], Result ) -> Out_0 = (A band 16#80) bor % a7 ((B band 16#80) bsr 1) bor % b7 ((C band 16#80) bsr 2) bor % c7 ((A band 16#40) bsr 2) bor % a6 ((B band 16#40) bsr 3) bor % b6 ((C band 16#40) bsr 4) bor % c6 ((A band 16#20) bsr 4) bor % a5 ((B band 16#20) bsr 5), % b5 Out_1 = ((C band 16#20) bsl 2) bor % c5 ((A band 16#10) bsl 2) bor % a4 ((B band 16#10) bsl 1) bor % b4 (C band 16#10) bor % c4 (A band 16#08) bor % a3 ((B band 16#08) bsr 1) bor % b3 ((C band 16#08) bsr 2) bor % c3 ((A band 16#04) bsr 2), % a2 Out_2 = ((B band 16#04) bsl 5) bor % b2 ((C band 16#04) bsl 4) bor % c2 ((A band 16#02) bsl 4) bor % a1 ((B band 16#02) bsl 3) bor % b1 ((C band 16#02) bsl 2) bor % c1 ((A band 16#01) bsl 2) bor % a0 ((B band 16#01) bsl 1) bor % b0 (C band 16#01), % c0 twist_tripod( LA, LB, LC, [Out_0, Out_1, Out_2|Result] ) . Believe me when I tell you that (B) is considerably faster than (A) -- I know, as I had to write (B) in order to optimize (A). However, if you look at (A), it seems so much easier to see what is being done. Not so with (B). As it is not always about the speed, maybe the myth is that we use binary matching because it is faster. I think we use it because it is so much more expressive, concise and therefore elegant. V/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Jun 4 15:31:47 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Jun 2016 16:31:47 +0300 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <5752B2F0.8020209@ninenines.eu> References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> <5752B2F0.8020209@ninenines.eu> Message-ID: I can tell a wonderful example of how _leaving_ integers in compressed form in binary could speedup our code from 1000 milliseconds per request to 5 milliseconds But it is is a complicated story because it is about unpacking compressed structure vs working with packed representation. On Sat, Jun 4, 2016 at 1:52 PM, Lo?c Hoguin wrote: > On 06/04/2016 09:41 AM, Bj?rn Gustavsson wrote: > >> On Sat, Jun 4, 2016 at 9:07 AM, Valentin Micic >> wrote: >> >>> >>> On 04 Jun 2016, at 8:13 AM, Bj?rn Gustavsson wrote: >>> >>> "I thought binaries were always faster than lists, so why is binary >>>> matching slower than list matching?" >>>> >>>> >>> Stated this way, the myth may be confusing to some people (well, it is >>> for me). >>> So, what is the myth? Is it that binaries are faster than lists, or >>> that binary matching is slower then list matching? >>> >>> >> Yes, the myth needs some polishing. >> >> Here are some more details of what I had in mind. >> >> It is often recommended to use binaries (for example in this mailing) >> instead of lists. >> >> Someone read this recommendation, tries it, and is then surprised that >> this code: >> >> f(<>) -> >> %% Do something with B. >> f(T); >> f(<<>>) -> >> %% Return something. >> >> is slower than: >> >> f([B|T]) -> >> %% Do something with B. >> f(T); >> f([]) -> >> %% Return something. >> >> Fact is, it is hard to beat the speed of lists, and that can be >> surprising. >> > > On the other hand, if your input is a binary, converting + looping might > be slower than just looping through the binary. Not to mention the extra > garbage created. > > This section should point this out, otherwise we'll end up with users > converting binaries to list "because lists are faster for this". > > Also I think there are too few cases where lists win, especially in that > kind of comparison. You are basically comparing binaries-as-string vs > strings, and when using those it's frequent to match against <<"abcdef", > T/binary>> forms (vs [$a, $b, $c, $d, $e, $f|T]) and as you pointed out, in > that case binaries win. > > IMO the myth is that list strings are efficient at all when in fact > parsing them or even just having them in memory is inefficient. They have > their use cases (Unicode related mostly). > > Of course if the section is about people storing lists of integers in a > binary then it makes perfect sense, but I haven't seen that one yet. :-) > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkbucc@REDACTED Sat Jun 4 15:40:14 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Sat, 4 Jun 2016 09:40:14 -0400 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: <5752da58.a62a8c0a.7641b.ffffd901@mx.google.com> That Erlang is slow for command line utils? Show a simple, working example of a custom boot that loads the minimum modules required. -----Original Message----- From: "Bj?rn Gustavsson" Sent: ?6/?4/?2016 2:13 To: "erlang-questions" Subject: [erlang-questions] Updating Myths of Erlang Performance We think that it is time to update the Myths of Erlang Performance in the Efficiency Guide. Some myths have probably died by now and some new may have arisen. I am open to suggestions for myths to retire (move to a separate section at very end of the guide) and new myths to add. Currently, I think that the following myths definitely should be retired. 2.1 Myth: Funs are Slow 2.2 Myth: List Comprehensions are Slow Are any other myths dead? I am considering adding a new myth, something like this: "I thought binaries were always faster than lists, so why is binary matching slower than list matching?" Then there should be an example for both binaries and lists, something like this: f(<>) -> %% Do something with B. f(T); f(<<>>) -> %% Return something. The answer should be that for more complicated binary patterns, binary matching wins, and also that the cost of using binaries vs. lists in the entire application must be considered. Long lists or many short lists in many processes will cost more in GC costs; then there are different costs for sending binaries vs. lists as messages. Are there any more myths I should consider? /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sat Jun 4 16:07:11 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sat, 4 Jun 2016 10:07:11 -0400 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <20160604083121.GA11334@corelatus.se> References: <575246D4.3080005@aim.com> <20160604083121.GA11334@corelatus.se> Message-ID: <5752E08F.1040902@aim.com> Thanks Matthias and Craig. I see the issue and I've got a work around. BTW, is there a way to truncate a float to a given # of places, not just 0 -- not just a formatted output, but a true truncation? Don On 06/04/2016 04:31 AM, Matthias Lang wrote: >> When I print them via io:format("~nX: ~f Y: ~f ", [X,Y]), >> >> I get: X: 0.000000 Y: -5.000000 >> >> which is correct. >> >> ====== >> >> When I make a tuple of them, add them to a list and print them via: >> io:format("~nL: ~p~n", [L]), >> >> I get: {4.440892098500626e-16,-5.0}, >> >> which is very strange. > Does this clear things up? > > 1> X = 4.44E-16. > 4.44e-16 > 2> io:fwrite("With twiddle-f: ~f With twiddle-p: ~p\n", [X, X]). > With twiddle-f: 0.000000 With twiddle-p: 4.44e-16 > > With the ~f format string, the default precision is 6, which is > why you see 0.000000 in that case. > > Here's a site which talks about the general problem without getting > unnecessarily abstract: http://floating-point-gui.de/ > > Matt From essen@REDACTED Sat Jun 4 20:57:04 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 4 Jun 2016 20:57:04 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <0562714D-7D3F-442C-8E37-DD9257AF17ED@pixie.co.za> References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> <5752B2F0.8020209@ninenines.eu> <0562714D-7D3F-442C-8E37-DD9257AF17ED@pixie.co.za> Message-ID: <57532480.5010708@ninenines.eu> On 06/04/2016 02:10 PM, Valentin Micic wrote: > On 04 Jun 2016, at 12:52 PM, Lo?c Hoguin wrote: > >> On the other hand, if your input is a binary, converting + looping >> might be slower than just looping through the binary. Not to mention >> the extra garbage created. >> >> This section should point this out, otherwise we'll end up with users >> converting binaries to list "because lists are faster for this". >> >> Also I think there are too few cases where lists win, especially in >> that kind of comparison. You are basically comparing >> binaries-as-string vs strings, and when using those it's frequent to >> match against <<"abcdef", T/binary>> forms (vs [$a, $b, $c, $d, $e, >> $f|T]) and as you pointed out, in that case binaries win. >> >> IMO the myth is that list strings are efficient at all when in fact >> parsing them or even just having them in memory is inefficient. They >> have their use cases (Unicode related mostly). >> >> Of course if the section is about people storing lists of integers in >> a binary then it makes perfect sense, but I haven't seen that one yet. :-) >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang > > Consider the following functional equivalents *(A)* and *(B)*: > > *(A)* > twist_tripod( <<>>, <<>>, <<>>, <> ) -> Result; > twist_tripod( < RemA/binary>>, > < RemB/binary>>, > < RemC/binary>>, > <> ) > -> > twist_tripod( RemA, RemB, RemC, < A1:1, B1:1, C1:1, > A2:1, B2:1, C2:1, > A3:1, B3:1, C3:1, > A4:1, B4:1, C4:1, > A5:1, B5:1, C5:1, > A6:1, B6:1, C6:1, > A7:1, B7:1, C7:1, Result/binary>> ) > . > > *(B)* > twist_tripod( [], [], [], Result ) -> Result; > twist_tripod( [A|LA], [B|LB], [C|LC], Result ) > -> > Out_0 = (A band 16#80) bor % a7 > ((B band 16#80) bsr 1) bor % b7 > ((C band 16#80) bsr 2) bor % c7 > ((A band 16#40) bsr 2) bor % a6 > ((B band 16#40) bsr 3) bor % b6 > ((C band 16#40) bsr 4) bor % c6 > ((A band 16#20) bsr 4) bor % a5 > ((B band 16#20) bsr 5), % b5 > Out_1 = ((C band 16#20) bsl 2) bor % c5 > ((A band 16#10) bsl 2) bor % a4 > ((B band 16#10) bsl 1) bor % b4 > (C band 16#10) bor % c4 > (A band 16#08) bor % a3 > ((B band 16#08) bsr 1) bor % b3 > ((C band 16#08) bsr 2) bor % c3 > ((A band 16#04) bsr 2), % a2 > > Out_2 = ((B band 16#04) bsl 5) bor % b2 > ((C band 16#04) bsl 4) bor % c2 > ((A band 16#02) bsl 4) bor % a1 > ((B band 16#02) bsl 3) bor % b1 > ((C band 16#02) bsl 2) bor % c1 > ((A band 16#01) bsl 2) bor % a0 > ((B band 16#01) bsl 1) bor % b0 > (C band 16#01), % c0 > > twist_tripod( LA, LB, LC, [Out_0, Out_1, Out_2|Result] ) > . > > Believe me when I tell you that *(B)* is considerably faster than *(A) > -- *I know, as I had to write *(B)* in order to optimize *(A)*. > However, if you look at *(A)*, it seems so much easier to see what is > being done. Not so with *(B)*. In this particular case I would be willing to bet that it's not so much because of lists, but because you are using band/bsr/bor. These make everything go faster. In turn, there's less variables, less garbage created and so on. If you rewrote your list function with binaries, you would probably get a large boost too. Of course in this particular case you also fall into the parameters Bjorn pointed out and lists would ultimately win (plus, no binary matching optimization here). But that difference is probably minor compared to the rest of the changes. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From tuncer.ayaz@REDACTED Sat Jun 4 22:33:06 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 4 Jun 2016 22:33:06 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On 4 June 2016 at 08:13, Bj?rn Gustavsson wrote: > 2.1 Myth: Funs are Slow What exactly? Calling a fun (object) or creating one? Isn't the time to construct fun objects considerable enough that one shouldn't do it in, say, a loop? From zxq9@REDACTED Sun Jun 5 06:11:30 2016 From: zxq9@REDACTED (zxq9) Date: Sun, 05 Jun 2016 13:11:30 +0900 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <5752E08F.1040902@aim.com> References: <575246D4.3080005@aim.com> <20160604083121.GA11334@corelatus.se> <5752E08F.1040902@aim.com> Message-ID: <2703139.QQ6THjvubS@changa> On 2016?6?4? ??? 10:07:11 Donald Steven wrote: > Thanks Matthias and Craig. I see the issue and I've got a work around. > BTW, is there a way to truncate a float to a given # of places, not just > 0 -- not just a formatted output, but a true truncation? Keep in mind that in this case you don't want mere truncation -- what if your teeny, tiny value winds up being teeny, tiny on the negative side of zero? You need a test to check for whether a value is "close enough" to a given value to be considered a valid approximation of it for your case, and have a way to declare how close "close enough" means. That's not truncation. You want to compare to 5.0, for example. If 5.0000000000001 is close enough then 4.9999999999999 is also. Unfortunately floats are just hard to deal with (in every language), but at least floats have strictly defined rules that limit how hard they are. The first thing to consider is whether you should really be doing fixed-point math, or if floats are good enough for your use. Assuming floats really are a good fit, something like this could help: approximate(Precision) -> fun(Z, Z) -> true; (Z, Q) -> (Z - Precision =< Q) and (Q =< Z + Precision) end. 1> Approx = numbers:approximate(0.00001). #Fun 2> Approx(1, 2). false 3> Approx(5, 5). true 4> Approx(-5.2, -5.2). true 5> Approx(10, 10.0000000001). true 6> Approx(10, 10.1). false 7> Approx(0.0, 4.440892098500626e-16). true 8> Approx(0.0, -4.440892098500626e-16). true This is probably an inefficient implementation -- I've not taken the time to really examine it carefully, but it should be a pretty clear what the intent is. You provide a value that represents how close a value must be to be considered approximately equal and you get back a function that will check if the input values are that close. (There are probably some float corner cases that can pop up here, and its probably more efficient to define precision as a Power base X and slide things around to compare ranges, etc. but the implementation isn't the point and I have no idea how any of that would actually hit the underlying machine without writing it in assembler, so clarity wins in the majority case.) Surely there is some library out there that already provides something like this. -Craig From zxq9@REDACTED Sun Jun 5 06:25:02 2016 From: zxq9@REDACTED (zxq9) Date: Sun, 05 Jun 2016 13:25:02 +0900 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <2703139.QQ6THjvubS@changa> References: <575246D4.3080005@aim.com> <5752E08F.1040902@aim.com> <2703139.QQ6THjvubS@changa> Message-ID: <8778039.cIz3klLbi9@changa> On 2016?6?5? ??? 13:11:30 zxq9 wrote: > On 2016?6?4? ??? 10:07:11 Donald Steven wrote: > > Thanks Matthias and Craig. I see the issue and I've got a work around. > > BTW, is there a way to truncate a float to a given # of places, not just > > 0 -- not just a formatted output, but a true truncation? > > approximate(Precision) -> > fun(Z, Z) -> true; > (Z, Q) -> (Z - Precision =< Q) and (Q =< Z + Precision) > end. I forgot to mention something... in your particular case you may really be wanting to convert to a specific value instead of just compare, in that case a small change is more direct: force_fit(Precision) -> Approx = approximate(Precision), fun(Z, Q) -> case Approx(Z, Q) of true -> Z; false -> Q end end. 1> Rough = numbers:force_fit(0.00001). #Fun 2> Rough(0.0, 4.440892098500626e-16). 0.0 3> Rough(0.0, 0.1). 0.1 Obviously this would be playing with fire on intermediate values (*much* more dangerous than the usual caveats that apply to iterative processing with floats). But it may be closer to the behavior you are looking for -- conceptually, anyway. I have no idea what you're really doing. -Craig From bjorn@REDACTED Sun Jun 5 07:46:38 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 5 Jun 2016 07:46:38 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On Sat, Jun 4, 2016 at 10:33 PM, Tuncer Ayaz wrote: > On 4 June 2016 at 08:13, Bj?rn Gustavsson wrote: > >> 2.1 Myth: Funs are Slow > > What exactly? Calling a fun (object) or creating one? Isn't the time > to construct fun objects considerable enough that one shouldn't > do it in, say, a loop? That was not the point with the myth at all, and I thinks that proves that it is time the myth is retired. Funs were really slow before R6B, at least an order of magnitude slower than today. The virtual machines (neither JAM nor BEAM) had any native support for funs. The function modules_lambdas() in each module had a clause for each fun defined in the module. The fun terms themselves were represented as 5-tuple. When a fun was called, the tuple had to be matched to extract the module that defined the fun, the arguments had to be packed into a tuple, and modules_lambdas() had to be applied. (Tony might remember more details about the implementation.) /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Sun Jun 5 07:58:07 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 5 Jun 2016 07:58:07 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <5752B2F0.8020209@ninenines.eu> References: <357164F2-75EC-4423-A5BD-2FEF1AD812D2@pixie.co.za> <5752B2F0.8020209@ninenines.eu> Message-ID: On Sat, Jun 4, 2016 at 12:52 PM, Lo?c Hoguin wrote: [...] > On the other hand, if your input is a binary, converting + looping might be > slower than just looping through the binary. Not to mention the extra > garbage created. > > This section should point this out, otherwise we'll end up with users > converting binaries to list "because lists are faster for this". Yes, I intended to point that out. But I think yours and Valentin's feedback shows that this issue is too complex and subtle to be meaningfully boiled down into a single myth. We might risk sending out the wrong message. So I will probably not attempt to add that myth. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From mononcqc@REDACTED Sun Jun 5 14:19:16 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 5 Jun 2016 08:19:16 -0400 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: <20160605121915.GB15183@ferdmbp.local> On 06/04, Bj?rn Gustavsson wrote: >We think that it is time to update the Myths of Erlang Performance in >the Efficiency Guide. Some myths have probably died by now and some >new may have arisen. > >I am open to suggestions for myths to retire (move to a separate >section at very end of the guide) and new myths to add. One I persistently see, and which is likely a consequence of how we teach Erlang and set up exercises, is that tail recursion is always the best choice and likely faster than body recursion. In cases where all your function does is build a new list (or any other accumulator whose size is equivalent to the number of iterations and hence the stack) such as map/2 over nearly any data structure or say zip/2 over lists, body recursion may not only be simpler, but also faster and save memory over time. I had covered this in an old blog post at http://ferd.ca/erlang-s-tail-recursion-is-not-a-silver-bullet.html From ben@REDACTED Sun Jun 5 14:29:34 2016 From: ben@REDACTED (Ben Adams) Date: Sun, 05 Jun 2016 08:29:34 -0400 Subject: [erlang-questions] Walkie Talkie app with Erlang backend In-Reply-To: References: Message-ID: <0ac4f6ed102b13b095fe72af11a73953@smtp.hushmail.com> Take a look at Matt protocol. -- Ben Adams On Jun 3, 2016, 3:18 AM, at 3:18 AM, Rizwan Khan wrote: >I am thinking of creating a clone for Voxer which is a walkie talkie >app >which uses data for transferring the messages. It uses nodejs at the >backend and I think erlang would be a better fit for this. > >What do you guys think? > >And rather than writing a new backend, cant I do the same thing with >and >XMPP server? > >Suggestions needed. > >Thanks, > > > >Rizwan Khan > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben@REDACTED Sun Jun 5 14:41:26 2016 From: ben@REDACTED (Ben Adams) Date: Sun, 05 Jun 2016 08:41:26 -0400 Subject: [erlang-questions] Walkie Talkie app with Erlang backend In-Reply-To: <0ac4f6ed102b13b095fe72af11a73953@smtp.hushmail.com> References: <0ac4f6ed102b13b095fe72af11a73953@smtp.hushmail.com> Message-ID: <3ca04688e071b30a425d9679889c6442@smtp.hushmail.com> MQTT "phone auto correct" You can pub/sub with push. -- Ben Adams On Jun 5, 2016, 8:38 AM, at 8:38 AM, Ben Adams wrote: >Take a look at Matt protocol. > >-- Ben Adams > > > >On Jun 3, 2016, 3:18 AM, at 3:18 AM, Rizwan Khan >wrote: >>I am thinking of creating a clone for Voxer which is a walkie talkie >>app >>which uses data for transferring the messages. It uses nodejs at the >>backend and I think erlang would be a better fit for this. >> >>What do you guys think? >> >>And rather than writing a new backend, cant I do the same thing with >>and >>XMPP server? >> >>Suggestions needed. >> >>Thanks, >> >> >> >>Rizwan Khan >> >> >>------------------------------------------------------------------------ >> >>_______________________________________________ >>erlang-questions mailing list >>erlang-questions@REDACTED >>http://erlang.org/mailman/listinfo/erlang-questions > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sun Jun 5 17:55:25 2016 From: tony@REDACTED (Tony Rogvall) Date: Sun, 5 Jun 2016 17:55:25 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: > On 5 jun 2016, at 07:46, Bj?rn Gustavsson wrote: > > On Sat, Jun 4, 2016 at 10:33 PM, Tuncer Ayaz wrote: >> On 4 June 2016 at 08:13, Bj?rn Gustavsson wrote: >> >>> 2.1 Myth: Funs are Slow >> >> What exactly? Calling a fun (object) or creating one? Isn't the time >> to construct fun objects considerable enough that one shouldn't >> do it in, say, a loop? > > That was not the point with the myth at all, and I thinks that proves > that it is time the myth is retired. > > Funs were really slow before R6B, at least an order of magnitude > slower than today. The virtual machines (neither JAM nor BEAM) had any > native support for funs. The function modules_lambdas() in each module > had a clause for each fun defined in the module. The fun terms > themselves were represented as 5-tuple. When a fun was called, the > tuple had to be matched to extract the module that defined the fun, > the arguments had to be packed into a tuple, and modules_lambdas() had > to be applied. (Tony might remember more details about the > implementation.) > We liked sweet syntactic sugar a lot more back then. ;-) But yes. It was more like a proof of concept. Fun objects looked like {?fun?, Module, Index, Uniq, Free} where Index was a local function index, Uniq a ?unique? number and Free was the closure. call to a function F(1,2,3) was transformed into apply(F, [1,2,3]) which in turn was transformed into a call to apply_lambda(F, [1,2,3]) if F = fun(X,Y,Z) -> A+X+Y+Z+B end ( A and B in closure ) then the function object F was transformed into something like {?fun?,mmm,1,1234,{A,B}} so the call F(1,2,3) would then look like: F = {?fun?, mmm, 1, 1234, {A,B}}, apply_lambda(F, [1,2,3]) the per module (mmm in this case) generated apply_lambda for F may have looked something like: -module(mmm). apply_lambda({fun, mmm, Index, Uniq, Free}, Args) -> module_lambdas(I, Uniq, Args, Free); ... So apply_lambda was just a dispatch for local and remote functions called from within a module. And module_lambdas: module_lambdas(1, 1234, [X,Y,Z], {A,B}) -> A+X+Y+Z+B; ? other functions But that was a loooooong time ago :-) /Tony > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jesper.louis.andersen@REDACTED Sun Jun 5 18:57:22 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 5 Jun 2016 18:57:22 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On Sun, Jun 5, 2016 at 5:55 PM, Tony Rogvall wrote: > We liked sweet syntactic sugar a lot more back then. ;-) Closure conversion in the compiler in order to support a runtime with no closure support. For a functional language where closures are central. How quaint :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From icfp.publicity@REDACTED Mon Jun 6 02:52:42 2016 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Sun, 5 Jun 2016 17:52:42 -0700 Subject: [erlang-questions] SIGPLAN Programming Languages Mentoring Workshop @ ICFP Message-ID: SIGPLAN Programming Languages Mentoring Workshop @ ICFP Nara, Japan (co-located with ICFP 2016) Sunday, September 18th, 2016 http://conf.researchr.org/track/icfp-2016/PLMW-ICFP-2016/ We are pleased to invite students interested in functional programming research to the programming languages mentoring workshop at ICFP. The goal of this workshop is to introduce senior undergraduate and early graduate students to research topics in functional programming as well as provide career mentoring advice. We have recruited leaders from the functional programming community to provide overviews of current research topics and give students valuable advice about how to thrive in graduate school, search for a job, and cultivate habits and skills that will help them in research careers. This workshop is part of the activities surrounding ICFP, the International Conference on Functional Programming, and takes place the day before the main conference. One goal of the workshop is to make the ICFP conference more accessible to newcomers and we hope participants will stay through the entire conference. Through the generous donation of our sponsors, we are able to provide travel scholarships to fund student participation. These travel scholarships will cover reasonable travel expenses (airfare and hotel) for attendance at both the workshop and the main three days of the ICFP conference. The workshop is open to all. Students with alternative sources of funding for their travel and registration fees are welcome. In particular, many student attendance programs provide full or partial travel funding for students to attend ICFP 2016, including the ACM Student Research Competition. More information about student attendance programs at ICFP is available: http://conf.researchr.org/home/icfp-2016 Application for Travel Support: The travel funding application is available from the PLMW webpage. The deadline for full consideration of funding is July 1st, 2016. Selected participants will be notified by July 15th. Organizers: Amal Ahmed, Northeastern University Robby Findler, Northwestern University Atsushi Igarashi, Kyoto University From bjorn@REDACTED Mon Jun 6 11:04:17 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 6 Jun 2016 11:04:17 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On Sat, Jun 4, 2016 at 11:43 AM, Joe Armstrong wrote: > 1) That using cases/if/multiple entry points is somehow more efficient > > I've see several posts suggesting that rewriting functions with a > single entry point > conta multiple entry points and so on will be magically more efficient > - when in fact > after compilation the code is pretty much identical. > Good suggestion. I think I will incorporate that into the existing myth: Myth: Use "_" to Speed Up Your Program When a Variable is Not Used That would describe the more general myth that trivial transformations of your source code would make it more efficient. > 2) That you can predict performance without doing measurements. > A simple guide to measuring performance might be good here. I have tried to make the myths Erlang-specific. Also, in the introduction to the Efficiency Guide there is the advice to profile the code before optimising: http://erlang.org/doc/efficiency_guide/introduction.html > 5) That string processing is inefficient (there are even Blog and rants about > Erlangs "string problem" - please tell people to avoid tail-appending > long lists A ++ LongList is a killer when I/O lists are perfect. > > 6) That Erlang has a string problem - no it doesn't - Erlang has no strings. > It has string literals and lists of integers - and libraries to > serialise lists of > integers into UTF8 byte streams and so on. There already is the myth: Myth: Strings are Slow But I can probably improve the text based on your suggestion. Thanks for the suggestions. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Mon Jun 6 11:08:46 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 6 Jun 2016 11:08:46 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: <20160605121915.GB15183@ferdmbp.local> References: <20160605121915.GB15183@ferdmbp.local> Message-ID: On Sun, Jun 5, 2016 at 2:19 PM, Fred Hebert wrote: > One I persistently see, and which is likely a consequence of how we teach > Erlang and set up exercises, is that tail recursion is always the best > choice and likely faster than body recursion. > > In cases where all your function does is build a new list (or any other > accumulator whose size is equivalent to the number of iterations and hence > the stack) such as map/2 over nearly any data structure or say zip/2 over > lists, body recursion may not only be simpler, but also faster and save > memory over time. > > I had covered this in an old blog post at > http://ferd.ca/erlang-s-tail-recursion-is-not-a-silver-bullet.html Thanks! That shows this existing myth is still relevant: Myth: Tail-Recursive Functions are Much Faster Than Recursive Functions But I think that I can probably improve on the text based on your feedback and blog post. Perhaps I should link to your blog post. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Mon Jun 6 11:21:42 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 6 Jun 2016 11:21:42 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: Thanks for all the feedback so far. I realise now that I should have linked to the current version of the myths. Here is the link a little bit late: http://erlang.org/doc/efficiency_guide/myths.html Based on the feedback, my current plan is that updated myths should be like this: 1. Myth: Tail-Recursive Functions are Much Faster Than Recursive Functions (Updated inspired by Fred's feedback, probably downplaying the technical details in the original text.) 2. Myth: Operator "++" is Always Bad 3. Myth: Strings are Slow (Updated with some more text inspired by Joe's feedback.) 4. Myth: Repairing a Dets File is Very Slow 5. Myth: BEAM is a Stack-Based Byte-Code Virtual Machine (and Therefore Slow) 6. Myth: Use "_" to Speed Up Your Program When a Variable is Not Used (Generalised to any trivial transformation of the source code inspired by Joe's example.) 7. Myth: NIFs Always Speeds Up Your Application (New, suggested by Max Lapshin.) I might not have time to do the updates for OTP 19; it might appear have to wait to 19.1. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From denc716@REDACTED Mon Jun 6 13:00:05 2016 From: denc716@REDACTED (derek) Date: Mon, 6 Jun 2016 04:00:05 -0700 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: http://erlang.org/doc/efficiency_guide/myths.html some myths there are talking about R6B R7B stuff, those could be true misunderstandings when R6B R7B were still around, but nowadays I believe such old releases should be very rare? in practice I am not hearing anything older than R15B; so those old misunderstandings should have been clarified? and can be removed? or move to an archive if still want to keep From t6sn7gt@REDACTED Mon Jun 6 16:02:51 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 6 Jun 2016 10:02:51 -0400 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <8778039.cIz3klLbi9@changa> References: <575246D4.3080005@aim.com> <5752E08F.1040902@aim.com> <2703139.QQ6THjvubS@changa> <8778039.cIz3klLbi9@changa> Message-ID: <5755828B.5090906@aim.com> This is most helpful. Thanks Craig. Don On 06/05/2016 12:25 AM, zxq9 wrote: > On 2016?6?5? ??? 13:11:30 zxq9 wrote: >> On 2016?6?4? ??? 10:07:11 Donald Steven wrote: >>> Thanks Matthias and Craig. I see the issue and I've got a work around. >>> BTW, is there a way to truncate a float to a given # of places, not just >>> 0 -- not just a formatted output, but a true truncation? >> approximate(Precision) -> >> fun(Z, Z) -> true; >> (Z, Q) -> (Z - Precision =< Q) and (Q =< Z + Precision) >> end. > I forgot to mention something... in your particular case you may really > be wanting to convert to a specific value instead of just compare, in that > case a small change is more direct: > > force_fit(Precision) -> > Approx = approximate(Precision), > fun(Z, Q) -> > case Approx(Z, Q) of > true -> Z; > false -> Q > end > end. > > 1> Rough = numbers:force_fit(0.00001). > #Fun > 2> Rough(0.0, 4.440892098500626e-16). > 0.0 > 3> Rough(0.0, 0.1). > 0.1 > > Obviously this would be playing with fire on intermediate values (*much* > more dangerous than the usual caveats that apply to iterative processing > with floats). But it may be closer to the behavior you are looking for -- > conceptually, anyway. I have no idea what you're really doing. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue Jun 7 02:56:58 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 7 Jun 2016 12:56:58 +1200 Subject: [erlang-questions] math ceil floor In-Reply-To: References: Message-ID: On 4/06/16 9:37 AM, Samuel wrote: [since Erlang has bignums, is there really a use case for ffloor/fceil?] It's a good question. Consider, though st> 1.234567892345d308 floor 1234567892345000045994571678608401106710733 5829204606430696621556869305373762371218809 5752361144544803343115156790092425340877711 3686294033518655059904033307189889822114268 2599325972381243272372510471571815988180764 3319328717607302344826137591755759767023287 7560828068588895392676568601198325562537703 49019136 Is this actually a *good* way to represent the answer? How should this deal with -0, infinities, or NaNs? Smalltalk and Lisp have both. From ok@REDACTED Tue Jun 7 03:18:02 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 7 Jun 2016 13:18:02 +1200 Subject: [erlang-questions] Strange float / tuple problem In-Reply-To: <5752E08F.1040902@aim.com> References: <575246D4.3080005@aim.com> <20160604083121.GA11334@corelatus.se> <5752E08F.1040902@aim.com> Message-ID: <7c44f6d4-090a-f60c-b18a-d75dde20793c@cs.otago.ac.nz> On 5/06/16 2:07 AM, Donald Steven wrote: > Thanks Matthias and Craig. I see the issue and I've got a work > around. BTW, is there a way to truncate a float to a given # of > places, not just 0 -- not just a formatted output, but a true truncation? I think you may need something more fundamental than a workaround. For what it's worth, DBL_EPSILON for IEEE doubles is 2.2204460492503130808473E-16. Now DBL_EPSILON is the smallest value of e such that 1.0+e /= 1.0, so the 4.4...e-16 value you reported is 2 units in the last place for a number close to 1 or 1 unit in the last place for a number close to 2, or to put it another way, one or two rounding errors. We are definitely dealing with some sort of calculation here, not a literal 0.0, and it would help a lot to show us that calculation. Since IEEE arithmetic gives you the closest representable value for the result of any one operation, I think you'll appreciate that truncation is, as a general rule, going to INCREASE errors. The best you can do to truncate a floating point number to a given number of decimal places is truncate(X*10**N)/10**N. Needless to say, the result is NOT exact, because almost all decimal fractions are not exactly representable in binary arithmetic. Why do you think that truncation rather than rounding is right here? Calculations where you expect exact answers should be done in integer arithmetic, if possible. From grimmer.matthias@REDACTED Tue Jun 7 13:31:12 2016 From: grimmer.matthias@REDACTED (Matthias Grimmer) Date: Tue, 7 Jun 2016 13:31:12 +0200 Subject: [erlang-questions] Call for Participation: 3rd Virtual Machine Meetup, September 1-2, Lugano, Switzerland Message-ID: # Virtual Machine Meetup ## Call for Participation at the VMM2016 The 3rd Virtual Machine Meetup (VMM'16) is a venue for discussing the latest research and developments in the area of managed language execution. It will be held on 1st and 2nd of September at the Universit? della Svizzera italiana (USI), Lugano, Switzerland and is part of the Managed Languages & Runtimes Week 2016 (http://manlang16.inf.usi.ch/ , other colocated events are PPPJ'16 and JTRES'16, room Auditorium from 9am - 5pm). We welcome presentations of new research results, experience reports, as well as position statements that can lead to interesting discussions. Topics include, but are not limited to: - Programming language design - Dynamic and static program analysis - Compiler construction - Managed runtime architectures - Data processing engines - Distributed execution environments To participate, please email thomas.wuerthinger@REDACTED stating your wish to attend, and your name and affiliation as you wish to have them on your name badge. There are limited participant slots due to the constraints of the room, so please register early, and by July 20th the latest. If you would like to give a presentation, please email Thomas with a title (max. 100 characters) and abstract (max. 400 characters). We may ask for additional information from you before making the program decision. Presentation slots are either 30 minutes (long) or 15 minutes (short) including Q/A. Important dates: - Submissions: July 10, 2016 - Author notification: July 17, 2016 - Registration for participation: July 20, 2016 - Virtual machine Meetup: Sep 1st + 2nd 2016 at USI Lugano - Social Event: Sep 3rd 2016, optional Program committee: - Stefan Marr, JKU Linz, Austria - Matthias Grimmer, JKU Linz, Austria - Laurence Tratt, King's College London - Thomas Wuerthinger, Oracle Labs Switzerland As an optional social event, we will plan this year for Saturday the 3rd of September a trip to the Lake Como - a gorgeous lake in Italy close to Lugano. Please let us know whether you intend to participate for planning purposes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From derekbrown121@REDACTED Tue Jun 7 16:14:57 2016 From: derekbrown121@REDACTED (Derek Brown) Date: Tue, 7 Jun 2016 10:14:57 -0400 Subject: [erlang-questions] Updating Myths of Erlang Performance Message-ID: This came up a few weeks ago on the Erlang Slack, and Bj?rn-Egil Dahlberg of the OTP team commented that updating the guide ( http://erlang.org/doc/efficiency_guide/myths.html) is on his short-term todo list. > We think that it is time to update the Myths of Erlang Performance in > the Efficiency Guide. Some myths have probably died by now and some > new may have arisen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Tue Jun 7 17:00:46 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Tue, 7 Jun 2016 17:00:46 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: <5756E19E.5000303@ericsson.com> On 2016-06-07 16:14, Derek Brown wrote: > This came up a few weeks ago on the Erlang Slack, and Bj?rn-Egil > Dahlberg of the OTP team commented that updating the guide > (http://erlang.org/doc/efficiency_guide/myths.html) is on his > short-term todo list. I delegated .. sort of =) // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From Aleksander.Nycz@REDACTED Tue Jun 7 21:32:14 2016 From: Aleksander.Nycz@REDACTED (Aleksander Nycz) Date: Tue, 7 Jun 2016 21:32:14 +0200 Subject: [erlang-questions] Efficient way to store message lists in mnesia In-Reply-To: References: Message-ID: Hello, Try this: Users table: {userId, firstMessageId, lastMessageId} Messages table: {MessageId, prevMessageId, nextMessageId, Message} Insert 1st message for user 1000: Users table: {1000, 1, 1} Message table: {1, undefined, undefined, "message1"} Insert 2nd message for user 1000: Users table: {1000, 1, 2} Message table: {1, undefined, 2, "message1"} {2, 1, undefined, "message2"} And so on... You can also agregate messages (yearly/monthly/daily): {{UserId, Year}, firstMessageId, lastMessageId} Regards Aleksander W dniu 2016-06-03 o 19:08, Khitai Pang pisze: > Hi, > > I need to store a list of unread messages for every user id. The order > of the messages needs to be preserved. Currently I store the messages > in a list as the value of the user id key in a table of type set: > > {user_id, [message]} > > The table is a disc_copies table. > > Whenever someone sends a message to a user, the message is prepended to > the head of the message list of the user. > > Now my concern is, adding a new message to the list requires reading and > writing the whole list, which can be pretty slow if the list is very > long, i.e. thousands of messages. > > There is another way to do this, which uses a table of type bag and > composite keys: > > {user_id, timestamp1, message1} > {user_id, timestamp2, message2} > {user_id, timestamp3, message3} > ... > > But I don't think this is any better because I heard that mnesia bag > tables performance is very bad: > http://erlang.org/pipermail/erlang-questions/2011-March/057044.html > > Are there other ways to store lists of messages in mnesia where > reading/updating the message lists can be very efficient? > > > -Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4936 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From be.dmitry@REDACTED Wed Jun 8 01:26:30 2016 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Wed, 08 Jun 2016 09:26:30 +1000 Subject: [erlang-questions] Efficient way to store message lists in mnesia In-Reply-To: References: Message-ID: <1FCBA854-68F2-45B2-B8F8-8BF160C0FF73@gmail.com> Alexander, thanks for a great example of a double-linked list implementation in a database. As we can see one insert operation requires 3 write operations in the database in this case. Anyway this is still a constant factor applied to a normal write operation. Wear we never heard is how the messages are read and if they are removed from the database. If as in most chat applications they need to be read and deleted in bulk, this approach might be tedious as you need to read the messages one by one to find out the next id. I suggest to try using table of type bag and see if performance it provides is enough for all the task use cases. I believe that implementation would be easiest to read and maintain. Another approach might be to use ordered set with records of following structure: {{UserId, Timestamp}, Message}. On 8 June 2016 5:32:14 AM AEST, Aleksander Nycz wrote: >Hello, > >Try this: > >Users table: > >{userId, firstMessageId, lastMessageId} > >Messages table: > >{MessageId, prevMessageId, nextMessageId, Message} > > >Insert 1st message for user 1000: > >Users table: > >{1000, 1, 1} > >Message table: > >{1, undefined, undefined, "message1"} > > >Insert 2nd message for user 1000: > >Users table: > >{1000, 1, 2} > >Message table: > >{1, undefined, 2, "message1"} > >{2, 1, undefined, "message2"} > >And so on... > >You can also agregate messages (yearly/monthly/daily): > >{{UserId, Year}, firstMessageId, lastMessageId} > > >Regards > >Aleksander > > >W dniu 2016-06-03 o 19:08, Khitai Pang pisze: >> Hi, >> >> I need to store a list of unread messages for every user id. The >order >> of the messages needs to be preserved. Currently I store the >messages >> in a list as the value of the user id key in a table of type set: >> >> {user_id, [message]} >> >> The table is a disc_copies table. >> >> Whenever someone sends a message to a user, the message is prepended >to >> the head of the message list of the user. >> >> Now my concern is, adding a new message to the list requires reading >and >> writing the whole list, which can be pretty slow if the list is very >> long, i.e. thousands of messages. >> >> There is another way to do this, which uses a table of type bag and >> composite keys: >> >> {user_id, timestamp1, message1} >> {user_id, timestamp2, message2} >> {user_id, timestamp3, message3} >> ... >> >> But I don't think this is any better because I heard that mnesia bag >> tables performance is very bad: >> http://erlang.org/pipermail/erlang-questions/2011-March/057044.html >> >> Are there other ways to store lists of messages in mnesia where >> reading/updating the message lists can be very efficient? >> >> >> -Khitai >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Best wishes, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Jun 8 06:22:00 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 8 Jun 2016 16:22:00 +1200 Subject: [erlang-questions] Efficient way to store message lists in mnesia In-Reply-To: <1FCBA854-68F2-45B2-B8F8-8BF160C0FF73@gmail.com> References: <1FCBA854-68F2-45B2-B8F8-8BF160C0FF73@gmail.com> Message-ID: On 8/06/16 11:26 AM, Dmitry Belyaev wrote: > Alexander, thanks for a great example of a double-linked list > implementation in a database. > > As we can see one insert operation requires 3 write operations in the > database in this case. Anyway this is still a constant factor applied > to a normal write operation. I can't recall, did this thread start with a presentation of - message sizes - rates at which messages are produced and consumed - number of registered and of active users - quality issues such as what proportion of messages may be lost that kind of stuff? I mean that's what tells us how many writes per message are a problem. It's also important to distinguish between *worst-case* performance and *amortised* performance. A message will go into the doubly linked list once (at O(1) cost) and come out of the doubly linked list once (at O(1) cost) so the amortised cost per message of the doubly linked list is O(1) per message. It's an unpleasantly high constant factor, but it is a constant factor. Reading and deleting a batch of k messages is then O(k), so a burst of stuff can happen at one time, but reading is going to take O(k) time anyway, so we're talking about a matter of constant factor. There are possible hybrid data structures where you choose a batch size B and buffer recent messages in memory until there are B of them, and run a doubly linked list of batches. From khitai.pang@REDACTED Wed Jun 8 09:33:20 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Wed, 8 Jun 2016 07:33:20 +0000 Subject: [erlang-questions] Efficient way to store message lists in mnesia In-Reply-To: References: <1FCBA854-68F2-45B2-B8F8-8BF160C0FF73@gmail.com> Message-ID: Thank you all for your input and let me provide more background of my question. We already have a messaging app with about 200,000 active users, the app uses MQTT publish/subscribe for messaging. Now we plan to change to a pure erlang solution with customized messaging protocol. Here is the idea, and please forgive me if I am providing too much unnecessary details: Every user can login, send and receive messages from a master client (mobile phone) and a slave client (iPad, Android pad, Windows, MAC), master and slave clients of the same user can be online at the same time. For every user, the master client must eventually have the full message history, i.e. outgoing messages sent by the user's slave client and all incoming messages when the master client is offline must be synced to the master client when it connects. When both clients of a user are online, incoming messages and sent to both clients, and outgoing messages from either client are synced to the other client. Messages which haven't been acked by master stays in the list. Messages that have been acked by both master and slave are immediately removed from the list. Messages that are acked by master but not slave also stays in the list, but the list is limited to the most recent 10 messages. Server keeps a message list for every user. The list has two non_neg_integer() versions: start_version and end_version, both starts from 1. The server also keeps track of the versions acked by master/slave. Every message has a timestamp of seconds from epoch. Messages should be kept in order by the time it's sent/received on server, but the order should not depend on timestamp since multiple messages can happen in the same second. Every time a user sends or receives a message, the message is inserted to the head of the user's message list, and end_version is bumped (+1). When a user client connects to the server, it sends its own end_version of its message list to the server, the server then compares it with the start_version and end_version in db, and sends to the client all messages that the client doesn't have as well as a new end_version. Receiving the messages, the client then sends ack with the new end_version. About 70% of the messages are small text messages, typically less than 48 bytes, 20% of the messages are audio messages typically less than 10k bytes, the other 10% are thumbnails less than 50k bytes. On average, every user sends 50 messages every day. No data on master/slave client usage because currently we only have mobile phone clients. My idea is to use two set type table: {user_id, start_version, end_version, [messages]} {user_id, master_acked_version, slave_acked_version} It may look pretty naive, anyway, this is my first erlang project and also my first messaging app. Any advices are appreciated! Regards, Khitai On 2016/6/8 12:22, Richard A. O'Keefe wrote: > On 8/06/16 11:26 AM, Dmitry Belyaev wrote: >> Alexander, thanks for a great example of a double-linked list >> implementation in a database. >> >> As we can see one insert operation requires 3 write operations in the >> database in this case. Anyway this is still a constant factor applied >> to a normal write operation. > I can't recall, did this thread start with a presentation of > - message sizes > - rates at which messages are produced and consumed > - number of registered and of active users > - quality issues such as what proportion of messages may be lost > that kind of stuff? I mean that's what tells us how many writes > per message are a problem. > > It's also important to distinguish between *worst-case* performance and > *amortised* performance. A message will go into the doubly linked list > once (at O(1) cost) and come out of the doubly linked list once (at > O(1) cost) > so the amortised cost per message of the doubly linked list is O(1) per > message. It's an unpleasantly high constant factor, but it is a constant > factor. Reading and deleting a batch of k messages is then O(k), so a > burst of stuff can happen at one time, but reading is going to take O(k) > time anyway, so we're talking about a matter of constant factor. > > There are possible hybrid data structures where you choose a batch > size B and buffer recent messages in memory until there are B of them, > and run a doubly linked list of batches. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tuncer.ayaz@REDACTED Wed Jun 8 10:49:46 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 8 Jun 2016 10:49:46 +0200 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: On 5 June 2016 at 07:46, Bj?rn Gustavsson wrote: > On Sat, Jun 4, 2016 at 10:33 PM, Tuncer Ayaz wrote: > > On 4 June 2016 at 08:13, Bj?rn Gustavsson wrote: > > > > > 2.1 Myth: Funs are Slow > > > > What exactly? Calling a fun (object) or creating one? Isn't the > > time to construct fun objects considerable enough that one > > shouldn't do it in, say, a loop? > > That was not the point with the myth at all, and I thinks that > proves that it is time the myth is retired. > > Funs were really slow before R6B, at least an order of magnitude > slower than today. The virtual machines (neither JAM nor BEAM) had > any native support for funs. The function modules_lambdas() in each > module had a clause for each fun defined in the module. The fun > terms themselves were represented as 5-tuple. When a fun was called, > the tuple had to be matched to extract the module that defined the > fun, the arguments had to be packed into a tuple, and > modules_lambdas() had to be applied. (Tony might remember more > details about the implementation.) Sorry, I should have pointed out that I was referring to the thread from April 2015, not the myths.html calling-fun-is-slow myth. http://erlang.org/pipermail/erlang-questions/2015-April/084363.html So, what I should have asked: Should we add that, if one has to optimize everything, avoiding construction of fun objects should be considered as well? Like object instantiation in Java, there can be times you want to optimize at that level, but in most code there's no point. From roberto@REDACTED Wed Jun 8 17:15:05 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 8 Jun 2016 17:15:05 +0200 Subject: [erlang-questions] FIFO queue Message-ID: Dear list, I need to use a FIFO queue in Erlang, disk-backed. Additions to the queue can be quite heavy (3000 / sec), so it's a write intensive operation. Any ideas on best practices? Thank you ^^_ r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jun 8 17:19:03 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 8 Jun 2016 17:19:03 +0200 Subject: [erlang-questions] FIFO queue In-Reply-To: References: Message-ID: To be a little more extensive: I'm familiar with the queue module, but it'd need an additional layer to write stuff to disk. Therefore, I'm currently considering using an ordered_set mnesia table, with os:timestamp() values as the primary key. Just wanted to avoid reinventing the wheel, so if something is out there... please let me know. Best, r. On Wed, Jun 8, 2016 at 5:15 PM, Roberto Ostinelli wrote: > Dear list, > I need to use a FIFO queue in Erlang, disk-backed. > > Additions to the queue can be quite heavy (3000 / sec), so it's a write > intensive operation. > > Any ideas on best practices? > > Thank you ^^_ > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.hague@REDACTED Wed Jun 8 17:50:24 2016 From: james.hague@REDACTED (James Hague) Date: Wed, 8 Jun 2016 10:50:24 -0500 Subject: [erlang-questions] Updating Myths of Erlang Performance In-Reply-To: References: Message-ID: I'd call these dead: 2.7 Myth: BEAM is a Stack-Based Byte-Code Virtual Machine (and Therefore Slow) 2.8 Myth: Use "_" to Speed Up Your Program When a Variable is Not Used I agree with Joe that the one about string processing should be expanded. "No string type" is the issue that comes up more frequently. James -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed Jun 8 17:59:22 2016 From: dmkolesnikov@REDACTED (dmkolesnikov@REDACTED) Date: Wed, 8 Jun 2016 18:59:22 +0300 Subject: [erlang-questions] FIFO queue In-Reply-To: References: Message-ID: Hello, You can look on my project, see branch on-disk-overflow. This is most recent best effort queue I am using in prod (optimized for read/write) https://github.com/fogfish/esq -Dmitry Sent from my iPhone > On 08 Jun 2016, at 18:19, Roberto Ostinelli wrote: > > To be a little more extensive: I'm familiar with the queue module, but it'd need an additional layer to write stuff to disk. > > Therefore, I'm currently considering using an ordered_set mnesia table, with os:timestamp() values as the primary key. > > Just wanted to avoid reinventing the wheel, so if something is out there... please let me know. > > Best, > r. > > > >> On Wed, Jun 8, 2016 at 5:15 PM, Roberto Ostinelli wrote: >> Dear list, >> I need to use a FIFO queue in Erlang, disk-backed. >> >> Additions to the queue can be quite heavy (3000 / sec), so it's a write intensive operation. >> >> Any ideas on best practices? >> >> Thank you ^^_ >> r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay@REDACTED Thu Jun 9 01:25:19 2016 From: jay@REDACTED (Jay Doane) Date: Wed, 8 Jun 2016 16:25:19 -0700 Subject: [erlang-questions] ssl:shutdown(Socket, write) unexpected behavior? Message-ID: I'm working on an erlang network file copier that should be able to use either tcp or ssl, depending on its configuration. The basic idea is that it listens on a specified port, and writes anything it receives to a file, then with the *same* connection, responds to the sender with some statistics, and finally closes the connection. A demonstration version can be found here: https://gist.github.com/jaydoane/65dc6b005788af3c49e2866ea7d03f09 For basic tcp, this can be achieved by the sender doing a gen_tcp:shutdown(Socket, write) after sending the payload, which -- if the receiving socket is configured with {exit_on_close, false} -- closes the connection in the sending direction, but leaves it open to receive the response. For example, this test works: 1> netcopy:test(tcp). Response #{bytecount => 393,checksum => 3827135813,path => "/tmp/netcp"} ok Unfortunately, trying to use ssl:shutdown/2 [1] in the same way seems to close the connection for both writing and reading, and therefore the attempt to use ssl:recv/3 fails: 2> netcopy:test(ssl). Generating a 2048 bit RSA private key ................................................................+++ .................+++ writing new private key to '/tmp/netcopy/key.pem' ----- =ERROR REPORT==== 8-Jun-2016::23:13:26 === Error in process <0.621.0> with exit value: {{badmatch,{error,closed}}, [{netcopy,listen,4,[{file,"netcopy.erl"},{line,38}]}]} ** exception error: no match of right hand side value {error,closed} in function netcopy:sendfile/5 (netcopy.erl, line 88) in call from netcopy:test/1 (netcopy.erl, line 172) After some research [2], I thought the problem might be related to using a Mac, but I've gotten the same results on Ubuntu 14.04. I've also seen this both with otp 18.3 and 19.0-rc2. Is this possible to do using ssl? Thanks, Jay [1] http://erlang.org/doc/man/ssl.html#shutdown-2 [2] https://docs.python.org/2/library/socket.html#socket.socket.shutdown -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Jun 9 02:32:09 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 09 Jun 2016 09:32:09 +0900 Subject: [erlang-questions] ssl:shutdown(Socket, write) unexpected behavior? In-Reply-To: References: Message-ID: <4792146.hxb7RT8U8N@burrito> On Wednesday 08 June 2016 16:25:19 Jay Doane wrote: > I'm working on an erlang network file copier that should be able to > use either tcp or ssl, depending on its configuration. > > The basic idea is that it listens on a specified port, and writes > anything it receives to a file, then with the *same* connection, > responds to the sender with some statistics, and finally closes the > connection. A demonstration version can be found here: > https://gist.github.com/jaydoane/65dc6b005788af3c49e2866ea7d03f09 > > For basic tcp, this can be achieved by the sender doing a > gen_tcp:shutdown(Socket, write) after sending the payload, which -- if > the receiving socket is configured with {exit_on_close, false} -- > closes the connection in the sending direction, but leaves it open to > receive the response. For example, this test works: In general it is not the easiest thing to manage several layers of the communication stack as part of a single protocol. What I mean is, there are several levels here, and the transport layer (TCP/UDP/SCTP -- which is what TLS is supposed to be wrapping) is below the application layer you are trying to build on top. So whether or not you *can* get the effect you want by closing one direction on the connection from one side and detecting that change on the other as an application-level signal isn't really the point -- its probably not a good idea to use this as a hidden protocol "verb". For this kind of thing I usually add a 32-bit header that is the total length to be sent in bytes, and have the receiver add up how much it is receiving, and stop once it receives that much or times out in the middle. At the moment your file transfer system has no indication which condition is occurring: * Did the other end finish sending, but improperly close its 'write' direction of the transport layer. * Did your side improperly interpret the other side closing its 'write' direction. If so is this a driver issue over which you have no control? * Did the other side drop off the network? * Did the other side stall? * Is that all there is to receive? (A few unknowns rolled into one...) Having a known size is much more reliable and also allows you to be more flexible with how you buffer the received data. Now how to react? Once Size == Received, then you initiate the stats response, and have the receiver close the connection. Or you respond with the stats, receive an ack from the sender, and then close the connection. Or whatever. My point being that the application-level signal should not be the transport-layer event. You're getting close to a nice little utility there. :-) Keep at it! Socket programming in Erlang can be a wonderful thing. -Craig From roger@REDACTED Thu Jun 9 09:25:02 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 9 Jun 2016 08:25:02 +0100 Subject: [erlang-questions] How does code:purge/1 know to kill processes? Message-ID: The documentation for code:purge/1 (http://erlang.org/doc/man/code.html#purge-1) says "If some processes still linger in the old code, these processes are killed before the code is removed." How is "lingering" defined? Does the VM kill the process when it attempts to invoke the old module? Does the VM walk the stack for all processes and kill those that have the old code somewhere in their call stack? Something else? From mikpelinux@REDACTED Thu Jun 9 09:44:05 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Thu, 9 Jun 2016 09:44:05 +0200 Subject: [erlang-questions] How does code:purge/1 know to kill processes? In-Reply-To: References: Message-ID: <22361.7749.917004.378754@gargle.gargle.HOWL> Roger Lipscombe writes: > The documentation for code:purge/1 > (http://erlang.org/doc/man/code.html#purge-1) says "If some processes > still linger in the old code, these processes are killed before the > code is removed." > > How is "lingering" defined? > > Does the VM kill the process when it attempts to invoke the old module? > Does the VM walk the stack for all processes and kill those that have > the old code somewhere in their call stack? My understanding is that it's the latter, i.e., if you're currently in a to-be-deleted module, or you have a return address referring to one, then you get killed. I'm not sure how FUNs are handled. > Something else? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- From zxq9@REDACTED Thu Jun 9 09:45:25 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 09 Jun 2016 16:45:25 +0900 Subject: [erlang-questions] How does code:purge/1 know to kill processes? In-Reply-To: References: Message-ID: <4261913.Uob4FFhZ2z@changa> On 2016?6?9? ??? 08:25:02 Roger Lipscombe wrote: > The documentation for code:purge/1 > (http://erlang.org/doc/man/code.html#purge-1) says "If some processes > still linger in the old code, these processes are killed before the > code is removed." > > How is "lingering" defined? > > Does the VM kill the process when it attempts to invoke the old module? > Does the VM walk the stack for all processes and kill those that have > the old code somewhere in their call stack? > Something else? The second one: ceverett@REDACTED:~/Code/erlang$ cat greeter.erl -module(greeter). -export([start_monitor/0, hello/1, loop/0]). start_monitor() -> spawn_monitor(?MODULE, loop, []). hello(Pid) -> Pid ! greeting. loop() -> receive greeting -> io:format("Hello!~n"), loop() end. ceverett@REDACTED:~/Code/erlang$ erlc greeter.erl ceverett@REDACTED:~/Code/erlang$ erl Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Eshell V7.2 (abort with ^G) 1> {Pid1, Ref1} = greeter:start_monitor(). {<0.35.0>,#Ref<0.0.1.31>} 2> [1]+ ?? erl ceverett@REDACTED:~/Code/erlang$ sed -i 's/Hello/Hey/' greeter.erl ceverett@REDACTED:~/Code/erlang$ erlc greeter.erl ceverett@REDACTED:~/Code/erlang$ fg erl 2> code:load_file(greeter). {module,greeter} 3> {Pid2, Ref2} = greeter:start_monitor(). {<0.38.0>,#Ref<0.0.1.40>} 4> greeter:hello(Pid1). Hello! greeting 5> greeter:hello(Pid2). Hey! greeting 6> code:purge(greeter). true 7> flush(). Shell got {'DOWN',#Ref<0.0.1.31>,process,<0.35.0>,killed} ok From samuelrivas@REDACTED Thu Jun 9 09:49:19 2016 From: samuelrivas@REDACTED (Samuel) Date: Thu, 9 Jun 2016 09:49:19 +0200 Subject: [erlang-questions] How does code:purge/1 know to kill processes? In-Reply-To: References: Message-ID: The processes are killed immediately. That happens, for example, if a process is in a loop that doesn't involve any external call (since external calls would load the latest version of the code) -module(test). -compile(export_all). start_process() -> spawn_link(fun foo/0). foo() -> timer:sleep(100000), foo(). 14> c(test). {ok,test} 15> test:start_process(). <0.81.0> 16> l(test). {ok,test} 17> l(test). ** exception exit: killed This doesn't happen if the code has time to reload code by doing an external call. So if we instead do -module(test). -compile(export_all). start_process() -> spawn_link(fun foo/0). foo() -> timer:sleep(100), ?MODULE:foo(). 26> c(test). {ok,test} 27> test:start_process(). <0.126.0> 28> l(test). {module,test} 29> l(test). {module,test} 30> l(test). {module,test} 31> l(test). {module,test} Unless we are fast enough to load code twice while the process is lingering in the sleep. On 9 June 2016 at 09:25, Roger Lipscombe wrote: > The documentation for code:purge/1 > (http://erlang.org/doc/man/code.html#purge-1) says "If some processes > still linger in the old code, these processes are killed before the > code is removed." > > How is "lingering" defined? > > Does the VM kill the process when it attempts to invoke the old module? > Does the VM walk the stack for all processes and kill those that have > the old code somewhere in their call stack? > Something else? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Samuel From richard.jonas@REDACTED Thu Jun 9 10:10:31 2016 From: richard.jonas@REDACTED (Richard Jonas) Date: Thu, 9 Jun 2016 10:10:31 +0200 Subject: [erlang-questions] FIFO queue In-Reply-To: References: Message-ID: Hi Roberto, when we needed to implement something like that I stole the idea from Joe's repo (fast term storage), which is simply a raw (processless) binary file with buffer. https://github.com/joearms/elib1/blob/master/src/elib1_fast_write.erl So you can implement something custom if you like. Also there is a good opportunity for using quickcheck. :) For enterprise ready solutions consider integrate bitcask: https://github.com/basho/bitcask I have never done that, but it is proven solution. Richard On Wed, Jun 8, 2016 at 5:59 PM, wrote: > Hello, > > You can look on my project, see branch on-disk-overflow. This is most > recent best effort queue I am using in prod (optimized for read/write) > > https://github.com/fogfish/esq > > -Dmitry > Sent from my iPhone > > On 08 Jun 2016, at 18:19, Roberto Ostinelli wrote: > > To be a little more extensive: I'm familiar with the queue module, but > it'd need an additional layer to write stuff to disk. > > Therefore, I'm currently considering using an ordered_set mnesia table, > with os:timestamp() values as the primary key. > > Just wanted to avoid reinventing the wheel, so if something is out > there... please let me know. > > Best, > r. > > > > On Wed, Jun 8, 2016 at 5:15 PM, Roberto Ostinelli > wrote: > >> Dear list, >> I need to use a FIFO queue in Erlang, disk-backed. >> >> Additions to the queue can be quite heavy (3000 / sec), so it's a write >> intensive operation. >> >> Any ideas on best practices? >> >> Thank you ^^_ >> r. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Richard Jonas Erlang Solutions Hungary Kft Address: Riverpark Office K.32 K?zrakt?r street 32. 3/1. 1093 Budapest Hungary Phone/fax: +36-1-7000-654 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Thu Jun 9 12:38:35 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 9 Jun 2016 12:38:35 +0200 Subject: [erlang-questions] How does code:purge/1 know to kill processes? In-Reply-To: References: Message-ID: <5759472B.605@ericsson.com> And make sure that external call is a tail call to avoid old return addresses on the stack. All not yet returned recursive calls from the old module will get your process killed by purge. Old live funs (that survived a GC) will also get you killed. This check is however not totally safe and can be fooled by sending funs between processes during the purge or storing them outside a process heap, like in ETS for example. Such orphan funs will lead to an 'undef' exception if called. /Sverker, Erlang/OTP On 06/09/2016 09:49 AM, Samuel wrote: > The processes are killed immediately. That happens, for example, if a > process is in a loop that doesn't involve any external call (since > external calls would load the latest version of the code) > > -module(test). > -compile(export_all). > > start_process() -> > spawn_link(fun foo/0). > > foo() -> > timer:sleep(100000), > foo(). > > > 14> c(test). > {ok,test} > 15> test:start_process(). > <0.81.0> > 16> l(test). > {ok,test} > 17> l(test). > ** exception exit: killed > > This doesn't happen if the code has time to reload code by doing an > external call. So if we instead do > > -module(test). > -compile(export_all). > > start_process() -> > spawn_link(fun foo/0). > > foo() -> > timer:sleep(100), > ?MODULE:foo(). > > 26> c(test). > {ok,test} > 27> test:start_process(). > <0.126.0> > 28> l(test). > {module,test} > 29> l(test). > {module,test} > 30> l(test). > {module,test} > 31> l(test). > {module,test} > > Unless we are fast enough to load code twice while the process is > lingering in the sleep. > > > On 9 June 2016 at 09:25, Roger Lipscombe wrote: >> The documentation for code:purge/1 >> (http://erlang.org/doc/man/code.html#purge-1) says "If some processes >> still linger in the old code, these processes are killed before the >> code is removed." >> >> How is "lingering" defined? >> >> Does the VM kill the process when it attempts to invoke the old module? >> Does the VM walk the stack for all processes and kill those that have >> the old code somewhere in their call stack? >> Something else? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From max.lapshin@REDACTED Thu Jun 9 15:33:37 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 Jun 2016 16:33:37 +0300 Subject: [erlang-questions] Measure all reductions in system Message-ID: I want to try to understand where does erlang spend time and CPU running my program. What do I try to do for it: 1) run erlang:statistics(reductions) before loop 2) run erlang:trace(all, true, [running]) 3) start collecting messages For each {trace,Pid,in,_} and {trace,Pid,out,_} I take process_info(Pid, reductions) and remember first seen reduction count from process and update last seen reduction count from process 4) after some time passes I stop this collection and sum all seen deltas between first seen reduction count and last seen reduction count for all processes 5) when I compare sum of reduction delta and numbers from statistics(reduction) I see that during 5 seconds sum is 152 242 208 and statistics is 802 940 107 Here is my code: https://gist.github.com/maxlapshin/bab46e137d81460672d133bf0cb46a16 Why do I see such difference? Where do I loose reduction count in processes? -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Thu Jun 9 15:49:13 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 9 Jun 2016 15:49:13 +0200 Subject: [erlang-questions] Measure all reductions in system In-Reply-To: References: Message-ID: On Thu, Jun 9, 2016 at 3:33 PM, Max Lapshin wrote: > > Why do I see such difference? > Do you take into account the tracing process? You will not be getting in/out events for it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jun 9 15:59:26 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 Jun 2016 16:59:26 +0300 Subject: [erlang-questions] Measure all reductions in system In-Reply-To: References: Message-ID: Fixed. Still 4 times difference: 10663507 in collected sum and 36976719 in erlang statistics =( Maybe I'm doing something wrong (processes are running outside of in/out) or reductions are calculated somehow else, for example some bifs or io are adding more reductions to global statistics? -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Thu Jun 9 16:06:34 2016 From: duncan@REDACTED (duncan@REDACTED) Date: Thu, 09 Jun 2016 07:06:34 -0700 Subject: [erlang-questions] =?utf-8?q?Random/Crypto_Issue_with_Erlang=3F?= Message-ID: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com> An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Thu Jun 9 16:11:52 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 9 Jun 2016 16:11:52 +0200 Subject: [erlang-questions] Measure all reductions in system In-Reply-To: References: Message-ID: On Thu, Jun 9, 2016 at 3:59 PM, Max Lapshin wrote: > Maybe I'm doing something wrong (processes are running outside of in/out) > or reductions are calculated somehow else, for example some bifs or io are > adding more reductions to global statistics? > There shouldn't be any extra reductions from processes running in secret, or bifs etc. There is however an inherent gap from where you will be miss reductions from when a process starts until you read the first reductions. Since process_info(Pid, reductions) needs the process' main lock, you have to wait until the target process finished executing until you get the reduction info, so you will be about 2000 reductions late. During this time the tracing process will be suspended, so it will be even longer until you are able to get the initial reduction count for the remaining processes. To get around this, maybe you just want to do a [{Pid, process_info(Pid, reductions)} || Pid <- erlang:processes()] to get the initial reduction count instead of reacting on trace messages. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jun 9 16:33:18 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 9 Jun 2016 17:33:18 +0300 Subject: [erlang-questions] Measure all reductions in system In-Reply-To: References: Message-ID: Thanks, fixed, but still the same. On 16-core server there is a stable difference in 4 times between my measurements and erlang statistics. However I think that I still can trust this statistics because I get some reductions info and can work with it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jun 9 18:46:45 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 9 Jun 2016 18:46:45 +0200 Subject: [erlang-questions] Random/Crypto Issue with Erlang? In-Reply-To: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com> References: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com> Message-ID: You probably want Kenji Rikitake. The thing is about CSPRNGs and using strong random bytes from crypto. On Jun 9, 2016 4:06 PM, wrote: > I believe one of the lightning talks at Erlang Factory SF 2016 was about > an issue with randomness and crypto and something we shouldn't use and what > we should do instead. Does this ring a bell? Can anyone point me to the > do's and don't's that were mentioned? Or who gave that talk so I can > contact him? > > Duncan Sparrell > s-Fractal Consulting LLC > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjotr.public12@REDACTED Thu Jun 9 19:31:27 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Thu, 9 Jun 2016 19:31:27 +0200 Subject: [erlang-questions] Reproducible Erlang builds - still some time stamps Message-ID: <20160609173127.GA30990@thebird.nl> Hi all, We noticed the great improvement in the recent release of Erlang, but there are still some problems. Leo found the following: ----- Forwarded message from Leo Famulari ----- Date: Tue, 24 May 2016 12:07:10 -0400 From: Leo Famulari To: Pjotr Prins Subject: Re: [v3] Erlang User-Agent: Mutt/1.6.0 (2016-04-01) Sort of! The compiler does reproducibly build the attached helloworld.erl. But, the Erlang package itself does not build repeatedly. I've attached the diffoscope report (in HTML format) of two build iterations from my machine. Basically, there are still timestamps in 'lib/erlang/erts-8.0/bin/beam' and it's neighbour, 'beam.smp'. There are also a variety of files that include lines like: "%%%??This??file??was??automatically??generated??by??snmpc_mib_to_hrl??version??5.?2.?3 %%%??Date:???24-?May-?2016:?:?15:?33:?12" I remember finding the code that generates these lines and thinking it should not be too hard for us to patch. I could crudely replace the timestamp generator with "1", or an Erlang programmer could make it respect SOURCE_DATE_EPOCH. Or, should we contact Erlang to see if they can fix these issues before the final release? On Tue, May 24, 2016 at 03:44:08PM +0200, Pjotr Prins wrote: > Does it work? > > On Mon, May 23, 2016 at 02:59:14PM -0400, Leo Famulari wrote: > > On Mon, May 23, 2016 at 01:48:11PM +0200, Pjotr Prins wrote: > > > Good news. The changelog for > > > > > > http://erlang.org/download/OTP-19.0-rc1.README > > > > > > says that > > > > > > OTP-13504 Application(s): compiler > > > > > > The compiler will no longer put the compilation date > > > and time into BEAM files. That means that two BEAM > > > files compiled on the same computer from the same > > > source code and compilation options will be identical. > > > > > > Note: If you want to find out whether a BEAM file on > > > disk is different from the loaded code, compared the > > > MD5 value obtained from Mod:module_info(md5) with the > > > MD5 value obtained from beam_lib:md5(BeamFileForMod)` > > > > > > This means that Erlang and Elixir may go into GNU Guix :) > > > > Indeed, great news :) I will try building it with Guix today. > > > > -- From wallentin.dahlberg@REDACTED Thu Jun 9 21:04:57 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Thu, 9 Jun 2016 21:04:57 +0200 Subject: [erlang-questions] Reproducible Erlang builds - still some time stamps In-Reply-To: <20160609173127.GA30990@thebird.nl> References: <20160609173127.GA30990@thebird.nl> Message-ID: For beam and beam.smp there is a define, ERLANG_COMPILE_DATE, set that holds the compile date for debugging purposes. It it used when generating an erl_crash.dump or for etp when viewing core (it reads a global symbol) files. You can turn off ERLANG_COMPILE_DATE usage with configure option: --disable-saved-compile-time // Bj?rn-Egil 2016-06-09 19:31 GMT+02:00 Pjotr Prins : > Hi all, > > We noticed the great improvement in the recent release of Erlang, but > there are still some problems. Leo found the following: > > > ----- Forwarded message from Leo Famulari ----- > > Date: Tue, 24 May 2016 12:07:10 -0400 > From: Leo Famulari > To: Pjotr Prins > Subject: Re: [v3] Erlang > User-Agent: Mutt/1.6.0 (2016-04-01) > > Sort of! The compiler does reproducibly build the attached > helloworld.erl. > > But, the Erlang package itself does not build repeatedly. I've attached > the diffoscope report (in HTML format) of two build iterations from my > machine. > > Basically, there are still timestamps in 'lib/erlang/erts-8.0/bin/beam' > and it's neighbour, 'beam.smp'. > > There are also a variety of files that include lines like: > > "%%%??This??file??was??automatically??generated??by??snmpc_mib_to_hrl??version??5.?2.?3 > %%%??Date:???24-?May-?2016:?:?15:?33:?12" > > I remember finding the code that generates these lines and thinking it > should not be too hard for us to patch. I could crudely replace the > timestamp generator with "1", or an Erlang programmer could make it > respect SOURCE_DATE_EPOCH. > > Or, should we contact Erlang to see if they can fix these issues before > the final release? > > On Tue, May 24, 2016 at 03:44:08PM +0200, Pjotr Prins wrote: > > Does it work? > > > > On Mon, May 23, 2016 at 02:59:14PM -0400, Leo Famulari wrote: > > > On Mon, May 23, 2016 at 01:48:11PM +0200, Pjotr Prins wrote: > > > > Good news. The changelog for > > > > > > > > http://erlang.org/download/OTP-19.0-rc1.README > > > > > > > > says that > > > > > > > > OTP-13504 Application(s): compiler > > > > > > > > The compiler will no longer put the compilation date > > > > and time into BEAM files. That means that two BEAM > > > > files compiled on the same computer from the same > > > > source code and compilation options will be identical. > > > > > > > > Note: If you want to find out whether a BEAM file on > > > > disk is different from the loaded code, compared the > > > > MD5 value obtained from Mod:module_info(md5) with the > > > > MD5 value obtained from beam_lib:md5(BeamFileForMod)` > > > > > > > > This means that Erlang and Elixir may go into GNU Guix :) > > > > > > Indeed, great news :) I will try building it with Guix today. > > > > > > > -- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Thu Jun 9 22:20:31 2016 From: lheadley@REDACTED (Lyn Headley) Date: Thu, 9 Jun 2016 13:20:31 -0700 Subject: [erlang-questions] Erlang cookies are secure Message-ID: I don't understand why so many people are so dismissive of cookie-based node security. Here's what Fred Hebert wrote, for example: "While documents like the official Erlang documentation put cookies under the topic of security, they're really not security at all. If it is, it has to be seen as a joke, because there's no way anybody serious considers the cookie a safe thing. Why? Simply because the cookie is a little unique value that must be shared between nodes to allow them to connect together. They're closer to the idea of user names than passwords and I'm pretty sure nobody would consider having a username (and nothing else) as a security feature. Cookies make way more sense as a mechanism used to divide clusters of nodes than as an authentication mechanism." In opposition to this extremely widespread sentiment, I believe that: 1) It is feasible to create an unguessable cookie. 2) It is feasible to prevent outsiders from seeing the cookie's value. Therefore attackers cannot take over my node by compromising my cookie. Where is the flaw in my reasoning? Lyn Headley From jay@REDACTED Fri Jun 10 05:03:11 2016 From: jay@REDACTED (Jay Doane) Date: Thu, 9 Jun 2016 20:03:11 -0700 Subject: [erlang-questions] ssl:shutdown(Socket, write) unexpected behavior? In-Reply-To: <4792146.hxb7RT8U8N@burrito> References: <4792146.hxb7RT8U8N@burrito> Message-ID: Craig, Thank you for the detailed reply! I appreciate the advantages of using some kind of header to send metadata (destination path immediately comes to mind as also possibly useful). The main use case I had considered for using ssl:shutdown(S, write) was to enable interaction from simple command line tools like netcat. But putting aside for the moment whether or not that's a good idea, I'm still curious about OTP's ssl documentation, which seems to indicate that it should be possible to do: http://erlang.org/doc/man/ssl.html#shutdown-2 Am I misinterpreting the documentation, or is this a bug? Thanks, Jay On Wed, Jun 8, 2016 at 5:32 PM, zxq9 wrote: > On Wednesday 08 June 2016 16:25:19 Jay Doane wrote: > > I'm working on an erlang network file copier that should be able to > > use either tcp or ssl, depending on its configuration. > > > > The basic idea is that it listens on a specified port, and writes > > anything it receives to a file, then with the *same* connection, > > responds to the sender with some statistics, and finally closes the > > connection. A demonstration version can be found here: > > https://gist.github.com/jaydoane/65dc6b005788af3c49e2866ea7d03f09 > > > > For basic tcp, this can be achieved by the sender doing a > > gen_tcp:shutdown(Socket, write) after sending the payload, which -- if > > the receiving socket is configured with {exit_on_close, false} -- > > closes the connection in the sending direction, but leaves it open to > > receive the response. For example, this test works: > > In general it is not the easiest thing to manage several layers of the > communication stack as part of a single protocol. What I mean is, there > are several levels here, and the transport layer (TCP/UDP/SCTP -- which > is what TLS is supposed to be wrapping) is below the application layer > you are trying to build on top. So whether or not you *can* get the > effect you want by closing one direction on the connection from one side > and detecting that change on the other as an application-level signal > isn't really the point -- its probably not a good idea to use this as a > hidden protocol "verb". > > For this kind of thing I usually add a 32-bit header that is the total > length to be sent in bytes, and have the receiver add up how much it is > receiving, and stop once it receives that much or times out in the middle. > At the moment your file transfer system has no indication which condition > is occurring: > > * Did the other end finish sending, but improperly close its 'write' > direction of the transport layer. > * Did your side improperly interpret the other side closing its 'write' > direction. If so is this a driver issue over which you have no control? > * Did the other side drop off the network? > * Did the other side stall? > * Is that all there is to receive? (A few unknowns rolled into one...) > > Having a known size is much more reliable and also allows you to be more > flexible with how you buffer the received data. > > Now how to react? > > Once Size == Received, then you initiate the stats response, and have the > receiver close the connection. Or you respond with the stats, receive an > ack from the sender, and then close the connection. Or whatever. My point > being that the application-level signal should not be the transport-layer > event. > > You're getting close to a nice little utility there. :-) Keep at it! > Socket programming in Erlang can be a wonderful thing. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From technion@REDACTED Fri Jun 10 05:06:01 2016 From: technion@REDACTED (Technion) Date: Fri, 10 Jun 2016 03:06:01 +0000 Subject: [erlang-questions] Random/Crypto Issue with Erlang? In-Reply-To: References: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com>, Message-ID: Hi, I note the manual for rand_bytes() describes it as note cryptographically strong: http://erlang.org/doc/man/crypto.html#rand_bytes-1 This function is not recommended for cryptographic purposes Reviewing the manual for "strong rand bytes" however: http://erlang.org/doc/man/crypto.html#strong_rand_bytes-1 By default this is the RAND_bytes method from OpenSSL. This isn't necessarily a great solution either, with both Node and Ruby being advised against this: https://bugs.ruby-lang.org/issues/9569 https://github.com/nodejs/node/issues/5798 ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Jesper Louis Andersen Sent: Friday, 10 June 2016 2:46:45 AM To: duncan@REDACTED Cc: Erlang (E-mail) Subject: Re: [erlang-questions] Random/Crypto Issue with Erlang? You probably want Kenji Rikitake. The thing is about CSPRNGs and using strong random bytes from crypto. On Jun 9, 2016 4:06 PM, > wrote: I believe one of the lightning talks at Erlang Factory SF 2016 was about an issue with randomness and crypto and something we shouldn't use and what we should do instead. Does this ring a bell? Can anyone point me to the do's and don't's that were mentioned? Or who gave that talk so I can contact him? Duncan Sparrell s-Fractal Consulting LLC _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From louis@REDACTED Thu Jun 9 23:44:57 2016 From: louis@REDACTED (Louis Pilfold) Date: Thu, 9 Jun 2016 22:44:57 +0100 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: References: Message-ID: Hi! In the event that the cookie is your only security, what do you do when your cookie gets out? Event if you cookie is not guessable, there is still a chance that through malicious act or human error a trusted person within your organisation shares your cookie with others. I've not got the evidence to hand, but while preparing for security audits at a previous workplace our trainer told us that most security breaches are due to the actions of people within the organisation rather than outside of it. This seems very plausible to me. Cheers, Louis On 9 June 2016 at 21:20, Lyn Headley wrote: > I don't understand why so many people are so dismissive of > cookie-based node security. Here's what Fred Hebert wrote, for > example: > > "While documents like the official Erlang documentation put cookies > under the topic of security, they're really not security at all. If it > is, it has to be seen as a joke, because there's no way anybody > serious considers the cookie a safe thing. Why? Simply because the > cookie is a little unique value that must be shared between nodes to > allow them to connect together. They're closer to the idea of user > names than passwords and I'm pretty sure nobody would consider having > a username (and nothing else) as a security feature. Cookies make way > more sense as a mechanism used to divide clusters of nodes than as an > authentication mechanism." > > In opposition to this extremely widespread sentiment, I believe that: > > 1) It is feasible to create an unguessable cookie. > 2) It is feasible to prevent outsiders from seeing the cookie's value. > > Therefore attackers cannot take over my node by compromising my cookie. > > Where is the flaw in my reasoning? > > Lyn Headley > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Fri Jun 10 06:33:25 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 10 Jun 2016 13:33:25 +0900 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: References: Message-ID: <4324236.69caBLZbcx@changa> On 2016?6?9? ??? 22:44:57 Louis Pilfold wrote: > Hi! > > In the event that the cookie is your only security, what do you do > when your cookie gets out? > > Event if you cookie is not guessable, there is still a chance that > through malicious act or human error a trusted person within your > organisation shares your cookie with others. I've not got the evidence > to hand, but while preparing for security audits at a previous > workplace our trainer told us that most security breaches are due to > the actions of people within the organisation rather than outside of > it. This seems very plausible to me. People are almost always easier to manipulate or catch in error than systems are to crack through exploitation of technical flaws. How is this not exactly the same as a password? Or AWS credentials? Or a secret key? Or any other of a host of similar schemes? -Craig From technion@REDACTED Fri Jun 10 08:40:00 2016 From: technion@REDACTED (Technion) Date: Fri, 10 Jun 2016 06:40:00 +0000 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <4324236.69caBLZbcx@changa> References: , <4324236.69caBLZbcx@changa> Message-ID: One thing here is that a cookie has to be constant across an environment. It's not easy to rotate it by tackling a few nodes at a time, and you can't define a new username, roll it out, and disable the old one later. As far as I know, it's always transferred in cleartext, and doesn't authenticate who it is being given to. By modern security standards, it's very poor. But I also agree, the mere use cookies, from an outside attacker, is still a mile ahead of authenticated access, unless I'm missing something. ________________________________ From: erlang-questions-bounces@REDACTED on behalf of zxq9 Sent: Friday, 10 June 2016 2:33:25 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Erlang cookies are secure On 2016?6?9? ??? 22:44:57 Louis Pilfold wrote: > Hi! > > In the event that the cookie is your only security, what do you do > when your cookie gets out? > > Event if you cookie is not guessable, there is still a chance that > through malicious act or human error a trusted person within your > organisation shares your cookie with others. I've not got the evidence > to hand, but while preparing for security audits at a previous > workplace our trainer told us that most security breaches are due to > the actions of people within the organisation rather than outside of > it. This seems very plausible to me. People are almost always easier to manipulate or catch in error than systems are to crack through exploitation of technical flaws. How is this not exactly the same as a password? Or AWS credentials? Or a secret key? Or any other of a host of similar schemes? -Craig _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Jun 10 08:40:20 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 10 Jun 2016 15:40:20 +0900 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: References: <4324236.69caBLZbcx@changa> Message-ID: <16493235.ks34GvqR50@changa> On 2016?6?10? ??? 07:18:51 you wrote: > Heya > > With the given examples each entity had their own password/key/secret, so a > breach means one node is secure, rather than all of them. Additionally each > piece of functionality can require different permissions, and not all nodes > can have permissions to request all tasks, so the scale of the potential > damage done is lower. > > Additionally one can rotate those values easily, this seems like it would > be much harder to do with cookies. Changing cookies is certainly an issue, but the partitioning issue is almost entirely moot these days. With single-sign-on via Kerberos, LDAP and especially let's-just-pretend-its-secure web auth systems that have a tendency to place a person's (and sometimes by extension entire organization's) data, management controls and platforms just one (publicly known) email and password away from complete compromise. How many companies are keeping systems-critical credentials in plain text within config files on s3 or private github repos? How many organizations have come to depend *entirely* on Google or fb credentials? This is the trend today -- to sacrifice security for usability, and it is a pretty steep tradeoff. My point isn't that cookies are secure -- I don't believe that they are. My point is that cookies are not inherently *less* secure than the mass of SSO schemes I have begun to see sprout up everywhere the word "cloud" is used. -Craig From chandrashekhar.mullaparthi@REDACTED Fri Jun 10 10:19:32 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 10 Jun 2016 09:19:32 +0100 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: References: Message-ID: Hi Lyn, There are several problems with Erlang's distribution model from a security standpoint. For e.g. if a host which has access to inter-node comms is compromised, it can sniff the cookie out and then establish connections to any node in the Erlang cluster. Okay, so you change distribution to use TLS so one cannot sniff cookies. But now imagine that someone got access to just one of the boxes in an Erlang cluster. The entire cluster is now open for abuse because there is no access control mechanism within distributed erlang. There is no audit log of which node connected when, what commands were executed, nothing at all. Even os:cmd/1 is available for executing anything as that user on the box. Distributed erlang assumes that the cluster is operating in a secure environment which may not always satisfy the requirements. For e.g. if you are building something in Erlang and you want it to be PCIDSS [1] compliant, you will want to turn off distributed erlang on that node because the combination of the openness of an Erlang cluster combined with its tracing capabilities means that anyone with access to a single node in a cluster can trace and capture data flowing through code handling credit card transactions. regards, Chandru [1] http://www.theukcardsassociation.org.uk/security/what_is_PCI%20DSS.asp On 9 June 2016 at 21:20, Lyn Headley wrote: > I don't understand why so many people are so dismissive of > cookie-based node security. Here's what Fred Hebert wrote, for > example: > > "While documents like the official Erlang documentation put cookies > under the topic of security, they're really not security at all. If it > is, it has to be seen as a joke, because there's no way anybody > serious considers the cookie a safe thing. Why? Simply because the > cookie is a little unique value that must be shared between nodes to > allow them to connect together. They're closer to the idea of user > names than passwords and I'm pretty sure nobody would consider having > a username (and nothing else) as a security feature. Cookies make way > more sense as a mechanism used to divide clusters of nodes than as an > authentication mechanism." > > In opposition to this extremely widespread sentiment, I believe that: > > 1) It is feasible to create an unguessable cookie. > 2) It is feasible to prevent outsiders from seeing the cookie's value. > > Therefore attackers cannot take over my node by compromising my cookie. > > Where is the flaw in my reasoning? > > Lyn Headley > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jun 10 10:28:30 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 10 Jun 2016 10:28:30 +0200 Subject: [erlang-questions] Random/Crypto Issue with Erlang? In-Reply-To: References: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com> Message-ID: On Fri, Jun 10, 2016 at 5:06 AM, Technion wrote: > *By default this is the RAND_bytes method from OpenSSL.* Indeed. Luckily crypto:random_bytes/1 is deprecated as of release 19.0 IIRC. And I do believe crypto:strong_random_bytes/1 still uses the SSL RAND_bytes generator. The `enacl` library I wrote for NaCl/libsodium based crypto uses libsodiums randombytes primitive which is indeed tied to the kernel CSPRNG, which avoids these problems on most modern operating systems. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jun 10 11:08:18 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 10 Jun 2016 11:08:18 +0200 Subject: [erlang-questions] FIFO queue In-Reply-To: References: Message-ID: Thank you Dmitry, Ulf and Richard! These seem all interesting possibilities and I will dive into them. Best, r. On Thu, Jun 9, 2016 at 10:10 AM, Richard Jonas < richard.jonas@REDACTED> wrote: > Hi Roberto, > > when we needed to implement something like that I stole the idea from > Joe's repo (fast term storage), which is simply a raw (processless) binary > file with buffer. > https://github.com/joearms/elib1/blob/master/src/elib1_fast_write.erl > So you can implement something custom if you like. Also there is a good > opportunity for using quickcheck. :) > > For enterprise ready solutions consider integrate bitcask: > https://github.com/basho/bitcask > I have never done that, but it is proven solution. > > Richard > > > On Wed, Jun 8, 2016 at 5:59 PM, wrote: > >> Hello, >> >> You can look on my project, see branch on-disk-overflow. This is most >> recent best effort queue I am using in prod (optimized for read/write) >> >> https://github.com/fogfish/esq >> >> -Dmitry >> Sent from my iPhone >> >> On 08 Jun 2016, at 18:19, Roberto Ostinelli wrote: >> >> To be a little more extensive: I'm familiar with the queue module, but >> it'd need an additional layer to write stuff to disk. >> >> Therefore, I'm currently considering using an ordered_set mnesia table, >> with os:timestamp() values as the primary key. >> >> Just wanted to avoid reinventing the wheel, so if something is out >> there... please let me know. >> >> Best, >> r. >> >> >> >> On Wed, Jun 8, 2016 at 5:15 PM, Roberto Ostinelli >> wrote: >> >>> Dear list, >>> I need to use a FIFO queue in Erlang, disk-backed. >>> >>> Additions to the queue can be quite heavy (3000 / sec), so it's a write >>> intensive operation. >>> >>> Any ideas on best practices? >>> >>> Thank you ^^_ >>> r. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Richard Jonas > Erlang Solutions Hungary Kft > > Address: > Riverpark Office K.32 > K?zrakt?r street 32. 3/1. > 1093 Budapest > Hungary > Phone/fax: > +36-1-7000-654 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From louis@REDACTED Fri Jun 10 08:18:51 2016 From: louis@REDACTED (Louis Pilfold) Date: Fri, 10 Jun 2016 07:18:51 +0100 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <4324236.69caBLZbcx@changa> References: <4324236.69caBLZbcx@changa> Message-ID: Heya With the given examples each entity had their own password/key/secret, so a breach means one node is secure, rather than all of them. Additionally each piece of functionality can require different permissions, and not all nodes can have permissions to request all tasks, so the scale of the potential damage done is lower. Additionally one can rotate those values easily, this seems like it would be much harder to do with cookies. Cheers, Louis On 10 Jun 2016 05:33, "zxq9" wrote: > On 2016?6?9? ??? 22:44:57 Louis Pilfold wrote: > > Hi! > > > > In the event that the cookie is your only security, what do you do > > when your cookie gets out? > > > > Event if you cookie is not guessable, there is still a chance that > > through malicious act or human error a trusted person within your > > organisation shares your cookie with others. I've not got the evidence > > to hand, but while preparing for security audits at a previous > > workplace our trainer told us that most security breaches are due to > > the actions of people within the organisation rather than outside of > > it. This seems very plausible to me. > > People are almost always easier to manipulate or catch in error than > systems are to crack through exploitation of technical flaws. > > How is this not exactly the same as a password? Or AWS credentials? > Or a secret key? Or any other of a host of similar schemes? > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Jun 10 12:14:09 2016 From: tony@REDACTED (Tony Rogvall) Date: Fri, 10 Jun 2016 12:14:09 +0200 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: References: Message-ID: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> Hi Chandru. I am not sure what you mean by sniff cookies? The distribution has been sending blank cookies since first open source release. The distribution do not send the cookie in clear text but rely on a MD5 challenge procedure at connection setup. So Erlang is more likely to be vulnerable to connection hijacking since not every message is signed. So keep the nodes safe and away from random users. At least until we get Safe Erlang ( any decade now ) /Tony > On 10 jun 2016, at 10:19, Chandru wrote: > > Hi Lyn, > > There are several problems with Erlang's distribution model from a security standpoint. For e.g. if a host which has access to inter-node comms is compromised, it can sniff the cookie out and then establish connections to any node in the Erlang cluster. > > Okay, so you change distribution to use TLS so one cannot sniff cookies. But now imagine that someone got access to just one of the boxes in an Erlang cluster. The entire cluster is now open for abuse because there is no access control mechanism within distributed erlang. There is no audit log of which node connected when, what commands were executed, nothing at all. Even os:cmd/1 is available for executing anything as that user on the box. > > Distributed erlang assumes that the cluster is operating in a secure environment which may not always satisfy the requirements. For e.g. if you are building something in Erlang and you want it to be PCIDSS [1] compliant, you will want to turn off distributed erlang on that node because the combination of the openness of an Erlang cluster combined with its tracing capabilities means that anyone with access to a single node in a cluster can trace and capture data flowing through code handling credit card transactions. > > regards, > Chandru > > [1] http://www.theukcardsassociation.org.uk/security/what_is_PCI%20DSS.asp > > > On 9 June 2016 at 21:20, Lyn Headley wrote: > I don't understand why so many people are so dismissive of > cookie-based node security. Here's what Fred Hebert wrote, for > example: > > "While documents like the official Erlang documentation put cookies > under the topic of security, they're really not security at all. If it > is, it has to be seen as a joke, because there's no way anybody > serious considers the cookie a safe thing. Why? Simply because the > cookie is a little unique value that must be shared between nodes to > allow them to connect together. They're closer to the idea of user > names than passwords and I'm pretty sure nobody would consider having > a username (and nothing else) as a security feature. Cookies make way > more sense as a mechanism used to divide clusters of nodes than as an > authentication mechanism." > > In opposition to this extremely widespread sentiment, I believe that: > > 1) It is feasible to create an unguessable cookie. > 2) It is feasible to prevent outsiders from seeing the cookie's value. > > Therefore attackers cannot take over my node by compromising my cookie. > > Where is the flaw in my reasoning? > > Lyn Headley > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From zxq9@REDACTED Fri Jun 10 12:45:14 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 10 Jun 2016 19:45:14 +0900 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> References: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> Message-ID: <1496450.yTyoMventr@changa> On 2016?6?10? ??? 12:14:09 Tony Rogvall wrote: > Hi Chandru. > > I am not sure what you mean by sniff cookies? > > The distribution has been sending blank cookies since first open source release. > The distribution do not send the cookie in clear text but rely on a MD5 challenge procedure > at connection setup. > > So Erlang is more likely to be vulnerable to connection hijacking since not every message > is signed. > > So keep the nodes safe and away from random users. At least until we get Safe Erlang ( any decade now ) Indeed! :-) On that note... Even if it were safe to run disterl across a WAN on an open network, does anyone really think operating a chatty mesh across variable latency connections is a good idea? "Variable" meaning: - Totally out of your control - Local nodes and distant nodes will have dramatic network performance differences. That's not at all the case disterl seems to have been intended for. Or at least it certainly doesn't seem to have been the case driving the disterl implementation. I've only ever seen localized disterl clusters connect (sometimes multiply) over TLS to distant other clusters -- treating each cluster, essentially, as an opaque supernode in a larger system. It would be pretty slick if we could link safely across the world with a very flexible distribution system that works over TLS, checks verifies peers with certificates, and can handle very long disruptions. Oh, and and automatically mend network damage. Oh, and while we're at it, automagically solve network partition problems. Oh, and... ...but we're running a bit short on unicorn blood to feed the machine so instead we have disterl, which seems very much to have been designed with a localized cluster of nodes in mind, all of which are under a reasonable level of control of the system operators -- and therefore constitutes an internally safe execution environment. In this context it is hard to view cookies as really being intended as a security mechanism. That said, disterl is a dodgy proposition in environments like hosted mesos, AWS, and similar environments where you can't know anything about internal security and there is a strong economic incentive to keep breaches and insecurities quiet. So now can someone please explain to me why my vision of the world is wrong? *Was* disterl intended to be used to link machines across very wide networks? Were cookies + MD5 intended as a partitioning mechanism, or did someone think that was secure at one time? In *practice* I've only encountered the situation I describe above -- but all this apparent confusion about the issue of what cookies were intended for makes me wonder whether that was the *intention*. -Craig From chandrashekhar.mullaparthi@REDACTED Fri Jun 10 12:54:23 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 10 Jun 2016 11:54:23 +0100 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> References: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> Message-ID: Hi Tony, On 10 June 2016 at 11:14, Tony Rogvall wrote: > Hi Chandru. > > I am not sure what you mean by sniff cookies? > > The distribution has been sending blank cookies since first open source > release. > The distribution do not send the cookie in clear text but rely on a MD5 > challenge procedure > at connection setup. > > So Erlang is more likely to be vulnerable to connection hijacking since > not every message > is signed. > Hmm...that is strange. My memory tells me that I saw the cookie while examining packet captures a long time ago - I could be mistaken. I'll go read the source again - thanks for correcting me. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From semmitmondo@REDACTED Fri Jun 10 13:14:16 2016 From: semmitmondo@REDACTED (semmit mondo) Date: Fri, 10 Jun 2016 13:14:16 +0200 (CEST) Subject: [erlang-questions] soap and utf8 problem Message-ID: Hi, I use this Erlang SOAP library https://github.com/bet365/soap from an Elixir application. Everything works fine up until I try to send strings with accents (or other unusal characters) in it. This is what I get: {:fault, 500, [{'connection', 'close'}, {'date', 'Fri, 10 Jun 2016 10:18:53 GMT'}, {'server', 'Apache/2.4.18 (Unix) OpenSSL/1.0.2g'}, {'content-length', '278'}, {'content-type', 'text/xml; charset=utf-8'}], [], {:soap_fault_1_1, {:faultcode, 'http://schemas.xmlsoap.org/soap/envelope/', 'Client', :undefined}, 'Bad Request', :undefined, []}, [], "nSOAP-ENV:ClientBad Requestn"} ...and the way I convert utf8 binaries to char lists is simply using the '#{some_utf8_binary_string}' Elixir syntax. (I believe this is the same as to_char_list(some_utf8_binary). Can anybody give me at least tips what to check and how to solve this issue? From technion@REDACTED Fri Jun 10 13:31:26 2016 From: technion@REDACTED (Technion) Date: Fri, 10 Jun 2016 11:31:26 +0000 Subject: [erlang-questions] Random/Crypto Issue with Erlang? In-Reply-To: References: <20160609070634.7e43b23f706d1a78218bd3e1c66e57ee.3ac5013266.wbe@email23.godaddy.com> , Message-ID: >uses libsodiums randombytes I would therefore suggest the correct answer to any crypto question should be "use Jesper's library". ________________________________ From: Jesper Louis Andersen Sent: Friday, 10 June 2016 6:28:30 PM To: Technion Cc: duncan@REDACTED; Erlang (E-mail) Subject: Re: [erlang-questions] Random/Crypto Issue with Erlang? On Fri, Jun 10, 2016 at 5:06 AM, Technion > wrote: By default this is the RAND_bytes method from OpenSSL. Indeed. Luckily crypto:random_bytes/1 is deprecated as of release 19.0 IIRC. And I do believe crypto:strong_random_bytes/1 still uses the SSL RAND_bytes generator. The `enacl` library I wrote for NaCl/libsodium based crypto uses libsodiums randombytes primitive which is indeed tied to the kernel CSPRNG, which avoids these problems on most modern operating systems. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From w.a.de.jong@REDACTED Fri Jun 10 13:58:58 2016 From: w.a.de.jong@REDACTED (Willem de Jong) Date: Fri, 10 Jun 2016 13:58:58 +0200 Subject: [erlang-questions] soap and utf8 problem In-Reply-To: References: Message-ID: Hi, If I understand correctly, you are using the library to implement a SOAP client. The response that you are quoting is the translation of a SOAP fault. Apparently the server is not accepting your request. Probably because something went wrong with the encoding, as you are suggesting. Could you give some additional information on how you are calling the service from within your Elixir application? And perhaps show the message that is sent to the server (as it is sent over the wire)? Regards, Willem On Fri, Jun 10, 2016 at 1:14 PM, semmit mondo wrote: > Hi, > > I use this Erlang SOAP library > > https://github.com/bet365/soap > > from an Elixir application. Everything works fine up until I try to > send strings with accents (or other unusal characters) in it. > > This is what I get: > > {:fault, 500, > [{'connection', 'close'}, {'date', 'Fri, 10 Jun 2016 10:18:53 GMT'}, > {'server', 'Apache/2.4.18 (Unix) OpenSSL/1.0.2g'}, {'content-length', > '278'}, > {'content-type', 'text/xml; charset=utf-8'}], [], > {:soap_fault_1_1, > {:faultcode, 'http://schemas.xmlsoap.org/soap/envelope/', 'Client', > :undefined}, 'Bad Request', :undefined, []}, [], > "n xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"> OAP-ENV:Body>SOAP-ENV:ClientBad > Request -ENV:Body>n"} > > > ...and the way I convert utf8 binaries to char lists is simply using > the '#{some_utf8_binary_string}' Elixir syntax. (I believe this is the > same as to_char_list(some_utf8_binary). > > Can anybody give me at least tips what to check and how to solve this > issue? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Fri Jun 10 22:10:47 2016 From: per@REDACTED (Per Hedeland) Date: Fri, 10 Jun 2016 22:10:47 +0200 (CEST) Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> Message-ID: <201606102010.u5AKAlOI024412@pluto.hedeland.org> Tony Rogvall wrote: > >I am not sure what you mean by sniff cookies? > >The distribution has been sending blank cookies since first open source release. >The distribution do not send the cookie in clear text but rely on a MD5 challenge procedure >at connection setup. Hi Tony! Indeed - and I will take credit for pestering you to fix that just before the first open source release:-) (I will not divulge what it did before that...). >So Erlang is more likely to be vulnerable to connection hijacking since not every message >is signed. Yes - the *default* distribution fulfills none of the CIA requirements (no, not that evil US thing, but Confidentiality, Integrity, and Availability). But this has nothing to do with the authentication mechanism as such, and can be fixed by using TLS - which also brings one or more other authentication mechanisms, but they are not in any fundamental sense more "secure" than the cookie authentication. >So keep the nodes safe and away from random users. At least until we get Safe Erlang ( any decade now ) Sure - but this point is actually also confusing in a cookie discussion, as shown by other messages in this thread - it is about the *authorization* you automatically get at the point when you have managed to break the authentication mechanism - i.e. basically you can do "anything". But this is independent of the strength of the authentication mechanism itself. I do find it rather tiresome with this constant ridicule of the cookie authentication from people who haven't even bothered to do a basic investigation of how it works, let alone done any actual security analysis. And just to put another myth to death, no, you are not required to use the same cookie on all your distributed erlang nodes - every node is capable of maintaining a specific cookie for every other node, RTFM erlang:set_cookie/2. It is absolutely true that *maintaining* security in a network with cookie-based authentication can be troublesome, and that e.g. TLS with certificate authentication can do much better in that respect, as long as you have mechanisms for certificate revocation properly set up (which in turn is not entirely trivial to do). But again, as long as you do not throw your cookies around, AFAIK no-one has demonstrated any fundamental weakness with the mechanism as such. --Per From serge@REDACTED Fri Jun 10 22:43:19 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 10 Jun 2016 16:43:19 -0400 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <201606102010.u5AKAlOI024412@pluto.hedeland.org> References: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> <201606102010.u5AKAlOI024412@pluto.hedeland.org> Message-ID: I guess it's worth mentioning in the current thread that the present architecture only supports one distributed protocol in a node at a time. So using TLS for some nodes over Internet would require all nodes in a cluster to use TLS, which is a waste of resources and additional latency for nodes located in the same local network not involving Internet. I'd love to see some progress by the OTP team in flexing this requirement, since the patch I submitted a while back that introduced support for distribution over multiple protocols was not accepted (*). Serge (*) http://erlang.org/pipermail/erlang-patches/2014-January/004522.html On Fri, Jun 10, 2016 at 4:10 PM, Per Hedeland wrote: > Tony Rogvall wrote: > > > >I am not sure what you mean by sniff cookies? > > > >The distribution has been sending blank cookies since first open source > release. > >The distribution do not send the cookie in clear text but rely on a MD5 > challenge procedure > >at connection setup. > > Hi Tony! > > Indeed - and I will take credit for pestering you to fix that just > before the first open source release:-) (I will not divulge what it did > before that...). > > >So Erlang is more likely to be vulnerable to connection hijacking since > not every message > >is signed. > > Yes - the *default* distribution fulfills none of the CIA requirements > (no, not that evil US thing, but Confidentiality, Integrity, and > Availability). But this has nothing to do with the authentication > mechanism as such, and can be fixed by using TLS - which also brings one > or more other authentication mechanisms, but they are not in any > fundamental sense more "secure" than the cookie authentication. > > >So keep the nodes safe and away from random users. At least until we get > Safe Erlang ( any decade now ) > > Sure - but this point is actually also confusing in a cookie discussion, > as shown by other messages in this thread - it is about the > *authorization* you automatically get at the point when you have managed > to break the authentication mechanism - i.e. basically you can do > "anything". But this is independent of the strength of the > authentication mechanism itself. > > I do find it rather tiresome with this constant ridicule of the cookie > authentication from people who haven't even bothered to do a basic > investigation of how it works, let alone done any actual security > analysis. > > And just to put another myth to death, no, you are not required to use > the same cookie on all your distributed erlang nodes - every node is > capable of maintaining a specific cookie for every other node, RTFM > erlang:set_cookie/2. > > It is absolutely true that *maintaining* security in a network with > cookie-based authentication can be troublesome, and that e.g. TLS with > certificate authentication can do much better in that respect, as long > as you have mechanisms for certificate revocation properly set up (which > in turn is not entirely trivial to do). > > But again, as long as you do not throw your cookies around, AFAIK no-one > has demonstrated any fundamental weakness with the mechanism as such. > > --Per > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Fri Jun 10 22:47:19 2016 From: per@REDACTED (Per Hedeland) Date: Fri, 10 Jun 2016 22:47:19 +0200 (CEST) Subject: [erlang-questions] Support for IPv6 link-local addresses Message-ID: <201606102047.u5AKlJ2B024548@pluto.hedeland.org> Hi, Has anyone implemented support for IPv6 link-local addresses in gen_tcp/gen_udp and friends, or attempted to, or even thought about the matter? The problem is that using such addresses require that you provide a "scope-id" - normally (always?) in the form of "%" appended to the address in the textual representation, the "interface index" (which can be derived from the interface name) in the actual OS API functions. And while making inet_drv handle it should be straightforward, the pervasive 8-tuple representation of an IPv6 address has no place for it. --Per From tony@REDACTED Fri Jun 10 23:52:52 2016 From: tony@REDACTED (Tony Rogvall) Date: Fri, 10 Jun 2016 23:52:52 +0200 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <201606102010.u5AKAlOI024412@pluto.hedeland.org> References: <201606102010.u5AKAlOI024412@pluto.hedeland.org> Message-ID: "typed while walking!" > On 10 juni 2016, at 22:10, Per Hedeland wrote: > > Tony Rogvall wrote: >> >> I am not sure what you mean by sniff cookies? >> >> The distribution has been sending blank cookies since first open source release. >> The distribution do not send the cookie in clear text but rely on a MD5 challenge procedure >> at connection setup. > > Hi Tony! > > Indeed - and I will take credit for pestering you to fix that just > before the first open source release:-) (I will not divulge what it did > before that...). > Yes, master. :-) >> So Erlang is more likely to be vulnerable to connection hijacking since not every message >> is signed. > > Yes - the *default* distribution fulfills none of the CIA requirements > (no, not that evil US thing, but Confidentiality, Integrity, and > Availability). But this has nothing to do with the authentication > mechanism as such, and can be fixed by using TLS - which also brings one > or more other authentication mechanisms, but they are not in any > fundamental sense more "secure" than the cookie authentication. > >> So keep the nodes safe and away from random users. At least until we get Safe Erlang ( any decade now ) > > Sure - but this point is actually also confusing in a cookie discussion, > as shown by other messages in this thread - it is about the > *authorization* you automatically get at the point when you have managed > to break the authentication mechanism - i.e. basically you can do > "anything". But this is independent of the strength of the > authentication mechanism itself. > My point was just that if the city wall has collapsed you could still have a couple of more walls to protect your but. > I do find it rather tiresome with this constant ridicule of the cookie > authentication from people who haven't even bothered to do a basic > investigation of how it works, let alone done any actual security > analysis. > > And just to put another myth to death, no, you are not required to use > the same cookie on all your distributed erlang nodes - every node is > capable of maintaining a specific cookie for every other node, RTFM > erlang:set_cookie/2. > > It is absolutely true that *maintaining* security in a network with > cookie-based authentication can be troublesome, and that e.g. TLS with > certificate authentication can do much better in that respect, as long > as you have mechanisms for certificate revocation properly set up (which > in turn is not entirely trivial to do). > > But again, as long as you do not throw your cookies around, AFAIK no-one > has demonstrated any fundamental weakness with the mechanism as such. > > --Per From mononcqc@REDACTED Sat Jun 11 01:00:49 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 10 Jun 2016 19:00:49 -0400 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <201606102010.u5AKAlOI024412@pluto.hedeland.org> References: <419331B3-5651-43CB-AC28-01829C24CEB0@rogvall.se> <201606102010.u5AKAlOI024412@pluto.hedeland.org> Message-ID: <20160610224348.GA1116@ferdmbp.local> On 06/10, Per Hedeland wrote: >I do find it rather tiresome with this constant ridicule of the cookie >authentication from people who haven't even bothered to do a basic >investigation of how it works, let alone done any actual security >analysis. > https://twitter.com/DonAndrewBailey/status/737693679997984770 https://twitter.com/DonAndrewBailey/status/737693957656698880 https://twitter.com/DonAndrewBailey/status/737699529701490688 This security researcher appears to have found issues with it as recently as the last 30 days. From per@REDACTED Sat Jun 11 15:15:32 2016 From: per@REDACTED (Per Hedeland) Date: Sat, 11 Jun 2016 15:15:32 +0200 (CEST) Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <20160610224348.GA1116@ferdmbp.local> Message-ID: <201606111315.u5BDFW1G027691@pluto.hedeland.org> Fred Hebert wrote: > >On 06/10, Per Hedeland wrote: >>I do find it rather tiresome with this constant ridicule of the cookie >>authentication from people who haven't even bothered to do a basic >>investigation of how it works, let alone done any actual security >>analysis. >> > >https://twitter.com/DonAndrewBailey/status/737693679997984770 >https://twitter.com/DonAndrewBailey/status/737693957656698880 >https://twitter.com/DonAndrewBailey/status/737699529701490688 > >This security researcher appears to have found issues with it as >recently as the last 30 days. He claims to have found *bugs*, and reported them to the OTP team - great! However it's not like the existence of bugs in software is something unique to the OTP implementation of the cookie-based authentication scheme (or wherever the bugs are). And I have never seen (I may of course have missed it) the argument "because it has bugs" being brought forward by those who claim "the cookie authentication isn't secure" on this mailing list. In fact there typically isn't any "because" at all. As for actual analysis of the mechanism as such, the only thing I can find is the statement "cookie key space by default is 26^20" - given as a good property, but it certainly makes me wonder about the depth of such an analysis, if it has indeed been undertaken (I see no claim that it has). The cookie is an arbitrary atom, and thus the value space is larger than 256^255 ((1 - 256^256)/(1 - 256) to be precise, or just a few bits short of 2048) - period. I guess his "default" refers to the cookie that is auto-generated if you don't provide one - I'm not sure why you would want to make use of that if you are attempting to set up a "secure" network of Erlang nodes. Besides the fact that you need to additionally figure out the value space of that generation, and analyze the actual mechanism used (maybe it's even bad old random(3):-), it seems pretty impractical compared to pre-generating the cookies yourself. --Per From mononcqc@REDACTED Sat Jun 11 15:32:22 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 11 Jun 2016 09:32:22 -0400 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <201606111315.u5BDFW1G027691@pluto.hedeland.org> References: <20160610224348.GA1116@ferdmbp.local> <201606111315.u5BDFW1G027691@pluto.hedeland.org> Message-ID: <20160611133221.GB1116@ferdmbp.local> On 06/11, Per Hedeland wrote: >As for actual analysis of the mechanism as such, the only thing I can >find is the statement "cookie key space by default is 26^20" - given as >a good property, but it certainly makes me wonder about the depth of >such an analysis, if it has indeed been undertaken (I see no claim that >it has). The cookie is an arbitrary atom, and thus the value space is >larger than 256^255 ((1 - 256^256)/(1 - 256) to be precise, or just a >few bits short of 2048) - period. The space is likely smaller since you're going for an MD5 challenge and only have to generate a conflicting MD5, not the actual cookie I believe? The challenge itself uses the cookie and then 'salts' it with the result of this function: https://github.com/erlang/otp/blob/e1489c448b7486cdcfec6a89fea238d88e6ce2f3/lib/kernel/src/dist_util.erl#L376-L388 which has no great source of randomness, especially on mostly idle nodes I'd guess. From per@REDACTED Sat Jun 11 23:20:01 2016 From: per@REDACTED (Per Hedeland) Date: Sat, 11 Jun 2016 23:20:01 +0200 (CEST) Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <20160611133221.GB1116@ferdmbp.local> Message-ID: <201606112120.u5BLK1kL028958@pluto.hedeland.org> Fred Hebert wrote: > >On 06/11, Per Hedeland wrote: >>As for actual analysis of the mechanism as such, the only thing I can >>find is the statement "cookie key space by default is 26^20" - given as >>a good property, but it certainly makes me wonder about the depth of >>such an analysis, if it has indeed been undertaken (I see no claim that >>it has). The cookie is an arbitrary atom, and thus the value space is >>larger than 256^255 ((1 - 256^256)/(1 - 256) to be precise, or just a >>few bits short of 2048) - period. > >The space is likely smaller since you're going for an MD5 challenge and >only have to generate a conflicting MD5, not the actual cookie I >believe? Hm, so now you're doing a security analysis, and asking me to comment on it? Probably a case of the blind leading the blind, but OK... The value space of the cookie is what it is, independent of how it is used - I was merely pointing out that the only statement that seemed indicative of a security analysis being done was inaccurate. How the value space of the cleartext and its relation to the value space (and quality?) of the digest translates into "security" in this context isn't something I feel qualified to comment on, but I think I know enough to say that it isn't just a matter of counting the respective number of bits. >The challenge itself uses the cookie and then 'salts' it with the result >of this function: >https://github.com/erlang/otp/blob/e1489c448b7486cdcfec6a89fea238d88e6ce2f3/lib/kernel/src/dist_util.erl#L376-L388 >which has no great source of randomness, especially on mostly idle nodes >I'd guess. AFAIK the "salt", or as it's more commonly called in this context, the "nonce", has no actual requirement of randomness, only of variation - i.e. it should not be repeated. It could quite possibly be done better, but it's certainly not obvious to me that there is any actual problem with the current implementation. --Per From per@REDACTED Sat Jun 11 23:54:06 2016 From: per@REDACTED (Per Hedeland) Date: Sat, 11 Jun 2016 23:54:06 +0200 (CEST) Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: Message-ID: <201606112154.u5BLs6re029033@pluto.hedeland.org> Ulf Wiger wrote: > >We should be able to agree that: > >- the cookie strategy and challenge aren???t necessarily broken in principle >- the MD5 hash is a weakness which could be addressed >- using TCP opens up for eavesdropping and MITM attacks >- the biggest weakness is the (human) practice of using hard-coded simple cookies for convenience >- little (albeit some) support exists for applying a more sophisticated cookie management strategy >- The simplest advice to heed is ???don???t expose your dist ports to strangers" I can agree to all of that, except that I'm not sure that the "weakness" of MD5, which pertains to its ability to produce a digest of a cleartext that can't be reproduced by applying it to a different cleartext, even when the original cleartext is known (i.e. the case of using a digest + signature to ensure integrity), is significant in this particular context. And unfortunately that's the only one of your points that addresses the security of the cookie authentication mechanism as such... --Per From moxford@REDACTED Sun Jun 12 01:56:50 2016 From: moxford@REDACTED (Mike Oxford) Date: Sat, 11 Jun 2016 16:56:50 -0700 Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: <201606112154.u5BLs6re029033@pluto.hedeland.org> References: <201606112154.u5BLs6re029033@pluto.hedeland.org> Message-ID: ??I've not looked into the Erlang use of the key, I'm only commenting on the use of MD5 sums ... MD5 effectively "normalizes" the input to a discrete output space, by design, as the output length/set is finite. ?If MD5 is weak to hash-collisions (which it is, relatively) then I don't need to spray the cluster with a single discrete value to get the command accepted - the range of possibilities goes from "1" to "N" where "N" is the discrete (but yet unknown) set of values which produce the collision. You may have used a super high-tech and secure 2048-bit key but if it just happens to collide with "password" then it'll be found relatively quickly. Again, haven't looked at Erlang code to see how it's actually used in the context of this discussion. -mox ? On Sat, Jun 11, 2016 at 2:54 PM, Per Hedeland wrote: > Ulf Wiger wrote: > > > >We should be able to agree that: > > > >- the cookie strategy and challenge aren???t necessarily broken in > principle > >- the MD5 hash is a weakness which could be addressed > >- using TCP opens up for eavesdropping and MITM attacks > >- the biggest weakness is the (human) practice of using hard-coded simple > cookies for convenience > >- little (albeit some) support exists for applying a more sophisticated > cookie management strategy > >- The simplest advice to heed is ???don???t expose your dist ports to > strangers" > > I can agree to all of that, except that I'm not sure that the "weakness" > of MD5, which pertains to its ability to produce a digest of a cleartext > that can't be reproduced by applying it to a different cleartext, even > when the original cleartext is known (i.e. the case of using a digest + > signature to ensure integrity), is significant in this particular > context. And unfortunately that's the only one of your points that > addresses the security of the cookie authentication mechanism as such... > > --Per > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sun Jun 12 14:58:19 2016 From: per@REDACTED (Per Hedeland) Date: Sun, 12 Jun 2016 14:58:19 +0200 (CEST) Subject: [erlang-questions] Erlang cookies are secure In-Reply-To: Message-ID: <201606121258.u5CCwJ3d031356@pluto.hedeland.org> Mike Oxford wrote: >I've not looked into the Erlang use of the key, I'm only commenting on >the use of MD5 sums ... > >MD5 effectively "normalizes" the input to a discrete output space, by >design, as the output length/set is finite. Yes, this is obviously true of all digest algorithms. >If MD5 is weak to hash-collisions (which it is, relatively) then I don't >need to spray the cluster with a single discrete value to get the command >accepted - the range of possibilities goes from "1" to "N" where "N" is the >discrete (but yet unknown) set of values which produce the collision. > >You may have used a super high-tech and secure 2048-bit key but if it just >happens to collide with "password" then it'll be found relatively quickly. > >Again, haven't looked at Erlang code to see how it's actually used in the >context of this discussion. OK, but even from just following the discussion, it should be clear that the authentication isn't done by sending the MD5 of only the cookie, which your reasoning seems to assume - this would be completely pointless, as it would be basically equivalent to sending the cookie itself from a security perspective. Google "digest authentication" to find examples of the general principle. --Per From neerajsharma.live@REDACTED Sun Jun 12 19:12:43 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Sun, 12 Jun 2016 22:42:43 +0530 Subject: [erlang-questions] Looking for details on static linking of NIFs Message-ID: Hi, I want to use crypto (and other NIFs) without letting Erlang dynamically load them when started. After going through a number of documents and particularly the following two I still cannot figure out why Erlang tries to load the crypto library when trying to start the crypto or via crypto:start(). http://erlang.org/doc/man/erl_nif.html http://erlang.org/doc/installation_guide/INSTALL.html I am building Erlang with --enable-static-nifs and --disable-dynamic-ssl-lib. Any clue or pointers to internal operations of the static NIFs will be of great help. Thanks, Neeraj From technion@REDACTED Tue Jun 14 03:52:20 2016 From: technion@REDACTED (Technion) Date: Tue, 14 Jun 2016 01:52:20 +0000 Subject: [erlang-questions] Pipes - systemd Message-ID: Hi, I've had a lot of frustrating times with this issue and wanted to mention it before anyone else does. Specifically, my erl pipes kept getting deleted, meaning you couldn't attach, or cleanly shutdown services. After far too long, I've tracked this to a new systemd service, and found the fix involves editing this file: /usr/lib/tmpfiles.d/tmp.conf And telling it not to apply the default ten day lifetime on this folder: x /tmp/erl_pipes I'm expecting that will sort it out (although I know now I need ten days to tell). -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Tue Jun 14 00:02:07 2016 From: vans_163@REDACTED (Vans S) Date: Mon, 13 Jun 2016 22:02:07 +0000 (UTC) Subject: [erlang-questions] SSL sni_fun modification to return empty string if no SNI References: <784961013.2480763.1465855327414.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <784961013.2480763.1465855327414.JavaMail.yahoo@mail.yahoo.com> The sni_fun currently imposes a logic problem on what to do with no SNI was supplied. ?Since it is callback oriented it is difficult to code against. I want to suggest to patch SSL module further to allow sni_fun callback to always be called whether SNI was supplied or not by the client. And another possible addition is to allow to partially complete a handshake, getting the option to modifiy the ssloptions() after the handshake was sent by the client but before the server reply. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjotr.public12@REDACTED Tue Jun 14 09:50:05 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Tue, 14 Jun 2016 09:50:05 +0200 Subject: [erlang-questions] Reproducible Erlang builds - still some time stamps (non-deterministic build) In-Reply-To: <20160612083500.GA13121@thebird.nl> References: <20160609173127.GA30990@thebird.nl> <20160612083500.GA13121@thebird.nl> Message-ID: <20160614075005.GA29200@thebird.nl> Dear all, At http://biobeat.org/report.html a diffoscope of two builds on GNU Guix of Erlang 19.0-rc2 (made by leo). It turns out some files are still different between builds. Some .script files contain time stamps like: %% script generated at {2016,6,11}{19,5,37} The snmp, otp_mips, os_mon, gs modules also injects time stamps. And some beam files in cosNotification and cosProperty introduce differences. I think none of these modules is critical and we can safely purge these modules from GNU Guix so it becomes deterministic (we only accept deterministic software packages) for the time being. Do you agree? Pj. From henrik.x.nord@REDACTED Tue Jun 14 10:28:13 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Tue, 14 Jun 2016 10:28:13 +0200 Subject: [erlang-questions] Patch package OTP 18.3.4 released Message-ID: <575FC01D.6030404@ericsson.com> Patch Package: OTP 18.3.4 Git Tag: OTP-18.3.4 Date: 2016-06-14 Trouble Report Id: OTP-13490, OTP-13546, OTP-13635, OTP-13663, OTP-13670 Seq num: seq13132 System: OTP Release: 18 Application: inets-6.2.4, ssl-7.3.3 Predecessor: OTP 18.3.3 Check out the git tag OTP-18.3.4, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- inets-6.2.4 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.2.4 application can be applied independently of other applications on a full OTP 18 installation. --- Improvements and New Features --- OTP-13663 Application(s): inets Related Id(s): seq13132 Handle multiple \t in mime types file Full runtime dependencies of inets-6.2.4: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- ssl-7.3.3 ------------------------------------------------------- --------------------------------------------------------------------- The ssl-7.3.3 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13546 Application(s): ssl Correct ssl:prf/5 to use the negotiated cipher suite's prf function in ssl:prf/5 instead of the default prf. OTP-13635 Application(s): ssl Timeouts may have the value 0, guards have been corrected to allow this OTP-13670 Application(s): ssl Change of internal handling of hash sign pairs as the used one enforced to much restrictions making some valid combinations unavailable. --- Improvements and New Features --- OTP-13490 Application(s): ssl Create a little randomness in sending of session invalidation messages, to mitigate load when whole table is invalidated. Full runtime dependencies of ssl-7.3.3: crypto-3.3, erts-7.0, inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 --------------------------------------------------------------------- --- Thanks to ------------------------------------------------------- --------------------------------------------------------------------- Joe DeVivo, Kenneth Lakin --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From mikpelinux@REDACTED Tue Jun 14 11:43:36 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 14 Jun 2016 11:43:36 +0200 Subject: [erlang-questions] Reproducible Erlang builds - still some time stamps (non-deterministic build) In-Reply-To: <20160614075005.GA29200@thebird.nl> References: <20160609173127.GA30990@thebird.nl> <20160612083500.GA13121@thebird.nl> <20160614075005.GA29200@thebird.nl> Message-ID: <22367.53704.838456.854291@gargle.gargle.HOWL> Pjotr Prins writes: > Dear all, > > At http://biobeat.org/report.html a diffoscope of two builds on GNU > Guix of Erlang 19.0-rc2 (made by leo). It turns out some files are > still different between builds. Some .script files contain time stamps > like: > > %% script generated at {2016,6,11}{19,5,37} > > The snmp, otp_mips, os_mon, gs modules also injects time stamps. > > And some beam files in cosNotification and cosProperty introduce > differences. > > I think none of these modules is critical and we can safely purge > these modules from GNU Guix so it becomes deterministic (we only > accept deterministic software packages) for the time being. Do you > agree? No. I acknowledge your need for deterministic builds, but the solution ought to be to eliminate the source of the non-determinism, not the files themselves. So: 1. Patch whatever inserts these time stamps to not do that, or 2. Post-process affected files and remove the time stamps. From jesper.louis.andersen@REDACTED Tue Jun 14 12:45:59 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 14 Jun 2016 12:45:59 +0200 Subject: [erlang-questions] Reproducible Erlang builds - still some time stamps (non-deterministic build) In-Reply-To: <22367.53704.838456.854291@gargle.gargle.HOWL> References: <20160609173127.GA30990@thebird.nl> <20160612083500.GA13121@thebird.nl> <20160614075005.GA29200@thebird.nl> <22367.53704.838456.854291@gargle.gargle.HOWL> Message-ID: I don't think it is good to remove modules from the OTP installation and then proceed to call it Erlang/OTP. Likewise, it is bad to split OTP into many submodules so people can mix'n'match subparts of the installation. Often, this adds another step for newcomers to the language since they are missing important parts of the installation. I'd much rather you patch the affected files. For instance by replacing the timestamps with SHA256 checksums ( crypto:hash(sha256, File) ), since this will still make the files change, yet be deterministic in their generation. On Tue, Jun 14, 2016 at 11:43 AM, Mikael Pettersson wrote: > Pjotr Prins writes: > > Dear all, > > > > At http://biobeat.org/report.html a diffoscope of two builds on GNU > > Guix of Erlang 19.0-rc2 (made by leo). It turns out some files are > > still different between builds. Some .script files contain time stamps > > like: > > > > %% script generated at {2016,6,11}{19,5,37} > > > > The snmp, otp_mips, os_mon, gs modules also injects time stamps. > > > > And some beam files in cosNotification and cosProperty introduce > > differences. > > > > I think none of these modules is critical and we can safely purge > > these modules from GNU Guix so it becomes deterministic (we only > > accept deterministic software packages) for the time being. Do you > > agree? > > No. > > I acknowledge your need for deterministic builds, but the solution > ought to be to eliminate the source of the non-determinism, not the > files themselves. So: > > 1. Patch whatever inserts these time stamps to not do that, or > 2. Post-process affected files and remove the time stamps. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Tue Jun 14 14:17:03 2016 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Tue, 14 Jun 2016 15:17:03 +0300 Subject: [erlang-questions] How to encode data with private key? Message-ID: <67f654d1-5fb1-f473-3ab1-8699a1299891@gmail.com> Hello, I am trying to implement the OAuth 2 Google web service 2-legged flow. When creating the service, I was granted a private key that I have to use to sign the request. I try the following: PrivKey = "-----BEGIN RSA PRIVATE KEY----- ...", [PrivEntry] = public_key:pem_decode(list_to_binary(PrivKey)), Priv = public_key:pem_entry_decode(PrivEntry), public_key:encrypt_private(Data, Priv). But I get the following error: ** exception error: no function clause matching public_key:encrypt_private(<<"..".>>, <<48,130,4,...>>, []) (public_key.erl, line 364) in function jwt:encode/4 (../src/jwt.erl, line 25) Does anyone know what the format of the Key must be for public_key:encrypt_private(PlainText, Key)? Thank you, Martin From t@REDACTED Tue Jun 14 16:45:33 2016 From: t@REDACTED (Tristan Sloughter) Date: Tue, 14 Jun 2016 09:45:33 -0500 Subject: [erlang-questions] Pipes - systemd In-Reply-To: References: Message-ID: <1465915533.3624188.637367641.227C9966@webmail.messagingengine.com> Thanks for sharing! That is going to save me a lot of future debugging time I'm sure :) This got me thinking, should these files really go to /tmp/? Would /var/run be more appropriate? -- Tristan Sloughter t@REDACTED On Mon, Jun 13, 2016, at 08:52 PM, Technion wrote: > Hi, > > I've had a lot of frustrating times with this issue and wanted to > mention it before anyone else does. > > Specifically, my erl pipes kept getting deleted, meaning you couldn't > attach, or cleanly shutdown services. > > After far too long, I've tracked this to a new systemd service, and > found the fix involves editing this file: > > /usr/lib/tmpfiles.d/tmp.conf > > And telling it not to apply the default ten day lifetime on > this folder: > > x /tmp/erl_pipes > > I'm expecting that will sort it out (although I know now I need ten > days to tell). > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Tue Jun 14 17:19:45 2016 From: ali.sabil@REDACTED (Ali Sabil) Date: Tue, 14 Jun 2016 15:19:45 +0000 Subject: [erlang-questions] Pipes - systemd In-Reply-To: <1465915533.3624188.637367641.227C9966@webmail.messagingengine.com> References: <1465915533.3624188.637367641.227C9966@webmail.messagingengine.com> Message-ID: I think /var/run is more appropriate indeed On Tue, Jun 14, 2016 at 4:45 PM Tristan Sloughter wrote: > Thanks for sharing! That is going to save me a lot of future debugging > time I'm sure :) > > This got me thinking, should these files really go to /tmp/? Would > /var/run be more appropriate? > > -- > Tristan Sloughter > t@REDACTED > > > > On Mon, Jun 13, 2016, at 08:52 PM, Technion wrote: > > Hi, > > > I've had a lot of frustrating times with this issue and wanted to mention > it before anyone else does. > > > Specifically, my erl pipes kept getting deleted, meaning you couldn't > attach, or cleanly shutdown services. > > > After far too long, I've tracked this to a new systemd service, and found > the fix involves editing this file: > > > /usr/lib/tmpfiles.d/tmp.conf > > > And telling it not to apply the default ten day lifetime on this folder: > > > x /tmp/erl_pipes > > > I'm expecting that will sort it out (although I know now I need ten days > to tell). > *_______________________________________________* > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Jun 14 17:49:37 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 14 Jun 2016 17:49:37 +0200 Subject: [erlang-questions] [ANN] PropEr version 1.2 Message-ID: <57602791.40507@cs.ntua.gr> We are happy to announce a new tagged version of PropEr, a Quickcheck inspired open source property based testing tool for Erlang. The new version is available at: https://github.com/manopapad/proper/releases This is a (long overdue) release that incorporates a long list of changes done since 2013 when the previous version was tagged, incorporates bug fixes and extensions to its basic functionality, many of which were sent by users. We sincerely thank thank them all. Please refer to the GitHub repository to see what has changed. However, as far as we know all changes are backwards compatible and therefore we recommend all users to upgrade to this tagged version. Perhaps the most important feature of this release is that it supports a wide range of Erlang/OTP releases. It most probably still works on R13B*, it is known to work correctly with OTP R15B* to 18.x. It also contains changes that are necessary for PropEr to also work OK on the upcoming 19.0. Most likely, it will be the last version to support all these Erlang/OTP releases. Our current plan is to start working on a 2.0 version of PropEr that will finally provide support for maps and will therefore only work on Erlang/OTP releases 17.x and newer. Contact us privately, using the contact address at the PropEr website (http://proper.softlab.ntua.gr) if you still need support for Erlang/OTP releases prior to 17.x. Once again, we would like to thank users and contributors to PropEr. Enjoy! Kostis From neerajsharma.live@REDACTED Tue Jun 14 19:56:17 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Tue, 14 Jun 2016 23:26:17 +0530 Subject: [erlang-questions] Looking for details on static linking of NIFs In-Reply-To: References: Message-ID: On Sun, Jun 12, 2016 at 10:42 PM, Neeraj Sharma wrote: > Hi, > > I want to use crypto (and other NIFs) without letting Erlang > dynamically load them when started. After going through a number of > documents and particularly the following two I still cannot figure out > why Erlang tries to load the crypto library when trying to start the > crypto or via crypto:start(). > > http://erlang.org/doc/man/erl_nif.html > http://erlang.org/doc/installation_guide/INSTALL.html > > I am building Erlang with --enable-static-nifs and > --disable-dynamic-ssl-lib. Any clue or pointers to internal operations > of the static NIFs will be of great help. > I finally got my build working and ran a quick test to see the successful outcome. I started direct conversation with Lukas (github: garazdawi) and got some pointers from him as well. In the end I found a build issue with Erlang 18.3 and 19.0-rc2 for my use case, where I am cross compiling and additionally providing static libraries for openssl as well. It appears that this use case is probably not tested and hence source of failure and confusion on my part. Eventually I found that using "--enable-static-nifs" enables static compilation for both crypto and asn1 by default so providing absolute paths to archives to both the NIFs is unnecessary (though that will work too). Additionally, there is trickiness to using a custom built openssl where certain ciphering alogs are required by the latest Erlang 19.0-rc2 and hence must be available. Also, as giving in the installation docs LIBS="${OPENSSL_PATH}/lib/libcrypto.a" is also required before running configure so that the crypto NIF gets built correctly. There is a build issue for both Erlang 18.3 and 19.0-rc2 (and probably others) in the above scenario, which I will be posting (as a patch) on github/erlang/otp tonight or tomorrow. Cheers, Neeraj From ok@REDACTED Wed Jun 15 05:38:55 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 15 Jun 2016 15:38:55 +1200 Subject: [erlang-questions] [ANN] PropEr version 1.2 In-Reply-To: <57602791.40507@cs.ntua.gr> References: <57602791.40507@cs.ntua.gr> Message-ID: <99336c4f-8942-8219-ab79-5b0a000e74e4@cs.otago.ac.nz> On 15/06/16 3:49 AM, Kostis Sagonas wrote: > We are happy to announce a new tagged version of PropEr, You heroes! From mrtndimitrov@REDACTED Wed Jun 15 10:03:53 2016 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Wed, 15 Jun 2016 11:03:53 +0300 Subject: [erlang-questions] How to encode data with private key? In-Reply-To: <67f654d1-5fb1-f473-3ab1-8699a1299891@gmail.com> References: <67f654d1-5fb1-f473-3ab1-8699a1299891@gmail.com> Message-ID: <1c8236e5-a35c-d056-927c-95b38186a056@gmail.com> I figured it out. The private key provided by Google contains algorithm and version information. It was with "-----BEGIN PRIVATE KEY-----" header. I had to convert it to only RSA key with "openssl rsa -in server.key -out server_new.key". Then I use the following to sign the data: [PrivEntry] = public_key:pem_decode(list_to_binary(PrivateKey)), Priv = public_key:pem_entry_decode(PrivEntry), public_key:sign(Data, sha256, Priv); That's how to sign Google's JSON Web Tokens. Hope this will be helpful to someone else. Regards, Martin On 6/14/2016 3:17 PM, Martin Koroudjiev wrote: > Hello, > > I am trying to implement the OAuth 2 Google web service 2-legged flow. > When creating the service, I was granted a private key that I have to > use to sign the request. I try the following: > > PrivKey = "-----BEGIN RSA PRIVATE KEY----- ...", > [PrivEntry] = public_key:pem_decode(list_to_binary(PrivKey)), > Priv = public_key:pem_entry_decode(PrivEntry), > public_key:encrypt_private(Data, Priv). > > But I get the following error: > > ** exception error: no function clause matching > public_key:encrypt_private(<<"..".>>, > > <<48,130,4,...>>, > > []) (public_key.erl, line 364) > in function jwt:encode/4 (../src/jwt.erl, line 25) > > Does anyone know what the format of the Key must be for > public_key:encrypt_private(PlainText, Key)? > > Thank you, > Martin > From neerajsharma.live@REDACTED Wed Jun 15 10:43:53 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Wed, 15 Jun 2016 14:13:53 +0530 Subject: [erlang-questions] How to encode data with private key? In-Reply-To: <1c8236e5-a35c-d056-927c-95b38186a056@gmail.com> References: <67f654d1-5fb1-f473-3ab1-8699a1299891@gmail.com> <1c8236e5-a35c-d056-927c-95b38186a056@gmail.com> Message-ID: On Wed, Jun 15, 2016 at 1:33 PM, Martin Koroudjiev wrote: > I figured it out. The private key provided by Google contains algorithm > and version information. It was with "-----BEGIN PRIVATE KEY-----" > header. I had to convert it to only RSA key with "openssl rsa -in > server.key -out server_new.key". Then I use the following to sign the data: > > [PrivEntry] = public_key:pem_decode(list_to_binary(PrivateKey)), > Priv = public_key:pem_entry_decode(PrivEntry), > public_key:sign(Data, sha256, Priv); > > That's how to sign Google's JSON Web Tokens. Hope this will be helpful > to someone else. > Hi Martin, I was about to go that path and your email saved me the effort :) Thanks for sharing, Neeraj From erlang@REDACTED Wed Jun 15 14:04:02 2016 From: erlang@REDACTED (Stefan Marr) Date: Wed, 15 Jun 2016 14:04:02 +0200 Subject: [erlang-questions] CFP: 1st Workshop on Meta-Programming Techniques and Reflection (META) with SPLASH 2016 Message-ID: **Call for Papers** ======================================================================== 1st Workshop on Meta-Programming Techniques and Reflection (META?16) Co-located with SPLASH 2016 October 30, 2016, Amsterdam, Netherlands http://2016.splashcon.org/track/meta2016 Follow us on twitter @MetaAtSPLASH ======================================================================== The Meta?16 workshop aims to bring together researchers working on metaprogramming and reflection, as well as users building applications, language extensions such as contracts, or software tools. With the changing hardware and software landscape, and increased heterogeneity of systems, metaprogramming becomes an important research topic to handle the associate complexity once more. Contributions to the workshop are welcome on a wide range of topics related to design, implementation, and application of metaprogramming techniques, as well as empirical studies on and typing for such systems and languages. ### Important Dates 27 Jul 2016 - Abstract Submission 1 Aug 2016 - Paper Submission 5 Sep 2016 - Notification 2 Sep 2016 - Demo, position or work-in-progress paper submission 23 Sep 2016 - Demo, position or work-in-progress paper notification 30 Oct 2016 Workshop ### Topics of Interest The workshop is a venue for all approaches that embrace metaprogramming: - from static to dynamic techniques - reflection, meta-level architectures, staging, open language runtimes applications to middleware, frameworks, and DSLs - optimization techniques to minimize runtime overhead - contract systems, or typing of reflective programs reflection and metaobject protocols to enable tooling - case studies and evaluation of such techniques, e.g., to build applications, language extensions, or tools - empirical evaluation of metaprogramming solutions - security in reflective systems and capability-based designs - meta-level architectures and reflective middleware for modern runtime platforms (e.g. IoT, cyber-physical systems, mobile/cloud/grid computing, etc) - surveys, conceptualization, and taxonomization of existing approaches ### Types of Submissions and Workshop format We welcome both technical papers and work-in-progress and position papers from the academic as well as industrial perspective. Position paper should present new ideas and emerging problems and take a perhaps controversial stance on a specific topic and argue the position well. Furthermore, we plan interactive brainstorming and demonstration sessions between the formal presentations to enable an active exchange of ideas. The workshop papers will be put on the workshop website, if not requested otherwise by the authors, but they are not part of formal proceedings. Papers are to be submitted using the ACM sigplanconf style at 10pt font size. See http://www.sigplan.org/Resources/Author/. - demos and posters: 1-page abstract - position and work-in-progress paper: 1-4 pages - technical paper: up to 8 pages Demos, posters, position and work-in-progress papers can be submitted on a second, later deadline to discuss the latest results and current work. For the submission, please use the submission system at: https://meta16.hotcrp.com/ ### Organizing Committee Shigeru Chiba, University of Tokyo Elisa Gonzalez Boix, Vrije Universiteit Brussel Stefan Marr, Johannes Kepler University Linz ### Program Committee Marcus Denker, INRIA, France Robert Hirschfeld, HPI, Germany Hridesh Rajan, Iowa State University, USA Romain Rouvoy, University Lille 1 and INRIA, France Eric Tanter, University of Chile, Chile Tom Van Cutsem, Bell Labs, Belgium Tijs van der Storm, CWI, Netherlands Takuo Watanabe, Tokyo Institute of Technology, Japan From g@REDACTED Wed Jun 15 16:21:37 2016 From: g@REDACTED (Garrett Smith) Date: Wed, 15 Jun 2016 09:21:37 -0500 Subject: [erlang-questions] Pipes - systemd In-Reply-To: References: Message-ID: Once you fix that behavior, you'll probably be up against others that sprout up in time. Don't use /tmp for process files, unless they're ad hoc or short lived. As others have said /var/run or /run is generally where you want those to land. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard On Mon, Jun 13, 2016 at 8:52 PM, Technion wrote: > Hi, > > > I've had a lot of frustrating times with this issue and wanted to mention it > before anyone else does. > > > Specifically, my erl pipes kept getting deleted, meaning you couldn't > attach, or cleanly shutdown services. > > > After far too long, I've tracked this to a new systemd service, and found > the fix involves editing this file: > > > /usr/lib/tmpfiles.d/tmp.conf > > > And telling it not to apply the default ten day lifetime on this folder: > > > x /tmp/erl_pipes > > > I'm expecting that will sort it out (although I know now I need ten days to > tell). > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From t@REDACTED Wed Jun 15 16:27:08 2016 From: t@REDACTED (Tristan Sloughter) Date: Wed, 15 Jun 2016 09:27:08 -0500 Subject: [erlang-questions] Pipes - systemd In-Reply-To: References: Message-ID: <1466000828.3897588.638514281.0758D4A1@webmail.messagingengine.com> Yea, I assume people are using /tmp because rebar 2.x and relx (based on the script from rebar 2.x) uses /tmp in the release start script. I am going to update the relx script to use /var/run by default, which will also update rebar3. -- Tristan Sloughter t@REDACTED On Wed, Jun 15, 2016, at 09:21 AM, Garrett Smith wrote: > Once you fix that behavior, you'll probably be up against others that > sprout up in time. Don't use /tmp for process files, unless they're ad > hoc or short lived. As others have said /var/run or /run is generally > where you want those to land. > > https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard > > On Mon, Jun 13, 2016 at 8:52 PM, Technion wrote: > > Hi, > > > > > > I've had a lot of frustrating times with this issue and wanted to mention it > > before anyone else does. > > > > > > Specifically, my erl pipes kept getting deleted, meaning you couldn't > > attach, or cleanly shutdown services. > > > > > > After far too long, I've tracked this to a new systemd service, and found > > the fix involves editing this file: > > > > > > /usr/lib/tmpfiles.d/tmp.conf > > > > > > And telling it not to apply the default ten day lifetime on this folder: > > > > > > x /tmp/erl_pipes > > > > > > I'm expecting that will sort it out (although I know now I need ten days to > > tell). > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From technion@REDACTED Thu Jun 16 05:24:58 2016 From: technion@REDACTED (Technion) Date: Thu, 16 Jun 2016 03:24:58 +0000 Subject: [erlang-questions] Pipes - systemd In-Reply-To: <1466000828.3897588.638514281.0758D4A1@webmail.messagingengine.com> References: , <1466000828.3897588.638514281.0758D4A1@webmail.messagingengine.com> Message-ID: That's correct for me - given it wasn't blindingly obvious how to change this in rebar3 (ie, under either the "getting started" or "configuration" section of documentation), it quickly becomes one of those "do I really want to change this and deal with being non-standard"? Those sorts of situations generally outweigh filesystem standards that a particular product may or may not use. ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Tristan Sloughter Sent: Thursday, 16 June 2016 12:27:08 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Pipes - systemd Yea, I assume people are using /tmp because rebar 2.x and relx (based on the script from rebar 2.x) uses /tmp in the release start script. I am going to update the relx script to use /var/run by default, which will also update rebar3. -- Tristan Sloughter t@REDACTED On Wed, Jun 15, 2016, at 09:21 AM, Garrett Smith wrote: > Once you fix that behavior, you'll probably be up against others that > sprout up in time. Don't use /tmp for process files, unless they're ad > hoc or short lived. As others have said /var/run or /run is generally > where you want those to land. > > https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard > > On Mon, Jun 13, 2016 at 8:52 PM, Technion wrote: > > Hi, > > > > > > I've had a lot of frustrating times with this issue and wanted to mention it > > before anyone else does. > > > > > > Specifically, my erl pipes kept getting deleted, meaning you couldn't > > attach, or cleanly shutdown services. > > > > > > After far too long, I've tracked this to a new systemd service, and found > > the fix involves editing this file: > > > > > > /usr/lib/tmpfiles.d/tmp.conf > > > > > > And telling it not to apply the default ten day lifetime on this folder: > > > > > > x /tmp/erl_pipes > > > > > > I'm expecting that will sort it out (although I know now I need ten days to > > tell). > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jun 16 07:25:47 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 16 Jun 2016 08:25:47 +0300 Subject: [erlang-questions] Cost of reduction penalty when sending message to process with long queue Message-ID: We have met strange situation when lager_event is overloaded with about million of messages and all processes trying to log are mentioned in sys monitor with long_schedule warning. I thought that it may be a problem with penalisation of sender with some reductions but couldn't find exact formula in OTP sources. Where is it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Thu Jun 16 08:00:28 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 16 Jun 2016 08:00:28 +0200 Subject: [erlang-questions] Cost of reduction penalty when sending message to process with long queue In-Reply-To: References: Message-ID: The value is gotten here: https://github.com/erlang/otp/blob/master/erts/emulator/beam/erl_message.c#L434 multiplied here: https://github.com/erlang/otp/blob/master/erts/emulator/beam/bif.c#L2310-L2311 and then bumped here: https://github.com/erlang/otp/blob/master/erts/emulator/beam/bif.c#L2374 https://github.com/erlang/otp/blob/master/erts/emulator/beam/bif.c#L2504 Lukas On Thu, Jun 16, 2016 at 7:25 AM, Max Lapshin wrote: > We have met strange situation when lager_event is overloaded with about > million of messages and all processes trying to log are mentioned in sys > monitor with long_schedule warning. > > > I thought that it may be a problem with penalisation of sender with some > reductions but couldn't find exact formula in OTP sources. Where is it? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jun 16 12:06:04 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 16 Jun 2016 13:06:04 +0300 Subject: [erlang-questions] Cost of reduction penalty when sending message to process with long queue In-Reply-To: References: Message-ID: If lager_event message queue is 1 mln length and about 300 processes are trying to send there message, may it lead to long_schedule sys monitor event? -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Thu Jun 16 13:21:40 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 16 Jun 2016 13:21:40 +0200 Subject: [erlang-questions] Cost of reduction penalty when sending message to process with long queue In-Reply-To: References: Message-ID: On Thu, Jun 16, 2016 at 12:06 PM, Max Lapshin wrote: > If lager_event message queue is 1 mln length and about 300 processes are > trying to send there message, may it lead to long_schedule sys monitor > event? > > Of the sending processes? Not because of sender punishment, if anything it should alleviate the problem. The main thing that I can think of is that very high contention on the receiving processes message queue lock is causing the sending processes to block for a long time. If you can, you may want to run with the lock counter and see if the message queue lock shows up a lot in there (especially check the histogram of wait times on the lock). -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.loscher@REDACTED Fri Jun 17 16:21:00 2016 From: andreas.loscher@REDACTED (=?UTF-8?Q?Andreas_L=c3=b6scher?=) Date: Fri, 17 Jun 2016 16:21:00 +0200 Subject: [erlang-questions] [ANN] Nifty Message-ID: <5764074C.6070901@it.uu.se> We are happy to announce Nifty a NIF interface generator that connects Erlang/OTP with programs written in C. Nifty generates wrappers in form of a NIF module for C functions that are defined in a header file. The tool is available at: https://github.com/parapluu/nifty The documentation and tutorials can be found on the webpage: http://parapluu.github.io/nifty/ Nifty automatically resolves types and comes with a support library for type and memory management. Additionally Nifty provides a safe remote execution mode that runs a NIF in a separate node. This prevents crashes like segfaults to bring down the VM and it will prevent long running NIF functions to interfere with the schedulers. Nifty also supports dirty schedulers. From 4crzen62cwqszy68g7al@REDACTED Sat Jun 18 12:54:34 2016 From: 4crzen62cwqszy68g7al@REDACTED (John Smith) Date: Sat, 18 Jun 2016 12:54:34 +0200 Subject: [erlang-questions] Disk-backed log Message-ID: For one of my systems in the financial area, I am in need of a disk-backed log that I could use as a backend for an Event Sourcing/CQRS store. Recently, I have read a bit about Kafka [1] and it seems like a good fit but, unfortunately, it is on JVM (written in Scala, to be exact) and depends heavily on ZooKeeper [2] for distribution, while I would prefer something similar for an Erlang ecosystem. Thus, ideally, I would like to have something that is: * small, * durable (checksummed, with a clear recovery procedure), * pure Erlang/Elixir (maybe with some native code, but tightly integrated), * (almost) not distributed - data fits on the single node (at least now; with replication for durability, though). Before jumping right into implementation, I have some questions: 1. Is there anything already available that fulfils above requirements? 2. Kafka uses different approach to persistence - instead of using in-process buffers and transferring data to disk, it writes straight to the filesystem which, actually, uses pagecache [3]. Can I achieve the same thing using Erlang or does it buffers writes in some other way? 3. ...also, Kafka has a log compaction [4] which can work not only in time but also in a key dimension - I need this, as I need to persist the last state for every key seen (user, transfer, etc.). As in Redis, Kafka uses the UNIX copy-on-write semantics (process fork) to avoid needless memory usage for log fragments (segments, in Kafka nomenclature) that have not changed. Can I mimick a similar behaviour in Erlang? Or if not, how can I deal with biggish (say, a couple of GB) logs that needs to be compacted? In other words, I would like to create something like a *Minimum Viable Log* (in Kafka style), only in Erlang/Elixir. I would be grateful for any kind of design/implementation hints. [1] http://kafka.apache.org/ [2] https://zookeeper.apache.org/ [3] http://kafka.apache.org/documentation.html#persistence [4] https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Jun 18 20:53:12 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 18 Jun 2016 11:53:12 -0700 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: <57659898.6080406@gmail.com> On 06/18/2016 03:54 AM, John Smith wrote: > For one of my systems in the financial area, I am in need of a disk-backed log that I could use as a backend for an Event Sourcing/CQRS store. Recently, I have read a bit about Kafka [1] and it seems like a good fit but, unfortunately, it is on JVM (written in Scala, to be exact) and depends heavily on ZooKeeper [2] for distribution, while I would prefer something similar for an Erlang ecosystem. Thus, ideally, I would like to have something that is: > > * small, > * durable (checksummed, with a clear recovery procedure), > * pure Erlang/Elixir (maybe with some native code, but tightly integrated), > * (almost) not distributed - data fits on the single node (at least now; with replication for durability, though). > > Before jumping right into implementation, I have some questions: > > 1. Is there anything already available that fulfils above requirements? > 2. Kafka uses different approach to persistence - instead of using in-process buffers and transferring data to disk, it writes straight to the filesystem which, actually, uses pagecache [3]. Can I achieve the same thing using Erlang or does it buffers writes in some other way? > 3. ...also, Kafka has a log compaction [4] which can work not only in time but also in a key dimension - I need this, as I need to persist the last state for every key seen (user, transfer, etc.). As in Redis, Kafka uses the UNIX copy-on-write semantics (process fork) to avoid needless memory usage for log fragments (segments, in Kafka nomenclature) that have not changed. Can I mimick a similar behaviour in Erlang? Or if not, how can I deal with biggish (say, a couple of GB) logs that needs to be compacted? > > In other words, I would like to create something like a *Minimum Viable Log* (in Kafka style), only in Erlang/Elixir. I would be grateful for any kind of design/implementation hints. > > [1] http://kafka.apache.org/ > [2] https://zookeeper.apache.org/ > [3] http://kafka.apache.org/documentation.html#persistence > [4] https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions If you use https://github.com/CloudI/cloudi_service_queue with https://github.com/CloudI/cloudi_core it would satisfy those requirements. However, its approach is simpler, it doesn't require checksums and the recovery is automatic (upon restart). The normal filesystem writes are buffered but can be flushed in Erlang with file:datasync/1 which is used within cloudi_service_queue source code. The logs are not compacted and the logs don't shrink to keep stuff efficient. If necessary, the incoming request rate could always be limited through cloudi_service_queue service configuration options (any of queue_limit, queue_size, or rate_request_max), to control the size of the logs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jflatow@REDACTED Sat Jun 18 21:14:30 2016 From: jflatow@REDACTED (Jared Flatow) Date: Sat, 18 Jun 2016 12:14:30 -0700 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: <4BE459E5-058A-425A-9161-1EBF022865B8@gmail.com> > On Jun 18, 2016, at 3:54 AM, John Smith <4crzen62cwqszy68g7al@REDACTED> wrote: > > For one of my systems in the financial area, I am in need of a disk-backed log that I could use as a backend for an Event Sourcing/CQRS store. Recently, I have read a bit about Kafka [1] and it seems like a good fit but, unfortunately, it is on JVM (written in Scala, to be exact) and depends heavily on ZooKeeper [2] for distribution, while I would prefer something similar for an Erlang ecosystem. Thus, ideally, I would like to have something that is: > > * small, > * durable (checksummed, with a clear recovery procedure), > * pure Erlang/Elixir (maybe with some native code, but tightly integrated), > * (almost) not distributed - data fits on the single node (at least now; with replication for durability, though). > > Before jumping right into implementation, I have some questions: > > 1. Is there anything already available that fulfils above requirements? > 2. Kafka uses different approach to persistence - instead of using in-process buffers and transferring data to disk, it writes straight to the filesystem which, actually, uses pagecache [3]. Can I achieve the same thing using Erlang or does it buffers writes in some other way? > 3. ...also, Kafka has a log compaction [4] which can work not only in time but also in a key dimension - I need this, as I need to persist the last state for every key seen (user, transfer, etc.). As in Redis, Kafka uses the UNIX copy-on-write semantics (process fork) to avoid needless memory usage for log fragments (segments, in Kafka nomenclature) that have not changed. Can I mimick a similar behaviour in Erlang? Or if not, how can I deal with biggish (say, a couple of GB) logs that needs to be compacted? > > In other words, I would like to create something like a *Minimum Viable Log* (in Kafka style), only in Erlang/Elixir. I would be grateful for any kind of design/implementation hints. > > [1] http://kafka.apache.org/ > [2] https://zookeeper.apache.org/ > [3] http://kafka.apache.org/documentation.html#persistence > [4] https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions I?ve been using this for several years: https://github.com/jflatow/erlkit/blob/master/src/log.erl It?s not checksummed, but the design is meant to be crash-proof. The log is a process with a directory. The files are soft-capped to a chunk size, the log rolls over to a new file when it hits that. The id of each log entry is {Path, Offs} relative to the log directory. Two checkpoints are kept at the top of each log file, when the log is opened, it checks for the greatest one, and checks if its a valid entry, if not, it falls back to the last one and truncates the file. When you write, you can choose whether to wait each time for the checkpoints to hit the disk or not. This is more of a primitive building block for the type of system you are talking about. I use it to build those other features (like compaction) in an ad-hoc way. Sorry for the lack of information, but its a tiny module that might be a good starting point. jared -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Jun 18 21:24:11 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 18 Jun 2016 14:24:11 -0500 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: <1466277851.1461274.641644673.6EE78B29@webmail.messagingengine.com> Do *not* try to use this https://github.com/tsloughter/vonnegut/ :) but it may help you if you decide to go down the route of implementing the features you need from something like kafka in Erlang. I'm adding an additional app that will be responsible for the role that zookeeper has with kafka, but in this case it is for chain replication. It uses the same on disk format as kafka and concept of topics/partitions, which may come in handy. While I must repeat it is very much a work in progress, something I've played around with to try out other libs like teleport, it may give you a place to start. -- Tristan Sloughter t@REDACTED On Sat, Jun 18, 2016, at 05:54 AM, John Smith wrote: > For one of my systems in the financial area, I am in need of a disk- > backed log that I could use as a backend for an Event Sourcing/CQRS > store. Recently, I have read a bit about Kafka [1] and it seems like a > good fit but, unfortunately, it is on JVM (written in Scala, to be > exact) and depends heavily on ZooKeeper [2] for distribution, while I > would prefer something similar for an Erlang ecosystem. Thus, ideally, > I would like to have something that is: > > * small, > * durable (checksummed, with a clear recovery procedure), > * pure Erlang/Elixir (maybe with some native code, but tightly > integrated), > * (almost) not distributed - data fits on the single node (at least > now; with replication for durability, though). > > Before jumping right into implementation, I have some questions: > > 1. Is there anything already available that fulfils above > requirements? > 2. Kafka uses different approach to persistence - instead of using in- > process buffers and transferring data to disk, it writes straight to > the filesystem which, actually, uses pagecache [3]. Can I achieve the > same thing using Erlang or does it buffers writes in some other way? > 3. ...also, Kafka has a log compaction [4] which can work not only in > time but also in a key dimension - I need this, as I need to persist > the last state for every key seen (user, transfer, etc.). As in Redis, > Kafka uses the UNIX copy-on-write semantics (process fork) to avoid > needless memory usage for log fragments (segments, in Kafka > nomenclature) that have not changed. Can I mimick a similar behaviour > in Erlang? Or if not, how can I deal with biggish (say, a couple of > GB) logs that needs to be compacted? > > In other words, I would like to create something like a *Minimum > Viable Log* (in Kafka style), only in Erlang/Elixir. I would be > grateful for any kind of design/implementation hints. > > [1] http://kafka.apache.org/ > [2] https://zookeeper.apache.org/ > [3] http://kafka.apache.org/documentation.html#persistence > [4] https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Sat Jun 18 21:41:42 2016 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sat, 18 Jun 2016 12:41:42 -0700 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: > On Sat, Jun 18, 2016 at 3:54 AM, John Smith < 4crzen62cwqszy68g7al@REDACTED> wrote: > For one of my systems in the financial area, I am in need of a > disk-backed log that I could use as a backend for an Event > Sourcing/CQRS store. Recently, I have read a bit about Kafka [1] and > it seems like a good fit but, unfortunately, it is on JVM (written > in Scala, to be exact) and depends heavily on ZooKeeper [2] for > distribution, while I would prefer something similar for an Erlang > ecosystem. [...] Howdy. I'd started working on a project quite similar to what you described, when I was at Basho Japan. That project has been cancelled and the Basho Japan development office closed, but the source still exists in a GitHub repo at https://github.com/basho/machi with an APLv2 license. The current state of the 'master' branch is messy, since the cancellation came during three different refactorings that never were fully finished & merged. But IIRC it works and is likely correct, most of the time. ^_^ The master branch does have a dependency on eleveldb which depends on the C++ code in the LevelDB fork that Basho uses for eleveldb. If that's a dealbreaker, just go backward in the repo's history a few months more to find the eleveldb-free version. The low-level API is UNIX'ish & Kafka'ish. It does not have the higher-level abstraction of the key-value store that you want. But it's easy to build that on top of a strictly ordered log. > In other words, I would like to create something like a *Minimum > Viable Log* (in Kafka style), only in Erlang/Elixir. Heh, defining "MVL" is tricky. A non-distributed version is pretty easy. (As Kafka devs discovered early on.) A distributed version that doesn't reorder log entries and doesn't lose log entries is not easy. (As most distributed systems devs tends to discover. ^_^) If the Machi code doesn't suit you, I wouldn't be surprised. However, the design papers in the 'doc' directory (look for the PDF files) can give you a lot of background for why Machi was designed the way it is. If that rationale fits your "MVL", then I'm glad the docs & code are helpful. -Scott -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkbucc@REDACTED Sun Jun 19 07:34:19 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Sun, 19 Jun 2016 01:34:19 -0400 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: Can you shard your event log by aggregate type and thus avoid the deletion/compaction issue altogether? I've read some people suggest sharding by aggregate ID if the aggregate type shard is not small enough [1]. If you haven't already, you may want to write a simple benchmark that append gobs of data to a file. I found a 2013 thread [2] on Erlang Questions with a similar sequential append use case where the OP was not happy with Erlang's speed. But I can't follow his math: writing 5 Gigabits in 104 seconds seems like a lot more that 504Hz. I also found people complaining that get_line was slow. I guess parallel reads would be possible inside an aggregate boundary ... CQRS is a topic I am very interested in, I hope you post again! [1] http://cqrs.nu/Faq/event-sourcing [2] http://erlang.org/pipermail/erlang-questions/2013-June/074190.html P.S. I've wondered why people don't treat a snapshot (or "compaction") as a command. One that emits a "special" event with the current state of that aggregate. Write this event to a fast durable key/value store (again, one per aggregate type) where the key is aggregate id and the value is the aggregate state and an offset into the main log where you should pick up reading from. On Sat, Jun 18, 2016 at 6:54 AM, John Smith <4crzen62cwqszy68g7al@REDACTED> wrote: > For one of my systems in the financial area, I am in need of a disk-backed > log that I could use as a backend for an Event Sourcing/CQRS store. > Recently, I have read a bit about Kafka [1] and it seems like a good fit > but, unfortunately, it is on JVM (written in Scala, to be exact) and > depends heavily on ZooKeeper [2] for distribution, while I would prefer > something similar for an Erlang ecosystem. Thus, ideally, I would like to > have something that is: > > * small, > * durable (checksummed, with a clear recovery procedure), > * pure Erlang/Elixir (maybe with some native code, but tightly > integrated), > * (almost) not distributed - data fits on the single node (at least now; > with replication for durability, though). > > Before jumping right into implementation, I have some questions: > > 1. Is there anything already available that fulfils above requirements? > 2. Kafka uses different approach to persistence - instead of using > in-process buffers and transferring data to disk, it writes straight to the > filesystem which, actually, uses pagecache [3]. Can I achieve the same > thing using Erlang or does it buffers writes in some other way? > 3. ...also, Kafka has a log compaction [4] which can work not only in > time but also in a key dimension - I need this, as I need to persist the > last state for every key seen (user, transfer, etc.). As in Redis, Kafka > uses the UNIX copy-on-write semantics (process fork) to avoid needless > memory usage for log fragments (segments, in Kafka nomenclature) that have > not changed. Can I mimick a similar behaviour in Erlang? Or if not, how can > I deal with biggish (say, a couple of GB) logs that needs to be compacted? > > In other words, I would like to create something like a *Minimum Viable > Log* (in Kafka style), only in Erlang/Elixir. I would be grateful for any > kind of design/implementation hints. > > [1] http://kafka.apache.org/ > [2] https://zookeeper.apache.org/ > [3] http://kafka.apache.org/documentation.html#persistence > [4] https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 4crzen62cwqszy68g7al@REDACTED Sun Jun 19 23:43:38 2016 From: 4crzen62cwqszy68g7al@REDACTED (John Smith) Date: Sun, 19 Jun 2016 23:43:38 +0200 Subject: [erlang-questions] Disk-backed log In-Reply-To: References: Message-ID: On Sun, 19 Jun 2016 07:34:19 +0200, Mark Bucciarelli wrote: > Can you shard your event log by aggregate type and thus avoid the > deletion/compaction issue altogether? I've read some people suggest > sharding by >aggregate ID if the aggregate type shard is not small > enough [1]. Probably not, or at least, I can't think of such sharding scheme now. Also, apart from that, log compaction is still useful in the case of failure (it is faster to start from an aggregated snapshot than applying all previous events). > > If you haven't already, you may want to write a simple benchmark that > append gobs of data to a file. I found a 2013 thread [2] on Erlang > Questions with a >similar sequential append use case where the OP was > not happy with Erlang's speed. But I can't follow his math: writing 5 > Gigabits in 104 seconds seems >like a lot more that 504Hz. I also found > people complaining that get_line was slow. I guess parallel reads would > be possible inside an aggregate >boundary ... I do not have any numbers as of now; I am still wandering around this whole "universal log" concept, but I find it appealing. I think I will start from a single, simple append-only file with data in the Kafka format and then I could provide some info. > > CQRS is a topic I am very interested in, I hope you post again! > > [1] http://cqrs.nu/Faq/event-sourcing > [2] http://erlang.org/pipermail/erlang-questions/2013-June/074190.html > > P.S. I've wondered why people don't treat a snapshot (or "compaction") > as a command. One that emits a "special" event with the current state of > that >aggregate. Write this event to a fast durable key/value store > (again, one per aggregate type) where the key is aggregate id and the > value is the aggregate >state and an offset into the main log where you > should pick up reading from. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Jun 20 10:29:14 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 20 Jun 2016 08:29:14 +0000 Subject: [erlang-questions] speed of neotoma parser In-Reply-To: References: <55A810C00B7CDA48A5AB43C566064B41014C52EAD0@SE-EX019.groupinfra.com> Message-ID: On Mon, May 30, 2016 at 1:54 PM Max Lapshin wrote: > This story ended with writing handmade parser. > > Just to say I would love to see such tool open :) Format seems similar to hcl [1] which would be useful :) - benoit [1] https://github.com/hashicorp/hcl -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb_duffy@REDACTED Mon Jun 20 21:04:46 2016 From: jb_duffy@REDACTED (John Duffy) Date: Mon, 20 Jun 2016 20:04:46 +0100 (BST) Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF Message-ID: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> Hi I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. Any help would be mud appreciated. Kind regards John #include "erl_nif.h" static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) { ErlNifBinary Bin; if (!enif_inspect_binary(env, argv[0], &Bin)) { return enif_make_badarg(env); } // So far, so good... // From the erl shell... // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 // // How do I access Bin.data as an array of doubles? // return enif_make_int(env, Bin.size); // For testing... this works. } static ErlNifFunc nif_funcs[] = { {"my_nif_1", 1, my_nif_1} }; ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Mon Jun 20 21:46:45 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 20 Jun 2016 15:46:45 -0400 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> Message-ID: On Mon, Jun 20, 2016 at 3:04 PM, John Duffy wrote: > Hi > > I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with > passing a binary of floats to a NIF for further processing as as array of C > doubles. > > My simple NIF module compiles and I can access my "my_nif_1" function from > the erlang shell. I can access the 'size' structure member but I can't > figure out how to access the 'data' structure member as a C array. I > understand that 'data' is a block of memory of UINT bytes but I can't > figure out how to "cast" this to a C array within the NIF. > This doesn't answer your casting question, but is there a reason you're using a binary and not a resource? See the "Resource Objects" section under http://erlang.org/doc/man/erl_nif.html . --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Mon Jun 20 21:50:58 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 20 Jun 2016 21:50:58 +0200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> Message-ID: <57684922.4060109@ericsson.com> If you use <<1.0/float-native>> then it's just type cast to a 'double' pointer: double* my_array; my_array = (double*) Bin.data; printf("%lf", my_array[0]); Default byte order is 'float-big', which will not work on little endian machines (x86). Depending on architecture you also might have to ensure 8 byte alignment. x86 will accept unaligned floats but the performance may suffer. /Sverker, Erlang/OTP On 06/20/2016 09:04 PM, John Duffy wrote: > Hi > I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. > My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. > Any help would be mud appreciated. > Kind regards > John > #include "erl_nif.h" > static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) > { > ErlNifBinary Bin; > if (!enif_inspect_binary(env, argv[0], &Bin)) { > return enif_make_badarg(env); > } > // So far, so good... > // From the erl shell... > // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 > // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 > // > // How do I access Bin.data as an array of doubles? > // > return enif_make_int(env, Bin.size); // For testing... this works. > } > static ErlNifFunc nif_funcs[] = { > {"my_nif_1", 1, my_nif_1} > }; > ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Mon Jun 20 21:59:28 2016 From: tony@REDACTED (Tony Rogvall) Date: Mon, 20 Jun 2016 21:59:28 +0200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <57684922.4060109@ericsson.com> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> <57684922.4060109@ericsson.com> Message-ID: <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> > On 20 jun 2016, at 21:50, Sverker Eriksson wrote: > > > If you use <<1.0/float-native>> > then it's just type cast to a 'double' pointer: > > double* my_array; > > my_array = (double*) Bin.data; > > printf("%lf", my_array[0]); > > > Default byte order is 'float-big', which will not work on little endian machines (x86). > > Depending on architecture you also might have to ensure 8 byte alignment. > x86 will accept unaligned floats but the performance may suffer. > Is there an easy way to ensure 8 byte alignment in Erlang, when constructing binaries? On a 64-bit machine/arch my guess is that Binary data is 8 byte aligned but what about a on 32-bit machine/arch? /Tony > /Sverker, Erlang/OTP > > > On 06/20/2016 09:04 PM, John Duffy wrote: >> Hi >> I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. >> My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. >> Any help would be mud appreciated. >> Kind regards >> John >> #include "erl_nif.h" >> static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) >> { >> ErlNifBinary Bin; >> if (!enif_inspect_binary(env, argv[0], &Bin)) { >> return enif_make_badarg(env); >> } >> // So far, so good... >> // From the erl shell... >> // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 >> // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 >> // >> // How do I access Bin.data as an array of doubles? >> // >> return enif_make_int(env, Bin.size); // For testing... this works. >> } >> static ErlNifFunc nif_funcs[] = { >> {"my_nif_1", 1, my_nif_1} >> }; >> ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tony@REDACTED Mon Jun 20 22:07:22 2016 From: tony@REDACTED (Tony Rogvall) Date: Mon, 20 Jun 2016 22:07:22 +0200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> <57684922.4060109@ericsson.com> <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> Message-ID: <363ABDA1-5323-4062-8B37-7588499C01EE@rogvall.se> I just found the answer in binary.c. The answer is that all binary data is always 8 byte aligned. And that is nice :-) /Tony > On 20 jun 2016, at 21:59, Tony Rogvall wrote: > >> >> On 20 jun 2016, at 21:50, Sverker Eriksson wrote: >> >> >> If you use <<1.0/float-native>> >> then it's just type cast to a 'double' pointer: >> >> double* my_array; >> >> my_array = (double*) Bin.data; >> >> printf("%lf", my_array[0]); >> >> >> Default byte order is 'float-big', which will not work on little endian machines (x86). >> >> Depending on architecture you also might have to ensure 8 byte alignment. >> x86 will accept unaligned floats but the performance may suffer. >> > > Is there an easy way to ensure 8 byte alignment in Erlang, when constructing binaries? > On a 64-bit machine/arch my guess is that Binary data is 8 byte aligned but > what about a on 32-bit machine/arch? > > /Tony > >> /Sverker, Erlang/OTP >> >> >> On 06/20/2016 09:04 PM, John Duffy wrote: >>> Hi >>> I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. >>> My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. >>> Any help would be mud appreciated. >>> Kind regards >>> John >>> #include "erl_nif.h" >>> static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> ErlNifBinary Bin; >>> if (!enif_inspect_binary(env, argv[0], &Bin)) { >>> return enif_make_badarg(env); >>> } >>> // So far, so good... >>> // From the erl shell... >>> // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 >>> // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 >>> // >>> // How do I access Bin.data as an array of doubles? >>> // >>> return enif_make_int(env, Bin.size); // For testing... this works. >>> } >>> static ErlNifFunc nif_funcs[] = { >>> {"my_nif_1", 1, my_nif_1} >>> }; >>> ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From sverker.eriksson@REDACTED Mon Jun 20 22:24:15 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 20 Jun 2016 22:24:15 +0200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <363ABDA1-5323-4062-8B37-7588499C01EE@rogvall.se> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> <57684922.4060109@ericsson.com> <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> <363ABDA1-5323-4062-8B37-7588499C01EE@rogvall.se> Message-ID: <576850EF.2090600@ericsson.com> Well, reference counted binaries have 8-byte aligned data. Heap binaries only half of the time on 32-bit :-) Both driver_alloc_binary() and enif_alloc_binary() guarantee 8-byte alignment of the returned payload data. /Sverker On 06/20/2016 10:07 PM, Tony Rogvall wrote: > I just found the answer in binary.c. > The answer is that all binary data is always 8 byte aligned. > And that is nice :-) > > /Tony > >> On 20 jun 2016, at 21:59, Tony Rogvall wrote: >> >>> On 20 jun 2016, at 21:50, Sverker Eriksson wrote: >>> >>> >>> If you use <<1.0/float-native>> >>> then it's just type cast to a 'double' pointer: >>> >>> double* my_array; >>> >>> my_array = (double*) Bin.data; >>> >>> printf("%lf", my_array[0]); >>> >>> >>> Default byte order is 'float-big', which will not work on little endian machines (x86). >>> >>> Depending on architecture you also might have to ensure 8 byte alignment. >>> x86 will accept unaligned floats but the performance may suffer. >>> >> Is there an easy way to ensure 8 byte alignment in Erlang, when constructing binaries? >> On a 64-bit machine/arch my guess is that Binary data is 8 byte aligned but >> what about a on 32-bit machine/arch? >> >> /Tony >> >>> /Sverker, Erlang/OTP >>> >>> >>> On 06/20/2016 09:04 PM, John Duffy wrote: >>>> Hi >>>> I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. >>>> My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. >>>> Any help would be mud appreciated. >>>> Kind regards >>>> John >>>> #include "erl_nif.h" >>>> static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) >>>> { >>>> ErlNifBinary Bin; >>>> if (!enif_inspect_binary(env, argv[0], &Bin)) { >>>> return enif_make_badarg(env); >>>> } >>>> // So far, so good... >>>> // From the erl shell... >>>> // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 >>>> // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 >>>> // >>>> // How do I access Bin.data as an array of doubles? >>>> // >>>> return enif_make_int(env, Bin.size); // For testing... this works. >>>> } >>>> static ErlNifFunc nif_funcs[] = { >>>> {"my_nif_1", 1, my_nif_1} >>>> }; >>>> ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Tue Jun 21 01:10:27 2016 From: tony@REDACTED (Tony Rogvall) Date: Tue, 21 Jun 2016 01:10:27 +0200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: <576850EF.2090600@ericsson.com> References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> <57684922.4060109@ericsson.com> <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> <363ABDA1-5323-4062-8B37-7588499C01EE@rogvall.se> <576850EF.2090600@ericsson.com> Message-ID: > On 20 jun 2016, at 22:24, Sverker Eriksson wrote: > > Well, reference counted binaries have 8-byte aligned data. > Heap binaries only half of the time on 32-bit :-) Did not think of that. I guess that it should be fixed ? I can see traces of an align flag... Maybe resurrect that flag? Or better to have something like unit syntax for alignment? <> But alignment just for powers of two. /Tony > > Both driver_alloc_binary() and enif_alloc_binary() guarantee > 8-byte alignment of the returned payload data. > > /Sverker > > > On 06/20/2016 10:07 PM, Tony Rogvall wrote: >> I just found the answer in binary.c. >> The answer is that all binary data is always 8 byte aligned. >> And that is nice :-) >> >> /Tony >> >>> On 20 jun 2016, at 21:59, Tony Rogvall wrote: >>> >>>> On 20 jun 2016, at 21:50, Sverker Eriksson wrote: >>>> >>>> >>>> If you use <<1.0/float-native>> >>>> then it's just type cast to a 'double' pointer: >>>> >>>> double* my_array; >>>> >>>> my_array = (double*) Bin.data; >>>> >>>> printf("%lf", my_array[0]); >>>> >>>> >>>> Default byte order is 'float-big', which will not work on little endian machines (x86). >>>> >>>> Depending on architecture you also might have to ensure 8 byte alignment. >>>> x86 will accept unaligned floats but the performance may suffer. >>>> >>> Is there an easy way to ensure 8 byte alignment in Erlang, when constructing binaries? >>> On a 64-bit machine/arch my guess is that Binary data is 8 byte aligned but >>> what about a on 32-bit machine/arch? >>> >>> /Tony >>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>>> On 06/20/2016 09:04 PM, John Duffy wrote: >>>>> Hi >>>>> I'm using FreeBSD 10.3 & Erlang 18.3 and I'm trying to get to grips with passing a binary of floats to a NIF for further processing as as array of C doubles. >>>>> My simple NIF module compiles and I can access my "my_nif_1" function from the erlang shell. I can access the 'size' structure member but I can't figure out how to access the 'data' structure member as a C array. I understand that 'data' is a block of memory of UINT bytes but I can't figure out how to "cast" this to a C array within the NIF. >>>>> Any help would be mud appreciated. >>>>> Kind regards >>>>> John >>>>> #include "erl_nif.h" >>>>> static ERL_NIF_TERM my_nif_1(ErlNifEnv * env, int argc, const ERL_NIF_TERM argv[]) >>>>> { >>>>> ErlNifBinary Bin; >>>>> if (!enif_inspect_binary(env, argv[0], &Bin)) { >>>>> return enif_make_badarg(env); >>>>> } >>>>> // So far, so good... >>>>> // From the erl shell... >>>>> // my_nifs:my_nif_1(<<1.0/float>>). returns a Bin.size of 8 >>>>> // my_nifs:my_nif_1(<<1.0/float,2.0/float>>. returns a Bin.size of 16 >>>>> // >>>>> // How do I access Bin.data as an array of doubles? >>>>> // >>>>> return enif_make_int(env, Bin.size); // For testing... this works. >>>>> } >>>>> static ErlNifFunc nif_funcs[] = { >>>>> {"my_nif_1", 1, my_nif_1} >>>>> }; >>>>> ERL_NIF_INIT(my_nifs, nif_funcs, NULL, NULL, NULL, NULL); >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ok@REDACTED Tue Jun 21 03:58:24 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 21 Jun 2016 13:58:24 +1200 Subject: [erlang-questions] Accessing A Binary as an Array of Floats/Doubles Within a NIF In-Reply-To: References: <16980831.78593.1466449486023.JavaMail.defaultUser@defaultHost> <57684922.4060109@ericsson.com> <288C09B0-FF41-4A9D-9F3C-B047FF2A188E@rogvall.se> <363ABDA1-5323-4062-8B37-7588499C01EE@rogvall.se> <576850EF.2090600@ericsson.com> Message-ID: The chances are that C code on a 32-bit machine will be happy with 32-bit alignment for doubles. (Unless your 32-bit C code is using x86 vector instructions, in which case 64-bit alignment would not be enough either.) From donpedrothird@REDACTED Tue Jun 21 10:11:35 2016 From: donpedrothird@REDACTED (John Doe) Date: Tue, 21 Jun 2016 11:11:35 +0300 Subject: [erlang-questions] mnesia ordered_set tables, select previous N records starting from given key Message-ID: Hi all, Is it possible to select previous N records from the ordered_set mnesia table starting from given key? Something like this mnesia:select(table, [{#table{key='$0', _='_'},[{'>', '$0', <<"key">>} ],['$_']}], 10, read) but with DESC order of sorting? Without using just prev and next functions, as guards can be more complex. Just changing '>' to '<' won't work as records will be selected from the beginning of the table instead of from the key. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Tue Jun 21 18:08:56 2016 From: olivier.boudeville@REDACTED (BOUDEVILLE Olivier) Date: Tue, 21 Jun 2016 16:08:56 +0000 Subject: [erlang-questions] net_kernel:start/1 and EPMD gotcha Message-ID: <270d9bab6926430c86d51291d4f7cc69@PCYINTPEXMU011.NEOPROD.EDF.FR> Hi, Shouldn't net_kernel:start/1 do the same as when a VM is run from the command-line with a short or long name, i.e. (among other things) also launch an EPMD instance if needed? For the context (maybe it will help people encountering the same issue): we had an application which was running with long names until we had to include an optional information (an instance identifier) in the node names, an information that gets in some cases only available at runtime. As renaming a node created with -sname or -name is apparently not possible (not_allowed), we resorted to changing the application so that its first node is launched initially in non-distributed mode and then, only once the identifier is available, turning this node in a distributed one, with net_kernel:start/1. Unfortunately past launches had spawned an EPMD that was lingering in the background and gave us the impression that everything was working properly. However, while we were working on another feature, an update and a reboot had to occur, and the application became then unable to launch at all, with a somewhat cryptic message akin to: {'EXIT',nodistribution}}},{child,undefined,net_sup_dynamic,{erl_distribution,start_link,[['my-foobar-node',longnames]]},permanent,1000,supervisor,[erl_distribution]}}... It has been a long way to find IT updates, IPv4/IPv6 settings, host resolution, firewalls, cookies and all not guilty, and that the application was working iff another Erlang program had triggered the launch of an EPMD that had not been wiped yet. At least, if EPMD was not launched transparently, as an improvement probably the error message could be clearer (like: "unable to connect to EPMD on port XXXX") and this sentence in http://erlang.org/doc/man/net_kernel.html#start-1 "Turns a non-distributed node into a distributed node by starting net_kernel and other necessary processes." could be rephrased in: "Turns a non-distributed node into a distributed node by starting net_kernel and other necessary (Erlang) processes, knowing that an EPMD instance is expected to be already running in the background." Maybe it would be more user-friendly and consistent to have EPMD be launched automatically with net_kernel:start/1, though. Thanks! Best, Olivier. Olivier Boudeville EDF R&D, D?partement SINETICS, Groupe ASICS, Bureau O2-E10 7, Boulevard Gaspard Monge, 91120 Palaiseau, France T?l. : +33 (0)1 78 19 43 63 T?l. mobile : +33 (0)6 16 83 37 22 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Tue Jun 21 21:11:54 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Tue, 21 Jun 2016 19:11:54 +0000 Subject: [erlang-questions] Inconsistent hostnames Message-ID: In deploying my erlang app to some new CentOS servers with erlang 18.3, I came across an oddity in networking. The server's networking config, which is admittedly messed up, looks something like: short hostname: bob long hostname: bob.prod.example.net domain: example.com Now when I start erlang with "erl -sname x", net_adm:localhost() returns " bob.prod.example.net". However, if I start erlang with "erl -name x", net_adm:localhost() returns " bob.example.com". I believe this comes from net_kernel:create_hostpart/2, where it follows different paths on short vs long node names. On a long node name, it appears to use short hostname + domain, where on a short node name, it simply uses the long hostname. That decision appears to trickle through the rest of the networking libraries. I understand that we need to fix the config on the server. I just wondered if this might be considered a bug in erlang as well. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Wed Jun 22 12:18:39 2016 From: kenneth@REDACTED (Kenneth Lundin) Date: Wed, 22 Jun 2016 12:18:39 +0200 Subject: [erlang-questions] Erlang/OTP 19.0 has been released Message-ID: Erlang/OTP 19.0 has been released -------------------------------------------------- Erlang/OTP 19.0 is a new major release with new features, quite a few (characteristics) improvements, as well as a few incompatibilities. Some highlights of the release are: - *compiler, stdlib*: - New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* - New preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or warning, respectively. - *gen_statem* a new state machine behavior - *mnesia_ext*: plugin of external storage solutions to mnesia - *crypto*: uses EVP interface in OpenSSL resulting in generally better performance and support for HW acceleration - *ssh*: performance improvements, uses the new gen_statem behavior - *ssl*: enhanced error log messages - *dialyzer*: the support for maps is very much extended both the type specification syntax and the type analysis. - *erts*: - erlang:open_port(spawn, ...) 3-5 times faster - tracing, extensive improvements (scalability, speed, match specifications on send/receive, support for lttng, ...) - dirty schedulers improved - per process configurable handling of message_queue - fast loading of multiple modules - max_heap_size process flag - *erts/kernel:* - Experimental support for Unix Domain Sockets Example: gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}]) - and much more .... You can find the Release Notes with more detailed info at http://www.erlang.org/download/otp_src_19.0.readme You find the source code at github.com in the official Erlang repository. Git tag OTP-19.0 https://github.com/erlang/otp/tree/OTP-19.0 You can also read the documentation on-line here: (see the Release Notes mentioned above for release notes which are not updated in the doc, but the new functionality is) http://www.erlang.org/doc We also want to thank all of you that have contributed with new features, bug-fixes, suggestions and bug reports. The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 22 17:25:07 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 22 Jun 2016 17:25:07 +0200 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: References: Message-ID: <576AADD3.8080506@ninenines.eu> That's a very nice set of features and changes! This release is impressive. Great job! On 06/22/2016 12:18 PM, Kenneth Lundin wrote: > Erlang/OTP 19.0 has been released > -------------------------------------------------- > > Erlang/OTP 19.0 is a new major release with new features, quite a few > (characteristics) improvements, as well as a few incompatibilities. > > Some highlights of the release are: > > * *compiler, stdlib*: > o New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* > o New preprocessor directives -error(Term) and -warning(Term) to > cause a compilation error or warning, respectively. > > * *gen_statem* a new state machine behavior > * *mnesia_ext*: plugin of external storage solutions to mnesia > * *crypto*: uses EVP interface in OpenSSL resulting in generally > better performance and support for HW acceleration > * *ssh*: performance improvements, uses the new gen_statem behavior > * *ssl*: enhanced error log messages > * *dialyzer*: the support for maps is very much extended both the type > specification syntax and the type analysis. > * *erts*: > o erlang:open_port(spawn, ...) 3-5 times faster > o tracing, extensive improvements (scalability, speed, match > specifications on send/receive, support for lttng, ...) > o dirty schedulers improved > o per process configurable handling of message_queue > o fast loading of multiple modules > o max_heap_size process flag > * *erts/kernel:* > o |Experimental support for Unix Domain Sockets > ||Example: ||gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}])||| > * and much more .... > > You can find the Release Notes with more detailed info at > > http://www.erlang.org/download/otp_src_19.0.readme > > You find the source code at github.com in the > official Erlang repository. > > Git tag OTP-19.0 > > https://github.com/erlang/otp/tree/OTP-19.0 > > You can also read the documentation on-line here: > (see the Release Notes mentioned above for release notes which > are not updated in the doc, but the new functionality is) > > http://www.erlang.org/doc > > We also want to thank all of you that have contributed with new > features, bug-fixes, suggestions and bug reports. > > The Erlang/OTP Team at Ericsson > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From mononcqc@REDACTED Wed Jun 22 17:58:55 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 22 Jun 2016 11:58:55 -0400 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <576AADD3.8080506@ninenines.eu> References: <576AADD3.8080506@ninenines.eu> Message-ID: <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> On 06/22, Lo?c Hoguin wrote: >That's a very nice set of features and changes! This release is >impressive. Great job! > Echoing Lo?c here. It feels good to see things moving forwards that way. Good job to the OTP team and all the contributors! From lheadley@REDACTED Wed Jun 22 18:58:16 2016 From: lheadley@REDACTED (Lyn Headley) Date: Wed, 22 Jun 2016 09:58:16 -0700 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> References: <576AADD3.8080506@ninenines.eu> <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> Message-ID: Let's say what we're most excited about. I'm excited about mnesia_ext because I think that removing the 2GB limit on disc_only_copies will open up whole new paradigms. I'm still working out the details of those paradigms though :-) On Wed, Jun 22, 2016 at 8:58 AM, Fred Hebert wrote: > On 06/22, Lo?c Hoguin wrote: >> >> That's a very nice set of features and changes! This release is >> impressive. Great job! >> > > Echoing Lo?c here. It feels good to see things moving forwards that way. > > Good job to the OTP team and all the contributors! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From heinz@REDACTED Wed Jun 22 19:28:30 2016 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Wed, 22 Jun 2016 19:28:30 +0200 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> References: <576AADD3.8080506@ninenines.eu> <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> Message-ID: <2EB428EE-EC0C-497D-B958-33F81B37E81F@licenser.net> +1 to that, really nice job with that very exciting thing :D > On Jun 22, 2016, at 17:58, Fred Hebert wrote: > > On 06/22, Lo?c Hoguin wrote: >> That's a very nice set of features and changes! This release is impressive. Great job! >> > > Echoing Lo?c here. It feels good to see things moving forwards that way. > > Good job to the OTP team and all the contributors! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From spanemangalore@REDACTED Wed Jun 22 21:22:22 2016 From: spanemangalore@REDACTED (Sachin Panemangalore) Date: Wed, 22 Jun 2016 12:22:22 -0700 Subject: [erlang-questions] [erlang-question][maps][comprehension] Message-ID: Hi A question on comprehension in erlang in general. [{X,Y} || X<-[1,2,3], Y<-[a,b,c]]. works but #{X=>Y || X<-[1,2,3], Y<-[a,b,c] } does not any specific reason why this was not implemented ( since Erlang is relative mature my guess is its a design choice by the creators of Erlang ). Regards sachin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Wed Jun 22 21:52:14 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Wed, 22 Jun 2016 15:52:14 -0400 Subject: [erlang-questions] [ANN] Chocolatey NuGet Package For Erlang Updated To R19 Message-ID: For the five Erlang developers who care :) https://chocolatey.org/packages/erlang/19.0 Just wanted to let people know. -- Onorio -------------- next part -------------- An HTML attachment was scrubbed... URL: From chassoul@REDACTED Thu Jun 23 02:25:31 2016 From: chassoul@REDACTED (Jean Chassoul) Date: Wed, 22 Jun 2016 20:25:31 -0400 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <2EB428EE-EC0C-497D-B958-33F81B37E81F@licenser.net> References: <576AADD3.8080506@ninenines.eu> <20160622155854.GH48593@fhebert-ltm2.internal.salesforce.com> <2EB428EE-EC0C-497D-B958-33F81B37E81F@licenser.net> Message-ID: This is awesome! congrats to all involved. (= On Wed, Jun 22, 2016 at 1:28 PM, Heinz Nikolaus Gies wrote: > +1 to that, really nice job with that very exciting thing :D > > > > On Jun 22, 2016, at 17:58, Fred Hebert wrote: > > > > On 06/22, Lo?c Hoguin wrote: > >> That's a very nice set of features and changes! This release is > impressive. Great job! > >> > > > > Echoing Lo?c here. It feels good to see things moving forwards that way. > > > > Good job to the OTP team and all the contributors! > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jun 23 02:59:26 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 23 Jun 2016 12:59:26 +1200 Subject: [erlang-questions] [erlang-question][maps][comprehension] In-Reply-To: References: Message-ID: On 23/06/16 7:22 AM, Sachin Panemangalore wrote: > Hi > A question on comprehension in erlang in general. > > [{X,Y} || X<-[1,2,3], Y<-[a,b,c]]. works > > but > > #{X=>Y || X<-[1,2,3], Y<-[a,b,c] } does not > > any specific reason why this was not implemented ( since Erlang is > relative mature my guess is its a design choice by the creators of > Erlang ). > What is it supposed to mean? Expanding the generators out, #{ 1 => a, 1 => b, 1 => c, 2 => a, 2 => b, 2 => c, 3 => a, 3 => b, 3 => c } is not really a meaningful map. From lloyd@REDACTED Thu Jun 23 04:16:43 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 22 Jun 2016 22:16:43 -0400 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: References: Message-ID: Let me be one of many to express deep appreciation for the hard work of the Erlang dev team! Mnesia_ext may definitely fit into my current work-in-progress. Can you point me to docs? Thanks again, LRP Sent from my iPad > On Jun 22, 2016, at 6:18 AM, Kenneth Lundin wrote: > > Erlang/OTP 19.0 has been released > -------------------------------------------------- > > Erlang/OTP 19.0 is a new major release with new features, quite a few (characteristics) improvements, as well as a few incompatibilities. > > Some highlights of the release are: > > compiler, stdlib: > New preprocessor macros ?FUNCTION_NAME, ?FUNCTION_ARITY > New preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or warning, respectively. > gen_statem a new state machine behavior > mnesia_ext: plugin of external storage solutions to mnesia > crypto: uses EVP interface in OpenSSL resulting in generally better performance and support for HW acceleration > ssh: performance improvements, uses the new gen_statem behavior > ssl: enhanced error log messages > dialyzer: the support for maps is very much extended both the type specification syntax and the type analysis. > erts: > erlang:open_port(spawn, ...) 3-5 times faster > tracing, extensive improvements (scalability, speed, match specifications on send/receive, support for lttng, ...) > dirty schedulers improved > per process configurable handling of message_queue > fast loading of multiple modules > max_heap_size process flag > erts/kernel: > Experimental support for Unix Domain Sockets > Example: gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}]) > and much more .... > You can find the Release Notes with more detailed info at > > http://www.erlang.org/download/otp_src_19.0.readme > > You find the source code at github.com in the official Erlang repository. > > Git tag OTP-19.0 > > https://github.com/erlang/otp/tree/OTP-19.0 > > You can also read the documentation on-line here: > (see the Release Notes mentioned above for release notes which > are not updated in the doc, but the new functionality is) > > http://www.erlang.org/doc > > We also want to thank all of you that have contributed with new features, bug-fixes, suggestions and bug reports. > > The Erlang/OTP Team at Ericsson > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Jun 23 05:34:30 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 22 Jun 2016 23:34:30 -0400 Subject: [erlang-questions] RocksDB Message-ID: <2771C38D-70D0-4388-A9DB-9F3E98A48EB1@writersglen.com> Hmmm... Wonder if this is embedded storage engine would be compatible with the new mnesia_ext feature: http://rocksdb.org/ Best wishes, LRP Sent from my iPad From ml-erlang@REDACTED Thu Jun 23 08:20:46 2016 From: ml-erlang@REDACTED (Matthias Rieber) Date: Thu, 23 Jun 2016 08:20:46 +0200 (CEST) Subject: [erlang-questions] mnesia_ext +leveldb Message-ID: Hi, On Tue, 23 Feb 2016, Richard Carlsson wrote: > Gah. Sorry, everybody, it's just been busy times here. Will try to start > pulling that string again. Thanks for the reminder. since Erlang 19 has been released with mnesia_ext I'd like to ask if there are any news on this topic. Kind Regards, Matthias > > /Richard > > 2016-02-23 13:38 GMT+01:00 Matthias Rieber : > > > Hello, > > > > is leveldb for mnesia_ext already open-sourced? I haven't found it on > > https://github.com/klarna > > > > Matthias > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > From dgud@REDACTED Thu Jun 23 08:40:51 2016 From: dgud@REDACTED (Dan Gudmundsson) Date: Thu, 23 Jun 2016 06:40:51 +0000 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: References: Message-ID: On Thu, Jun 23, 2016 at 4:16 AM Lloyd R. Prentice wrote: > Let me be one of many to express deep appreciation for the hard work of > the Erlang dev team! > > Mnesia_ext may definitely fit into my current work-in-progress. Can you > point me to docs? > > There are no docs and there where no docs in the patch sent. :-/ The api is still experimental, until you/we have something that works for more than one usage of the api. There is a test module that basically wraps ets for the few tests I have done. More PR's with tests and docs are appreciated. /Dan Thanks again, > > LRP > > Sent from my iPad > > On Jun 22, 2016, at 6:18 AM, Kenneth Lundin wrote: > > Erlang/OTP 19.0 has been released > -------------------------------------------------- > > Erlang/OTP 19.0 is a new major release with new features, quite a few > (characteristics) improvements, as well as a few incompatibilities. > > Some highlights of the release are: > > > - *compiler, stdlib*: > - New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* > - New preprocessor directives -error(Term) and -warning(Term) to > cause a compilation error or warning, respectively. > > > - *gen_statem* a new state machine behavior > - *mnesia_ext*: plugin of external storage solutions to mnesia > - *crypto*: uses EVP interface in OpenSSL resulting in generally > better performance and support for HW acceleration > - *ssh*: performance improvements, uses the new gen_statem behavior > - *ssl*: enhanced error log messages > - *dialyzer*: the support for maps is very much extended both the type > specification syntax and the type analysis. > - *erts*: > - erlang:open_port(spawn, ...) 3-5 times faster > - tracing, extensive improvements (scalability, speed, match > specifications on send/receive, support for lttng, ...) > - dirty schedulers improved > - per process configurable handling of message_queue > - fast loading of multiple modules > - max_heap_size process flag > - *erts/kernel:* > - Experimental support for Unix Domain Sockets > Example: gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}]) > - and much more .... > > You can find the Release Notes with more detailed info at > > http://www.erlang.org/download/otp_src_19.0.readme > > You find the source code at github.com in the > official Erlang repository. > > Git tag OTP-19.0 > > https://github.com/erlang/otp/tree/OTP-19.0 > > You can also read the documentation on-line here: > (see the Release Notes mentioned above for release notes which > are not updated in the doc, but the new functionality is) > > http://www.erlang.org/doc > > We also want to thank all of you that have contributed with new features, > bug-fixes, suggestions and bug reports. > > The Erlang/OTP Team at Ericsson > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli.iser@REDACTED Thu Jun 23 12:35:19 2016 From: eli.iser@REDACTED (Eli Iser) Date: Thu, 23 Jun 2016 13:35:19 +0300 Subject: [erlang-questions] Identifying causes for high CPU usage Message-ID: I'm running several ejabberd nodes in a cluster. Sadly, it uses a very old Erlang version - R13B04, on top of CentOS with 4 cores. I am using 5 async-threads with kernel poll enabled. The old Erlang version might perhaps explain the problem I saw, but I believe that the question is general enough. At one time the CPU usage of some of the nodes started climbing (from about 20% to near 100%). This happened only once and doesn't happen anymore (after a full cluster power off and power on again). Connecting to the nodes with a remote shell showed almost everything was the same to nodes that were un-affected: * cprof and eprof - showed the same usage patterns. * Listing top most erlang:process_info on all processes for reductions and message_queue_len showed similar patterns. * erlang:statistics - looked the same for wall_clock and reductions. The only concrete difference between the affected and un-affected nodes were: * run_queue - affected nodes had a run queue of several dozens (less than 100), while un-affected nodes had 0 (always). Since run_queues is undocumented (at least I didn't see it in the documentation), I didn't run it at the time of the problem. * runtime - affected nodes progress the runtime at about 150% of the wall time, while un-affected nodes progress the runtime at about 20% of the wall time. All of this made me suspect some NIFs and/or ports taking a long time to complete, taking CPU time but not increasing reductions and calls. Looking at the various profilers for Erlang, I couldn't find anything that can profile NIFs or even reveal that they indeed do take a long time to finish. Is there a better way to diagnose a high CPU usage issue? Cheers, Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jun 23 14:24:48 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 23 Jun 2016 14:24:48 +0200 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <576AADD3.8080506@ninenines.eu> References: <576AADD3.8080506@ninenines.eu> Message-ID: On Wed, Jun 22, 2016 at 5:25 PM, Lo?c Hoguin wrote: > That's a very nice set of features and changes! This release is > impressive. Great job! > I will also echo Lo?c here. One of the things that has always amazed me is that each Erlang release adds a number of useful features, which I did not think about before they were added, but turns out to be important for the correct operation of systems going forward. Doing this consistently over the last 10 years is called sound engineering. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Thu Jun 23 14:25:52 2016 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 23 Jun 2016 14:25:52 +0200 Subject: [erlang-questions] Identifying causes for high CPU usage In-Reply-To: References: Message-ID: Hi, Try lock counter during load-testing http://erlang.org/doc/apps/tools/lcnt_chapter.html It's very unsafe to run it in prod. Check locking time of the port drivers.Sometimes setting ERL_DRV_FLAG_USE_PORT_LOCKING flag can help. All long operations in ports cause the same scheduling problems you heard about NIFs. Unless these operations are async and driver is well-written. Scheduling/locking problems cause random latency spikes in all time metrics. So, check 95 percentile for this. Regards, Michael On 23 June 2016 at 12:35, Eli Iser wrote: > I'm running several ejabberd nodes in a cluster. Sadly, it uses a very old > Erlang version - R13B04, on top of CentOS with 4 cores. I am using 5 > async-threads with kernel poll enabled. > > The old Erlang version might perhaps explain the problem I saw, but I > believe that the question is general enough. > > At one time the CPU usage of some of the nodes started climbing (from > about 20% to near 100%). This happened only once and doesn't happen anymore > (after a full cluster power off and power on again). > > Connecting to the nodes with a remote shell showed almost everything was > the same to nodes that were un-affected: > > * cprof and eprof - showed the same usage patterns. > * Listing top most erlang:process_info on all processes for reductions and > message_queue_len showed similar patterns. > * erlang:statistics - looked the same for wall_clock and reductions. > > The only concrete difference between the affected and un-affected nodes > were: > > * run_queue - affected nodes had a run queue of several dozens (less than > 100), while un-affected nodes had 0 (always). Since run_queues is > undocumented (at least I didn't see it in the documentation), I didn't run > it at the time of the problem. > * runtime - affected nodes progress the runtime at about 150% of the wall > time, while un-affected nodes progress the runtime at about 20% of the wall > time. > > All of this made me suspect some NIFs and/or ports taking a long time to > complete, taking CPU time but not increasing reductions and calls. > > Looking at the various profilers for Erlang, I couldn't find anything that > can profile NIFs or even reveal that they indeed do take a long time to > finish. > > Is there a better way to diagnose a high CPU usage issue? > > Cheers, > Eli > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- ? ?????????, ?????? ??????. Best regards, Uvarov Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jun 23 14:33:02 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 23 Jun 2016 14:33:02 +0200 Subject: [erlang-questions] Identifying causes for high CPU usage In-Reply-To: References: Message-ID: On Thu, Jun 23, 2016 at 12:35 PM, Eli Iser wrote: > * run_queue - affected nodes had a run queue of several dozens (less than > 100), while un-affected nodes had 0 (always). Since run_queues is > undocumented (at least I didn't see it in the documentation), I didn't run > it at the time of the problem. A queues load can be seen as a (real) number K. If K < 1 it means your system can dequeue messages faster then they arrive. This leads to a queue size of 0 over time. If K = 1 it means that your system dequeues at the same rate as the arrival rate. This leads to a standing queue. If K > 1 the queue will slowly fill up because the arrival rate is larger than the processing/dequeue-rate. In your case, you are either in the K = 1 or the K > 1 situation for a while. This usually leads to more load on the system because there is more work to do. Note, however, that a 100% CPU load isn't necessarily a problem, unless response latencies are also affected. If you start a periodic background job which is CPU bound, this will take up all the free resources, but it will hopefully be scheduled out of the core whenever other work arrives to make way for faster processing. In other words, you may want to figure out what happens inside the processes with the larger message queues, and what events could lead to the longer message queues. A common case is that there is a specific user or subsystem which invokes the situation through normal use. But the use hits an edge-case. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From siddharth.karandikar@REDACTED Thu Jun 23 15:24:46 2016 From: siddharth.karandikar@REDACTED (Siddharth Karandikar) Date: Thu, 23 Jun 2016 18:54:46 +0530 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> <4FCF835A.4080004@eonblast.com> Message-ID: Hi Henning, After keeping notanswered.org running for 4 years, I think its time to take it down. There are 2 reasons behind this. Primary reason being its very low usage. Second - I am not able to take out some free cycles to maintain my hosted server and update the program with bug-fixes and/or new features. I think its time to retire this service. You can take back this domain. I will create a github project of the software and scripts running behind it, and if time permits, we can restart the service with some more interesting ideas if you have any. Thanks, Siddharth On Thu, Jun 7, 2012 at 4:37 PM, Siddharth Karandikar wrote: > This is cool! So kind of you to offer a domain name. > notanswered.org and www.notanswered.org now go to correct page. > > Thanks, > Siddharth > > On Wed, Jun 6, 2012 at 9:50 PM, Henning Diedrich wrote: >> On 6/6/12 5:40 PM, CGS wrote: >> >> more "energy drinks". >> >> I like it. I'll donate www.notanswered.org >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From pierrefenoll@REDACTED Thu Jun 23 15:41:37 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 23 Jun 2016 15:41:37 +0200 Subject: [erlang-questions] unanswered beginner questions in the list In-Reply-To: References: <4FC86EE6.3050509@gmail.com> <4FCF835A.4080004@eonblast.com> Message-ID: That's a neat idea! Please share your code. I'm sure we can find a way to keep this running with minimal supervision :) Cheers, -- Pierre Fenoll On 23 June 2016 at 15:24, Siddharth Karandikar < siddharth.karandikar@REDACTED> wrote: > Hi Henning, > > After keeping notanswered.org running for 4 years, I think its time to > take it down. There are 2 reasons behind this. Primary reason being > its very low usage. Second - I am not able to take out some free > cycles to maintain my hosted server and update the program with > bug-fixes and/or new features. > > I think its time to retire this service. You can take back this domain. > > I will create a github project of the software and scripts running > behind it, and if time permits, we can restart the service with some > more interesting ideas if you have any. > > > Thanks, > Siddharth > > > On Thu, Jun 7, 2012 at 4:37 PM, Siddharth Karandikar > wrote: > > This is cool! So kind of you to offer a domain name. > > notanswered.org and www.notanswered.org now go to correct page. > > > > Thanks, > > Siddharth > > > > On Wed, Jun 6, 2012 at 9:50 PM, Henning Diedrich > wrote: > >> On 6/6/12 5:40 PM, CGS wrote: > >> > >> more "energy drinks". > >> > >> I like it. I'll donate www.notanswered.org > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Jun 23 15:55:57 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 23 Jun 2016 09:55:57 -0400 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: References: Message-ID: <432135B0-F0E7-40E7-B974-FEC9E25AEE8A@writersglen.com> Thanks Dan. A challenge! Best, Lloyd Sent from my iPad > On Jun 23, 2016, at 2:40 AM, Dan Gudmundsson wrote: > >> On Thu, Jun 23, 2016 at 4:16 AM Lloyd R. Prentice wrote: >> Let me be one of many to express deep appreciation for the hard work of the Erlang dev team! >> >> Mnesia_ext may definitely fit into my current work-in-progress. Can you point me to docs? > > There are no docs and there where no docs in the patch sent. :-/ > > The api is still experimental, until you/we have something that works > for more than one usage of the api. > > There is a test module that basically wraps ets for the few tests I have done. > > More PR's with tests and docs are appreciated. > > /Dan > >> Thanks again, >> >> LRP >> >> Sent from my iPad >> >>> On Jun 22, 2016, at 6:18 AM, Kenneth Lundin wrote: >>> >>> Erlang/OTP 19.0 has been released >>> -------------------------------------------------- >>> >>> Erlang/OTP 19.0 is a new major release with new features, quite a few (characteristics) improvements, as well as a few incompatibilities. >>> >>> Some highlights of the release are: >>> >>> compiler, stdlib: >>> New preprocessor macros ?FUNCTION_NAME, ?FUNCTION_ARITY >>> New preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or warning, respectively. >>> gen_statem a new state machine behavior >>> mnesia_ext: plugin of external storage solutions to mnesia >>> crypto: uses EVP interface in OpenSSL resulting in generally better performance and support for HW acceleration >>> ssh: performance improvements, uses the new gen_statem behavior >>> ssl: enhanced error log messages >>> dialyzer: the support for maps is very much extended both the type specification syntax and the type analysis. >>> erts: >>> erlang:open_port(spawn, ...) 3-5 times faster >>> tracing, extensive improvements (scalability, speed, match specifications on send/receive, support for lttng, ...) >>> dirty schedulers improved >>> per process configurable handling of message_queue >>> fast loading of multiple modules >>> max_heap_size process flag >>> erts/kernel: >>> Experimental support for Unix Domain Sockets >>> Example: gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}]) >>> and much more .... >>> You can find the Release Notes with more detailed info at >>> >>> http://www.erlang.org/download/otp_src_19.0.readme >>> >>> You find the source code at github.com in the official Erlang repository. >>> >>> Git tag OTP-19.0 >>> >>> https://github.com/erlang/otp/tree/OTP-19.0 >>> >>> You can also read the documentation on-line here: >>> (see the Release Notes mentioned above for release notes which >>> are not updated in the doc, but the new functionality is) >>> >>> http://www.erlang.org/doc >>> >>> We also want to thank all of you that have contributed with new features, bug-fixes, suggestions and bug reports. >>> >>> The Erlang/OTP Team at Ericsson >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli.iser@REDACTED Thu Jun 23 14:36:32 2016 From: eli.iser@REDACTED (Eli Iser) Date: Thu, 23 Jun 2016 15:36:32 +0300 Subject: [erlang-questions] Identifying causes for high CPU usage In-Reply-To: References: Message-ID: The interesting thing is that I didn't see any processes that have long message queues. During regular operation of the nodes, I rarely see more than 5 messages in the queue of the most loaded processes. During the period of high CPU usage this didn't change. This made me think that this is not a problem with regular Erlang code load, but with something else. On 23 June 2016 at 15:33, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Thu, Jun 23, 2016 at 12:35 PM, Eli Iser wrote: > >> * run_queue - affected nodes had a run queue of several dozens (less than >> 100), while un-affected nodes had 0 (always). Since run_queues is >> undocumented (at least I didn't see it in the documentation), I didn't run >> it at the time of the problem. > > > A queues load can be seen as a (real) number K. > > If K < 1 it means your system can dequeue messages faster then they > arrive. This leads to a queue size of 0 over time. > > If K = 1 it means that your system dequeues at the same rate as the > arrival rate. This leads to a standing queue. > > If K > 1 the queue will slowly fill up because the arrival rate is larger > than the processing/dequeue-rate. > > In your case, you are either in the K = 1 or the K > 1 situation for a > while. This usually leads to more load on the system because there is more > work to do. Note, however, that a 100% CPU load isn't necessarily a > problem, unless response latencies are also affected. If you start a > periodic background job which is CPU bound, this will take up all the free > resources, but it will hopefully be scheduled out of the core whenever > other work arrives to make way for faster processing. > > In other words, you may want to figure out what happens inside the > processes with the larger message queues, and what events could lead to the > longer message queues. A common case is that there is a specific user or > subsystem which invokes the situation through normal use. But the use hits > an edge-case. > > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Thu Jun 23 17:41:53 2016 From: ddosia@REDACTED (Daniil Churikov) Date: Thu, 23 Jun 2016 16:41:53 +0100 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: Hi Andrew. If you have variety of distros used by developers it might make sense to write docker file, which will have instructions how to compile your project and produce release for a certain OS which is used in production. It can print result to stdout and you can redirect it, say, to release.tar file. Quite convenient and repeatable. 2016-05-18 1:42 GMT+01:00 Andrew Berman : > Hi List, > > So today I tried to create a release using rebar3 from my Mac and deploy > it to a CentOS box. Everything works when I do not include ERTS (and have > Erlang installed on the target), but if I do include ERTS, things fail > (can't run run_erl), I'm assuming because the Mac version of Erlang can't > be run on CentOS, which makes sense. So, I wanted to know how people do > deployments of their Erlang releases. Do you set up boxes with each OS > that matches your target OS and compile on there, including ERTS in the > release? Do you not include ERTS and make sure that all your Erlang > installs are the exact same version? Just trying to figure out a good > workflow. > > Thanks for any advice! > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Thu Jun 23 17:07:49 2016 From: emeka_1978@REDACTED (emeka_1978@REDACTED) Date: Thu, 23 Jun 2016 18:07:49 +0300 Subject: [erlang-questions] new stuff Message-ID: <00009fe3f8cd$f351a9aa$8ad720d7$@yahoo.com> Hi, I found some new stuff for you, i think you're gonna like it, more info at In haste, emeka_1978@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Fri Jun 24 00:09:28 2016 From: eric@REDACTED (Eric Entin) Date: Thu, 23 Jun 2016 18:09:28 -0400 Subject: [erlang-questions] Fwd: Callbacks no longer in Module:module_info(attributes) on OTP 19 In-Reply-To: References: Message-ID: Apologies if anyone receives this twice, but I think I was not able to post to the group before because I had not joined it. On OTP 19: Erlang/OTP 19 [erts-8.0] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > Eshell V8.0 (abort with ^G) > 1> gen_server:module_info(attributes). > [{vsn,[312724683823627977474402165944618744937]}] but on OTP 18: Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > Eshell V7.3 (abort with ^G) > 1> gen_server:module_info(attributes). > [{vsn,[103744906244088013982217220225633001614]}, > {callback,[{{init,1}, > [{type,"t",'fun', > [{type,"t",product, > > [{ann_type,"t",[{var,"t",'Args'},{type,"t",term,[]}]}]}, > {type,"u",union, > > [{type,"u",tuple,[{atom,"u",ok},{ann_type,"u",[{...}|...]}]}, > {type,"u",tuple, > > [{atom,"u",ok},{ann_type,"u",[...]},{type,[...],...}]}, > > {type,"v",tuple,[{atom,"v",stop},{ann_type,[...],...}]}, > {atom,"v",ignore}]}]}]}]}, ...] The data is still present in the beam file on OTP 19: Erlang/OTP 19 [erts-8.0] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > Eshell V8.0 (abort with ^G) > 1> beam_lib:chunks(code:which(gen_server), [abstract_code]). > {ok,{gen_server,[{abstract_code,{raw_abstract_v1,[{attribute,1, > file, > > {"gen_server.erl",1}}, > > {attribute,20,module,gen_server}, > {attribute,90,export, > [{start,3}, > {start,4}, > > {start_link,3}, > > {start_link,4}, > {stop,1}, > {stop,3}, > {call,2}, > {call,3}, > {cast,2}, > {reply,2}, > {abcast,2}, > {abcast,...}, > {...}|...]}, > {attribute,100,export, > > [{system_continue,3}, > > {system_terminate,4}, > > {system_code_change,4}, > > {system_get_state,1}, > > {system_replace_state,2}, > > {format_status,2}]}, > > {attribute,108,export,[{init_it,6}]}, > > {attribute,110,import,{error_logger,[{format,2}]}}, > {attribute,116,callback, > > {{init,1},[{type,116,'fun',[{type,...},{...}]}]}}, > {attribute,119,callback, > > {{handle_call,3},[{type,119,'fun',[{...}|...]}]}}, > {attribute,127,callback, > > {{handle_cast,2},[{type,127,'fun',[...]}]}}, > {attribute,131,callback, > > {{handle_info,2},[{type,131,'fun',...}]}}, > > {attribute,135,callback,{{terminate,2},[{type,135,...}]}}, > > {attribute,139,callback,{{code_change,3},[{type,...}]}}, > > {attribute,142,callback,{{format_status,2},[{...}]}}, > > {attribute,149,optional_callbacks,[{format_status,...}]}, > > {function,168,start,3,[...]}, > > {function,171,start,4,...}, > > {function,174,start_link,...}, > {function,177,...}, > {function,...}, > {...}|...]}}]}} but I can't find any details on this change in the OTP 19 changelog, in a PR on GitHub, or in the issues tracker. Can someone point me to why this behavior changed? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jun 24 14:14:14 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 24 Jun 2016 14:14:14 +0200 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: References: Message-ID: <576D2416.7040502@ninenines.eu> Noticed one small oddity: OTP-12837 Application(s): kernel Add {line_delim, byte()} option to inet:setopts/2 and decode_packet/3 In the documentation it appears to be {line_delimiter, byte()}. In the documentation it is listed in the text below the function, but not in the types (true for both decode_packet and inet:setopts/gen_tcp:option). Finally the choice of a single byte is curious. For example in HTTP the line delimiter is \r\n. On 06/22/2016 12:18 PM, Kenneth Lundin wrote: > Erlang/OTP 19.0 has been released > -------------------------------------------------- > > Erlang/OTP 19.0 is a new major release with new features, quite a few > (characteristics) improvements, as well as a few incompatibilities. > > Some highlights of the release are: > > * *compiler, stdlib*: > o New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* > o New preprocessor directives -error(Term) and -warning(Term) to > cause a compilation error or warning, respectively. > > * *gen_statem* a new state machine behavior > * *mnesia_ext*: plugin of external storage solutions to mnesia > * *crypto*: uses EVP interface in OpenSSL resulting in generally > better performance and support for HW acceleration > * *ssh*: performance improvements, uses the new gen_statem behavior > * *ssl*: enhanced error log messages > * *dialyzer*: the support for maps is very much extended both the type > specification syntax and the type analysis. > * *erts*: > o erlang:open_port(spawn, ...) 3-5 times faster > o tracing, extensive improvements (scalability, speed, match > specifications on send/receive, support for lttng, ...) > o dirty schedulers improved > o per process configurable handling of message_queue > o fast loading of multiple modules > o max_heap_size process flag > * *erts/kernel:* > o |Experimental support for Unix Domain Sockets > ||Example: ||gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}])||| > * and much more .... > > You can find the Release Notes with more detailed info at > > http://www.erlang.org/download/otp_src_19.0.readme > > You find the source code at github.com in the > official Erlang repository. > > Git tag OTP-19.0 > > https://github.com/erlang/otp/tree/OTP-19.0 > > You can also read the documentation on-line here: > (see the Release Notes mentioned above for release notes which > are not updated in the doc, but the new functionality is) > > http://www.erlang.org/doc > > We also want to thank all of you that have contributed with new > features, bug-fixes, suggestions and bug reports. > > The Erlang/OTP Team at Ericsson > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From serge@REDACTED Fri Jun 24 14:44:21 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 24 Jun 2016 08:44:21 -0400 Subject: [erlang-questions] Erlang/OTP 19.0 has been released In-Reply-To: <576D2416.7040502@ninenines.eu> References: <576D2416.7040502@ninenines.eu> Message-ID: The documentation is correct - the option was renamed from 'line_delim' to 'line_delimiter'. In terms of multi-character line delimiters - the default delimiter is '\n', but it will correctly handle lines separated by "\r\n" - that was the unmodified legacy behavior. So it'll correctly parse HTTP headers. What's changed was the ability to specify an alternative delimiting character, which is useful for some protocols (e.g. FIX that uses '\1'). The choice of a single character was for having minimal impact on the existing implementation. Note that there is a similar PR for file:read_line/1 ( https://github.com/erlang/otp/pull/757), but it's currently stalled as it presently only supports reading in raw mode. If there are any volunteers to help it get working in the non-raw mode, that would help to get it merged sooner. On Fri, Jun 24, 2016 at 8:14 AM, Lo?c Hoguin wrote: > Noticed one small oddity: > > OTP-12837 Application(s): kernel > > Add {line_delim, byte()} option to inet:setopts/2 and > decode_packet/3 > > In the documentation it appears to be {line_delimiter, byte()}. > > In the documentation it is listed in the text below the function, but not > in the types (true for both decode_packet and inet:setopts/gen_tcp:option). > > Finally the choice of a single byte is curious. For example in HTTP the > line delimiter is \r\n. > > On 06/22/2016 12:18 PM, Kenneth Lundin wrote: > >> Erlang/OTP 19.0 has been released >> -------------------------------------------------- >> >> Erlang/OTP 19.0 is a new major release with new features, quite a few >> (characteristics) improvements, as well as a few incompatibilities. >> >> Some highlights of the release are: >> >> * *compiler, stdlib*: >> o New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* >> o New preprocessor directives -error(Term) and -warning(Term) to >> cause a compilation error or warning, respectively. >> >> * *gen_statem* a new state machine behavior >> * *mnesia_ext*: plugin of external storage solutions to mnesia >> * *crypto*: uses EVP interface in OpenSSL resulting in generally >> better performance and support for HW acceleration >> * *ssh*: performance improvements, uses the new gen_statem behavior >> * *ssl*: enhanced error log messages >> * *dialyzer*: the support for maps is very much extended both the type >> specification syntax and the type analysis. >> * *erts*: >> o erlang:open_port(spawn, ...) 3-5 times faster >> o tracing, extensive improvements (scalability, speed, match >> specifications on send/receive, support for lttng, ...) >> o dirty schedulers improved >> o per process configurable handling of message_queue >> o fast loading of multiple modules >> o max_heap_size process flag >> * *erts/kernel:* >> o |Experimental support for Unix Domain Sockets >> ||Example: ||gen_udp:open(0, [{ifaddr,{local,"/tmp/socket"}}])||| >> * and much more .... >> >> You can find the Release Notes with more detailed info at >> >> http://www.erlang.org/download/otp_src_19.0.readme >> >> You find the source code at github.com in the >> official Erlang repository. >> >> Git tag OTP-19.0 >> >> https://github.com/erlang/otp/tree/OTP-19.0 >> >> You can also read the documentation on-line here: >> (see the Release Notes mentioned above for release notes which >> are not updated in the doc, but the new functionality is) >> >> http://www.erlang.org/doc >> >> We also want to thank all of you that have contributed with new >> features, bug-fixes, suggestions and bug reports. >> >> The Erlang/OTP Team at Ericsson >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben@REDACTED Fri Jun 24 21:07:56 2016 From: ben@REDACTED (Ben Adams) Date: Fri, 24 Jun 2016 15:07:56 -0400 Subject: [erlang-questions] Transfer files over WAN Message-ID: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> What is the best way (protocol) to transfer files concurrent over WAN?? And have SSL.? Still to do ftps?? Or is there an Erlang solution? Thanks -- Ben Adams -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 24 23:08:03 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 25 Jun 2016 00:08:03 +0300 Subject: [erlang-questions] Transfer files over WAN In-Reply-To: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> References: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> Message-ID: It is 2016, http is already ok for it without any serious issues about it. GET and PUT will be ok. On Fri, Jun 24, 2016 at 10:07 PM, Ben Adams wrote: > What is the best way (protocol) to transfer files concurrent over WAN? > And have SSL. Still to do ftps? Or is there an Erlang solution? > > Thanks > > -- Ben Adams > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben@REDACTED Fri Jun 24 23:20:08 2016 From: ben@REDACTED (Ben Adams) Date: Fri, 24 Jun 2016 17:20:08 -0400 Subject: [erlang-questions] Transfer files over WAN In-Reply-To: References: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> Message-ID: <5c776b9032cbdf6d266c7a5b6a3a878f@smtp.hushmail.com> Even for GB size files? On Jun 24, 2016, 5:08 PM, at 5:08 PM, Max Lapshin wrote: >It is 2016, http is already ok for it without any serious issues about >it. > >GET and PUT will be ok. > >On Fri, Jun 24, 2016 at 10:07 PM, Ben Adams >wrote: > >> What is the best way (protocol) to transfer files concurrent over >WAN? >> And have SSL. Still to do ftps? Or is there an Erlang solution? >> >> Thanks >> >> -- Ben Adams >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Fri Jun 24 23:46:40 2016 From: mfidelman@REDACTED (Miles Fidelman) Date: Fri, 24 Jun 2016 17:46:40 -0400 Subject: [erlang-questions] Transfer files over WAN In-Reply-To: <5c776b9032cbdf6d266c7a5b6a3a878f@smtp.hushmail.com> References: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> <5c776b9032cbdf6d266c7a5b6a3a878f@smtp.hushmail.com> Message-ID: Yup. Works fine for uploads to various sites, as well as for Amazon S3. On 6/24/16 5:20 PM, Ben Adams wrote: > > Even for GB size files? > > On Jun 24, 2016, at 5:08 PM, Max Lapshin > wrote: > > It is 2016, http is already ok for it without any serious issues > about it. > > GET and PUT will be ok. > > On Fri, Jun 24, 2016 at 10:07 PM, Ben Adams > > wrote: > > What is the best way (protocol) to transfer files concurrent > over WAN? And have SSL. Still to do ftps? Or is there an > Erlang solution? > > Thanks > > -- Ben Adams > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Jun 25 08:41:33 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 25 Jun 2016 09:41:33 +0300 Subject: [erlang-questions] Transfer files over WAN In-Reply-To: References: <6d1601adccb2875b25ede6d8f0e6cfa6@smtp.hushmail.com> <5c776b9032cbdf6d266c7a5b6a3a878f@smtp.hushmail.com> Message-ID: Even for TB file size. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sat Jun 25 11:44:36 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sat, 25 Jun 2016 05:44:36 -0400 Subject: [erlang-questions] OTP19 on Fedora Message-ID: <576E5284.3070706@aim.com> Any suggestions as to the easiest way of installing the new release on Fedora? Thanks. Don From lemenkov@REDACTED Sat Jun 25 12:04:11 2016 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sat, 25 Jun 2016 12:04:11 +0200 Subject: [erlang-questions] OTP19 on Fedora In-Reply-To: <576E5284.3070706@aim.com> References: <576E5284.3070706@aim.com> Message-ID: 2016-06-25 11:44 GMT+02:00 Donald Steven : > Any suggestions as to the easiest way of installing the new release on > Fedora? Thanks. We plan to upgrade Erlang to ver. 19 in Fedora 25 (not yet released). Until now you have either compile it by yourself or by installing it from somewhere else. -- With best regards, Peter Lemenkov. From t6sn7gt@REDACTED Sat Jun 25 12:49:39 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sat, 25 Jun 2016 06:49:39 -0400 Subject: [erlang-questions] OTP19 on Fedora In-Reply-To: References: <576E5284.3070706@aim.com> Message-ID: <576E61C3.2000008@aim.com> Thanks Peter, that's very helpful. Don On 06/25/2016 06:04 AM, Peter Lemenkov wrote: > 2016-06-25 11:44 GMT+02:00 Donald Steven : >> Any suggestions as to the easiest way of installing the new release on >> Fedora? Thanks. > We plan to upgrade Erlang to ver. 19 in Fedora 25 (not yet released). > Until now you have either compile it by yourself or by installing it > from somewhere else. > > From henrik.x.nord@REDACTED Mon Jun 27 12:49:01 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Mon, 27 Jun 2016 12:49:01 +0200 Subject: [erlang-questions] Patch package OTP 18.3.4.1 released Message-ID: <5771049D.3040606@ericsson.com> Patch Package: OTP 18.3.4.1 Git Tag: OTP-18.3.4.1 Date: 2016-06-27 Trouble Report Id: OTP-13674 Seq num: System: OTP Release: 18 Application: ssh-4.2.2.1 Predecessor: OTP 18.3.4 Check out the git tag OTP-18.3.4.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- ssh-4.2.2.1 ----------------------------------------------------- --------------------------------------------------------------------- The ssh-4.2.2.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13674 Application(s): ssh Related Id(s): TR-HU92273 SSH client does not any longer retry a bad password given as option to ssh:connect et al. Full runtime dependencies of ssh-4.2.2.1: crypto-3.3, erts-6.0, kernel-3.0, public_key-0.22, stdlib-2.3 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From bsvancara@REDACTED Mon Jun 27 15:04:13 2016 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Mon, 27 Jun 2016 15:04:13 +0200 Subject: [erlang-questions] os:cmd - changed output capture in Erlang 19? Message-ID: Hello! I noticed a difference in the output capturing mechanism in the Erlang 19 release. It seems this new version captures output to STDERR too. This is a testing perl code (in "perl_output_test.pl"): print STDERR "Printed to STDERR\n"; print STDOUT "Printed to STDOUT\n"; This is my Erlang code: os:cmd("perl_output_test.pl"). Running in Windows 10: Erlang 18: Eshell V7.3 (abort with ^G) 1> os:cmd("perl_output_test.pl"). Printed to STDERR "Printed to STDOUT\r\n" Erlang 19: Eshell V8.0 (abort with ^G) 1> os:cmd("perl_output_test.pl"). "Printed to STDERR\r\nPrinted to STDOUT\r\n" -- Best regards, Bohuslav Svancara -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 27 16:43:48 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 27 Jun 2016 17:43:48 +0300 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? Message-ID: I've compiled erlang 19.0 with extra microstate accounting. Right now on my test stand my software is consuming about 1,5 gbit of input traffic from peer server. I see that on all 4 schedulers busy_wait consumes about 26%, sleep is about 17% erlang is launched with: erl +K true +A 16 +zdbbl 20480 +a 32 +MBas aoffcaobf +MBacul 0 -name flussonic -boot start_sasl -s flussonic -sasl errlog_type error What does busy_wait means here? Is it a waisted CPU that I need to optimize to handle more input traffic? Or is it cpu that will be used as soon as my load grows? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Jun 27 17:05:11 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 27 Jun 2016 17:05:11 +0200 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: On Mon, Jun 27, 2016 at 4:43 PM, Max Lapshin wrote: > What does busy_wait means here? Is it a waisted CPU that I need to > optimize to handle more input traffic? Or is it cpu that will be used as > soon as my load grows? My guess is that it is busy-waiting on the CPU in order to be able to serve new events with lower latency. So once your system increases its traffic load, less time will be spent waiting. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon Jun 27 17:13:36 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 27 Jun 2016 17:13:36 +0200 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: On Mon, Jun 27, 2016 at 5:05 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Mon, Jun 27, 2016 at 4:43 PM, Max Lapshin > wrote: > >> What does busy_wait means here? Is it a waisted CPU that I need to >> optimize to handle more input traffic? Or is it cpu that will be used as >> soon as my load grows? > > > My guess is that it is busy-waiting on the CPU in order to be able to > serve new events with lower latency. So once your system increases its > traffic load, less time will be spent waiting. > > It is indeed. If you want to play with less (or more) spinning you can play with +sbwt, +swt and +swct. +sbwt controls most (but not all) of the spinning that is done by schedulers. Btw, are you seeing any performance degradation when running with msacc=extra? If so how much? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 27 18:58:58 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 27 Jun 2016 19:58:58 +0300 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: I don't see any visible performance degradation with msacc=extra I don't even see it with lcnt build. Btw, about sbwt and so all. We have to launch erlang in our flussonic with flags: +sbwt short +swt very_low +sfwi 20 +sub true +MBas aoffcaobf +MBacul 0 without +MBas aoffcaobf +MBacul 0 it is just leaking memory (maybe not leaking, but using it so much, that OOM comes earlier). flag +stbt s that we were using earlier is important to capture better multicast UDP, but make system absolutely unstable (it eats all memory and dies) when some other task is launched on a server, so we have refused from it. Flags +sbwt short +swt very_low +sfwi 20 +sub true are recommended by our customers. Without them since 18.3 we get messages from sys_monitor like long_schedule a and long_gc on a server with 50% usage of CPU. If this is considered a problem (because default settings don't allow to work at all =( ), then maybe I can help somehow? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jun 27 19:18:58 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 27 Jun 2016 19:18:58 +0200 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: <49165DB2-39E0-40E5-987B-C50F15FF7D2A@gmail.com> What happens if you use +S 3 ? Fewer schedulers can actually perform better sometimes. There will be less need for busy wait and sleep if there are fewer schedulers competing for jobs. Sergej > On 27 Jun 2016, at 18:58, Max Lapshin wrote: > > I don't see any visible performance degradation with msacc=extra > > I don't even see it with lcnt build. > > Btw, about sbwt and so all. We have to launch erlang in our flussonic with flags: +sbwt short +swt very_low +sfwi 20 +sub true +MBas aoffcaobf +MBacul 0 > > without +MBas aoffcaobf +MBacul 0 it is just leaking memory (maybe not leaking, but using it so much, that OOM comes earlier). > > flag +stbt s that we were using earlier is important to capture better multicast UDP, but make system absolutely unstable (it eats all memory and dies) when some other task is launched on a server, so we have refused from it. > > Flags +sbwt short +swt very_low +sfwi 20 +sub true are recommended by our customers. Without them since 18.3 we get messages from sys_monitor like long_schedule a and long_gc on a server with 50% usage of CPU. > > If this is considered a problem (because default settings don't allow to work at all =( ), then maybe I can help somehow? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon Jun 27 19:29:08 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 27 Jun 2016 20:29:08 +0300 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: <49165DB2-39E0-40E5-987B-C50F15FF7D2A@gmail.com> References: <49165DB2-39E0-40E5-987B-C50F15FF7D2A@gmail.com> Message-ID: Just removing schedulers online shows better stability of work for a while, but after some time long_schedule still comes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon Jun 27 19:49:16 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 27 Jun 2016 19:49:16 +0200 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: On Mon, Jun 27, 2016 at 6:58 PM, Max Lapshin wrote: > without +MBas aoffcaobf +MBacul 0 it is just leaking memory (maybe not > leaking, but using it so much, that OOM comes earlier). > Running both of those seems counter intuitive, basically aoffcaobf is needed in order to enable acul, but since you disable acul it makes little sense to have it. If the default (bf) is not good enough, aobf should be better than aoffcaobf. IIRC I talked to you in early 18.x about a bug in the acul pool which has since been fixed, have you checked if your issues are still present after that fix? If they are, it would be great if you could send me a recon_alloc snapshot of both a healthy system and one that is about to crash (or the crash dump if it has already crashed). > > flag +stbt s that we were using earlier is important to capture better > multicast UDP, but make system absolutely unstable (it eats all memory and > dies) when some other task is launched on a server, so we have refused from > it. > that is indeed the problem with +sbt, which is why don't recommend you run it unless you know that no cpu intensive tasks will compete on the same cores. > > Flags +sbwt short +swt very_low +sfwi 20 +sub true are recommended by > our customers. Without them since 18.3 we get messages from sys_monitor > like long_schedule a and long_gc on a server with 50% usage of CPU. > I don't really see how any of those flags could directly influence the amount of long_schedule or long_gc you get on your system. There could be some subtle effect in how things are scheduled and woken that changes. Do you run on real hw or in a virtual machine of some sort? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 27 20:12:43 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 27 Jun 2016 21:12:43 +0300 Subject: [erlang-questions] what means busy_wait in msacc erl 19.0? In-Reply-To: References: Message-ID: Only on real hardware. We don't even debug such issues on a virtual machines. It is not possible to take a crash dump, but it is possible to make recon_alloc. I will make it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.14159@REDACTED Tue Jun 28 01:23:53 2016 From: j.14159@REDACTED (Jeremy Pierre) Date: Mon, 27 Jun 2016 23:23:53 +0000 Subject: [erlang-questions] ML shaped language for the Erlang VM Message-ID: Hi all, I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. Blog post: http://noisycode.com/blog/2016/06/27/introducing-mlfe/ Repo: https://github.com/j14159/mlfe TLDR: - written in Erlang with leex, yecc, and cerl - all inferenced, typer works (albeit lots of testing still to do, no binaries nor maps) - PIDs are typed, as are message flows (but not if you're calling from Erlang) - Apache 2.0 Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Tue Jun 28 07:39:19 2016 From: raould@REDACTED (Raoul Duke) Date: Mon, 27 Jun 2016 22:39:19 -0700 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: > I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. super keen! (scary name, tho.) ATS supports erlang, somehow, apparently: https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl From alin.popa@REDACTED Tue Jun 28 07:44:31 2016 From: alin.popa@REDACTED (Alin Popa) Date: Tue, 28 Jun 2016 06:44:31 +0100 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: Well done; I can't wait to see how it goes. (indeed, something's weird about that name). Alin On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote: > > I've been hacking away at an ML-shaped language for the BEAM for the > past few months and just opened it up. I figure a few people on the list > might be interested in poking at it and may have thoughts on improvements. > > > super keen! > > (scary name, tho.) > > ATS supports erlang, somehow, apparently: > > https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From louis@REDACTED Tue Jun 28 09:28:19 2016 From: louis@REDACTED (Louis Pilfold) Date: Tue, 28 Jun 2016 08:28:19 +0100 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: Hello! How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? I also find the name quite unwieldy. Cheers, Louis On 28 Jun 2016 06:44, "Alin Popa" wrote: > Well done; I can't wait to see how it goes. > > (indeed, something's weird about that name). > > Alin > > On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote: > >> > I've been hacking away at an ML-shaped language for the BEAM for the >> past few months and just opened it up. I figure a few people on the list >> might be interested in poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Jun 28 10:06:49 2016 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 28 Jun 2016 10:06:49 +0200 Subject: [erlang-questions] os:cmd - changed output capture in Erlang 19? In-Reply-To: References: Message-ID: Hello, On Mon, Jun 27, 2016 at 3:04 PM, Bohuslav Svancara wrote: > > I noticed a difference in the output capturing mechanism in the Erlang 19 > release. > > It seems this new version captures output to STDERR too. > > Windows and Unix platforms have behaved differently in this regard up until 19. On Unix stderr has been redirected to stdout, but on Windows it has not been. In Erlang/OTP 19.0 I changed it so that both Unix and Windows behave the same way, i.e. they both redirect stderr to stdout for os:cmd. It would seem that I have forgotten to mention this incompatibility in the release notes. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshmbarney@REDACTED Tue Jun 28 13:54:57 2016 From: joshmbarney@REDACTED (Joshua Barney) Date: Tue, 28 Jun 2016 07:54:57 -0400 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> This seems like a very fun project. Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) Sent from my iPhone > On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: > > Hello! > > How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > > Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > > I also find the name quite unwieldy. > > Cheers, > Louis > >> On 28 Jun 2016 06:44, "Alin Popa" wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >>> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote: >>> > I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >>> >>> >>> super keen! >>> >>> (scary name, tho.) >>> >>> ATS supports erlang, somehow, apparently: >>> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Tue Jun 28 14:42:06 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Tue, 28 Jun 2016 14:42:06 +0200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> References: , <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: Great to see such a worthy project. I also wholeheartedly support "BeaML" as a title. Cheers, Oliver Gesendet:?Dienstag, 28. Juni 2016 um 13:54 Uhr Von:?"Joshua Barney" An:?"Louis Pilfold" Cc:?erlang-questions Betreff:?Re: [erlang-questions] ML shaped language for the Erlang VM This seems like a very fun project. Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) ? Sent from my iPhone On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: ? Hello! How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? I also find the name quite unwieldy. Cheers, Louis On 28 Jun 2016 06:44, "Alin Popa" wrote: Well done; I can't wait to see how it goes. ? (indeed, something's weird about that name). ? Alin ? On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up.? I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. super keen! (scary name, tho.) ATS supports erlang, somehow, apparently: https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED[erlang-questions@REDACTED] http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED[erlang-questions@REDACTED] http://erlang.org/mailman/listinfo/erlang-questions ? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED[erlang-questions@REDACTED] http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From louis@REDACTED Tue Jun 28 18:10:35 2016 From: louis@REDACTED (Louis Pilfold) Date: Tue, 28 Jun 2016 17:10:35 +0100 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: Hey How would pronounce BeaML? It seems clumsy to say, and would be easy to confuse with BEAM if said too quickly. Cheers, Louis On 28 June 2016 at 13:42, Oliver Korpilla wrote: > Great to see such a worthy project. > > I also wholeheartedly support "BeaML" as a title. > > Cheers, > Oliver > > Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr > Von: "Joshua Barney" > An: "Louis Pilfold" > Cc: erlang-questions > Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM > > This seems like a very fun project. > Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > > Sent from my iPhone > On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: > > Hello! > How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > I also find the name quite unwieldy. > Cheers, > Louis > > On 28 Jun 2016 06:44, "Alin Popa" wrote: > Well done; I can't wait to see how it goes. > > (indeed, something's weird about that name). > > Alin > > On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > > > super keen! > > (scary name, tho.) > > ATS supports erlang, somehow, apparently: > https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From oliver.korpilla@REDACTED Tue Jun 28 18:43:02 2016 From: oliver.korpilla@REDACTED (Oliver Korpilla) Date: Tue, 28 Jun 2016 18:43:02 +0200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: Beam-el. On June 28, 2016 6:10:35 PM CEST, Louis Pilfold wrote: >Hey > >How would pronounce BeaML? It seems clumsy to say, and would be easy >to confuse with BEAM if said too quickly. > >Cheers, >Louis > >On 28 June 2016 at 13:42, Oliver Korpilla >wrote: >> Great to see such a worthy project. >> >> I also wholeheartedly support "BeaML" as a title. >> >> Cheers, >> Oliver >> >> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >> Von: "Joshua Barney" >> An: "Louis Pilfold" >> Cc: erlang-questions >> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >> >> This seems like a very fun project. >> Might I suggest the name BeaML? (Current I can't stop seeing "milf" >:-/) >> >> Sent from my iPhone >> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >> >> Hello! >> How exciting, I've been toying with creation of something like this >myself, but my compiler knowledge isn't quite there yet. Looking >forward to learning from your work! >> Are there ways in which you would like people to contribute or get >involved? Or anything you would like feedback on? >> I also find the name quite unwieldy. >> Cheers, >> Louis >> >> On 28 Jun 2016 06:44, "Alin Popa" > wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke > wrote:> I've been hacking away at >an ML-shaped language for the BEAM for the past few months and just >opened it up. I figure a few people on the list might be interested in >poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> >https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> >http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ >erlang-questions mailing list erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshmbarney@REDACTED Tue Jun 28 19:00:34 2016 From: joshmbarney@REDACTED (Joshua Barney) Date: Tue, 28 Jun 2016 13:00:34 -0400 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: <7CB06DBD-C43A-466B-BB6E-92E1B807AE41@icloud.com> Yeah I can see that being a concern, but I think "bee-mul" or "beem-el" with two syllables is clearly different from beam. Do you have a preferred pronunciation of MLFE? Sent from my iPhone > On Jun 28, 2016, at 12:10 PM, Louis Pilfold wrote: > > Hey > > How would pronounce BeaML? It seems clumsy to say, and would be easy > to confuse with BEAM if said too quickly. > > Cheers, > Louis > >> On 28 June 2016 at 13:42, Oliver Korpilla wrote: >> Great to see such a worthy project. >> >> I also wholeheartedly support "BeaML" as a title. >> >> Cheers, >> Oliver >> >> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >> Von: "Joshua Barney" >> An: "Louis Pilfold" >> Cc: erlang-questions >> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >> >> This seems like a very fun project. >> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >> >> Sent from my iPhone >> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >> >> Hello! >> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >> I also find the name quite unwieldy. >> Cheers, >> Louis >> >> On 28 Jun 2016 06:44, "Alin Popa" wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From rickard.m.andersson@REDACTED Tue Jun 28 18:22:03 2016 From: rickard.m.andersson@REDACTED (Rickard Andersson) Date: Tue, 28 Jun 2016 19:22:03 +0300 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: Simply like "BML", no? Something like "BeeML", only spelled "BeaML". On 28-Jun-16 19:10, Louis Pilfold wrote: > Hey > > How would pronounce BeaML? It seems clumsy to say, and would be easy > to confuse with BEAM if said too quickly. > > Cheers, > Louis > > On 28 June 2016 at 13:42, Oliver Korpilla wrote: >> Great to see such a worthy project. >> >> I also wholeheartedly support "BeaML" as a title. >> >> Cheers, >> Oliver >> >> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >> Von: "Joshua Barney" >> An: "Louis Pilfold" >> Cc: erlang-questions >> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >> >> This seems like a very fun project. >> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >> >> Sent from my iPhone >> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >> >> Hello! >> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >> I also find the name quite unwieldy. >> Cheers, >> Louis >> >> On 28 Jun 2016 06:44, "Alin Popa" wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From felixgallo@REDACTED Tue Jun 28 19:25:32 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 28 Jun 2016 10:25:32 -0700 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: MLFE is clearly a tribute to Dr. Melfi, Tony Soprano's therapist ( http://sopranos.wikia.com/wiki/Jennifer_Melfi), and would have to be pronounced that way... F. On Tue, Jun 28, 2016 at 9:22 AM, Rickard Andersson < rickard.m.andersson@REDACTED> wrote: > Simply like "BML", no? Something like "BeeML", only spelled "BeaML". > > > > On 28-Jun-16 19:10, Louis Pilfold wrote: > >> Hey >> >> How would pronounce BeaML? It seems clumsy to say, and would be easy >> to confuse with BEAM if said too quickly. >> >> Cheers, >> Louis >> >> On 28 June 2016 at 13:42, Oliver Korpilla wrote: >> >>> Great to see such a worthy project. >>> >>> I also wholeheartedly support "BeaML" as a title. >>> >>> Cheers, >>> Oliver >>> >>> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >>> Von: "Joshua Barney" >>> An: "Louis Pilfold" >>> Cc: erlang-questions >>> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >>> >>> This seems like a very fun project. >>> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >>> >>> Sent from my iPhone >>> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >>> >>> Hello! >>> How exciting, I've been toying with creation of something like this >>> myself, but my compiler knowledge isn't quite there yet. Looking forward to >>> learning from your work! >>> Are there ways in which you would like people to contribute or get >>> involved? Or anything you would like feedback on? >>> I also find the name quite unwieldy. >>> Cheers, >>> Louis >>> >>> On 28 Jun 2016 06:44, "Alin Popa" >> alin.popa@REDACTED]> wrote: >>> Well done; I can't wait to see how it goes. >>> >>> (indeed, something's weird about that name). >>> >>> Alin >>> >>> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke >> raould@REDACTED]> wrote:> I've been hacking away at an ML-shaped >>> language for the BEAM for the past few months and just opened it up. I >>> figure a few people on the list might be interested in poking at it and may >>> have thoughts on improvements. >>> >>> >>> super keen! >>> >>> (scary name, tho.) >>> >>> ATS supports erlang, somehow, apparently: >>> >>> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> >>> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ >>> erlang-questions mailing list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Tue Jun 28 19:34:31 2016 From: t@REDACTED (Tristan Sloughter) Date: Tue, 28 Jun 2016 12:34:31 -0500 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: <1467135271.726796.651076969.404B64A0@webmail.messagingengine.com> I like BML, it is similar to BMO from Adventure Time :) -- which gives the opening for a mascot! -- Tristan Sloughter t@REDACTED On Tue, Jun 28, 2016, at 11:22 AM, Rickard Andersson wrote: > Simply like "BML", no? Something like "BeeML", only spelled "BeaML". > > > On 28-Jun-16 19:10, Louis Pilfold wrote: > > Hey > > > > How would pronounce BeaML? It seems clumsy to say, and would be easy > > to confuse with BEAM if said too quickly. > > > > Cheers, > > Louis > > > > On 28 June 2016 at 13:42, Oliver Korpilla wrote: > >> Great to see such a worthy project. > >> > >> I also wholeheartedly support "BeaML" as a title. > >> > >> Cheers, > >> Oliver > >> > >> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr > >> Von: "Joshua Barney" > >> An: "Louis Pilfold" > >> Cc: erlang-questions > >> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM > >> > >> This seems like a very fun project. > >> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > >> > >> Sent from my iPhone > >> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: > >> > >> Hello! > >> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > >> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > >> I also find the name quite unwieldy. > >> Cheers, > >> Louis > >> > >> On 28 Jun 2016 06:44, "Alin Popa" wrote: > >> Well done; I can't wait to see how it goes. > >> > >> (indeed, something's weird about that name). > >> > >> Alin > >> > >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > >> > >> > >> super keen! > >> > >> (scary name, tho.) > >> > >> ATS supports erlang, somehow, apparently: > >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED[erlang-questions@REDACTED] > >> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED[erlang-questions@REDACTED] > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED[erlang-questions@REDACTED] > >> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed Jun 29 01:39:57 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 29 Jun 2016 11:39:57 +1200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> Message-ID: <953a300f-3b35-4d54-4ece-1eb3e3558bcd@cs.otago.ac.nz> On 29/06/16 4:10 AM, Louis Pilfold wrote: > How would pronounce BeaML? bee-mill, presumably. I note that Elixir is sort of Ruby-Flavour Erlang, but it's not called RFE. Since it's CAML-flavoured rather than (S)ML-flavoured, perhaps CARMEL (CAml-Resembling Modified Erlang Language) might work as a name. It's an excellent thing that Jeremy Pierre is doing. I have never been able to stomach CAML syntax despite my great respect for the work of Xavier Leroy and awed admiration for some of the things that have been done with it. F# now has a "lite" syntax that comes closer to something I could bear to use. There is some very interesting data mining work being done in F#, and that might also serve as an inspiration. I got quite excited at the thought of something like ML running in Erlang, and was bitterly disappointed to find it was no such thing. If you put "ML" in the name, you *will* raise and dash just such hopes. From n.oxyde@REDACTED Wed Jun 29 12:31:52 2016 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 29 Jun 2016 12:31:52 +0200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: <908D5277-8476-41AE-AB62-108DC0EB6C35@gmail.com> I'm sorry for the documentation pain you encountered by not having the documentation for the cerl module. Unfortunately the OTP team decided it was more important to not have utilitarian modules listed first in the documentation, rather than having documentation at all. https://github.com/erlang/otp/pull/363#issuecomment-99829062 > Le 28 juin 2016 ? 01:23, Jeremy Pierre a ?crit : > > Hi all, > > I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > > Blog post: http://noisycode.com/blog/2016/06/27/introducing-mlfe/ > Repo: https://github.com/j14159/mlfe > > TLDR: > - written in Erlang with leex, yecc, and cerl > - all inferenced, typer works (albeit lots of testing still to do, no binaries nor maps) > - PIDs are typed, as are message flows (but not if you're calling from Erlang) > - Apache 2.0 > > Jeremy > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Jun 29 16:35:44 2016 From: g@REDACTED (Garrett Smith) Date: Wed, 29 Jun 2016 09:35:44 -0500 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: <7CB06DBD-C43A-466B-BB6E-92E1B807AE41@icloud.com> References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> <7CB06DBD-C43A-466B-BB6E-92E1B807AE41@icloud.com> Message-ID: On Tue, Jun 28, 2016 at 12:00 PM, Joshua Barney wrote: > Yeah I can see that being a concern, but I think "bee-mul" or "beem-el" with two syllables is clearly different from beam. > > Do you have a preferred pronunciation of MLFE? The market will determine this one - and I'm _pretty_ sure how that's going to play out. I'm with Tristan on BML (bee em el). There no current command or package that collides. It's accurate, descriptive and sounds cool - esp if you're a BMX/motorcross fan. BeaML is cool on paper, but IMO sounds lamo - bee-mul. Rhymes with weevil. From daniel.goertzen@REDACTED Wed Jun 29 16:40:27 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 29 Jun 2016 14:40:27 +0000 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: This would go really well with Elm. I've been studying Rust, Haskell and Elm lately which has made me yearn for more static-ness in Erlang. I would love to see something like this reach the same stature as Elixir. I had some fun at http://www.litscape.com/word_tools/contains_only.php . Promising candidates were AMBLE, BALM, BLAME, LAMB. But BML is good too. :) On Mon, Jun 27, 2016 at 11:30 PM Jeremy Pierre wrote: > Hi all, > > I've been hacking away at an ML-shaped language for the BEAM for the past > few months and just opened it up. I figure a few people on the list might > be interested in poking at it and may have thoughts on improvements. > > Blog post: http://noisycode.com/blog/2016/06/27/introducing-mlfe/ > Repo: https://github.com/j14159/mlfe > > TLDR: > - written in Erlang with leex, yecc, and cerl > - all inferenced, typer works (albeit lots of testing still to do, no > binaries nor maps) > - PIDs are typed, as are message flows (but not if you're calling from > Erlang) > - Apache 2.0 > > Jeremy > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Jun 29 16:43:50 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 29 Jun 2016 23:43:50 +0900 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: <7CB06DBD-C43A-466B-BB6E-92E1B807AE41@icloud.com> Message-ID: <2657519.fOG2KmjfZe@changa> On 2016?6?29? ??? 09:35:44 Garrett Smith wrote: > On Tue, Jun 28, 2016 at 12:00 PM, Joshua Barney wrote: > > Yeah I can see that being a concern, but I think "bee-mul" or "beem-el" with two syllables is clearly different from beam. > > > > Do you have a preferred pronunciation of MLFE? > > The market will determine this one - and I'm _pretty_ sure how that's > going to play out. > > I'm with Tristan on BML (bee em el). There no current command or > package that collides. It's accurate, descriptive and sounds cool - > esp if you're a BMX/motorcross fan. > > BeaML is cool on paper, but IMO sounds lamo - bee-mul. Rhymes with weevil. I actually like MLFE -> "milf-ee" -> Milfie. In part because of the unforgettable (if risque) association, and in part because I letter sounded acronyms drive me nuts. Anyway, we obssess about this too much, and yet still tend to come up with forgettable, graybeard, IETF-worthy jargonish names for everything. Just call it Hooey and be done with it! ;-) -Craig From j.14159@REDACTED Wed Jun 29 17:44:24 2016 From: j.14159@REDACTED (Jeremy Pierre) Date: Wed, 29 Jun 2016 15:44:24 +0000 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: Apologies for the lack of quoted content here, just switched off digest mode. I don't have particularly strong opinions on the name of the language beyond simplicity/straightforwardness to be honest. Mostly I've just been saying the letters individually or "ML Flavoured Erlang". I don't really like "BeaML" but "BML" is fine and has the virtue of shorter file extensions. I'll give it some thought, a few others have suggested different names off-list as well and I've heard from a few people now that they expected something closer to standard ML given the naming :) With respect to contributions, I'm really fine with most any help, especially around finding the almost-certain-legion-of-bugs in the type inferencer. I'm planning on tackling maps and binaries in the very near term and I'm curious to hear thoughts on eunit-like testing facilities. I haven't thought about it a lot beyond something like: 

module example

export add/2

add x y = x + y

test add with
  "An add test"
  maybe_some_fixture_fun
  [list, of, tests]]
But that might be terrible for reasons not immediately obvious to me. Other low hanging fruit includes stuff like adding annotations in the code generator (e.g. line numbers) and filling in useful/missing BIFs. Near to medium term things I have more particular thoughts on are type annotations and code formatters I'll write blog posts about. If there are any emacs wizards around, I certainly wouldn't say no to a major mode and I'm game to debate most ;) Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From imet.ilm@REDACTED Wed Jun 29 18:45:39 2016 From: imet.ilm@REDACTED (=?utf-8?Q?Iuri_Lug=C3=A3o_Machado?=) Date: Wed, 29 Jun 2016 13:45:39 -0300 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: References: Message-ID: Nice! I?ve been toying with the idea for a while. Something that resembles F# or Elm (since Erlang has some side effects). But there are some really cool and tricky things that might be fun to solve, such as: 1. How to combine strong and static typing with the let it crash philosophy of Erlang. Since one of the objectives of any strong, static typed language is to avoid runtime errors due to wrong data types being passed, I wonder how to deal with this. Maybe using Elm?s approach of ports and subscriptions would be a good fit. 2. How to make OTP constructs (gen_server, gen_event, gen_statem?) and all the related distribution libraries look ?native? to the language. By native, I mean, make language constructs that doesn?t care if your system is single or multi node. The idea might be having something called ?The architecture? (again, similar to Elm?s Architecture) that is a single, and sane way to write distributed systems and let the language implementation deal with the details. 3. There is no global state in Erlang. You can simulate that using ETS but I don?t see that scaling well in the long run. Probably there are more caveats than I?ve pointed here. I find the idea of Functional Reactive Programming - FRP - quite fitting the Erlang/OTP ecosystem. With kind regards, I. > Em 29 de jun de 2016, ?(s) 07:00, erlang-questions-request@REDACTED escreveu: > > Send erlang-questions mailing list submissions to > erlang-questions@REDACTED > > To subscribe or unsubscribe via the World Wide Web, visit > http://erlang.org/mailman/listinfo/erlang-questions > or, via email, send a message with subject or body 'help' to > erlang-questions-request@REDACTED > > You can reach the person managing the list at > erlang-questions-owner@REDACTED > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of erlang-questions digest..." > Today's Topics: > > 1. Re: ML shaped language for the Erlang VM (Joshua Barney) > 2. Re: ML shaped language for the Erlang VM (Oliver Korpilla) > 3. Re: ML shaped language for the Erlang VM (Louis Pilfold) > 4. Re: ML shaped language for the Erlang VM (Oliver Korpilla) > 5. Re: ML shaped language for the Erlang VM (Joshua Barney) > 6. Re: ML shaped language for the Erlang VM (Rickard Andersson) > 7. Re: ML shaped language for the Erlang VM (Felix Gallo) > 8. Re: ML shaped language for the Erlang VM (Tristan Sloughter) > 9. Re: ML shaped language for the Erlang VM (Richard A. O'Keefe) > > De: Joshua Barney > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 08:54:57 BRT > Para: Louis Pilfold > Cc: erlang-questions > > > This seems like a very fun project. > Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > > Sent from my iPhone > > On Jun 28, 2016, at 3:28 AM, Louis Pilfold > wrote: > >> Hello! >> >> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >> >> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >> >> I also find the name quite unwieldy. >> >> Cheers, >> Louis >> >> On 28 Jun 2016 06:44, "Alin Popa" > wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke > wrote: >> > I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > De: "Oliver Korpilla" > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 09:42:06 BRT > Para: "Joshua Barney" > Cc: erlang-questions > > > Great to see such a worthy project. > > I also wholeheartedly support "BeaML" as a title. > > Cheers, > Oliver > > Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr > Von: "Joshua Barney" > An: "Louis Pilfold" > Cc: erlang-questions > Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM > > This seems like a very fun project. > Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > > Sent from my iPhone > On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: > > Hello! > How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > I also find the name quite unwieldy. > Cheers, > Louis > > On 28 Jun 2016 06:44, "Alin Popa" wrote: > Well done; I can't wait to see how it goes. > > (indeed, something's weird about that name). > > Alin > > On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > > > super keen! > > (scary name, tho.) > > ATS supports erlang, somehow, apparently: > https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > > > De: Louis Pilfold > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 13:10:35 BRT > Para: Oliver Korpilla > Cc: erlang-questions > > > Hey > > How would pronounce BeaML? It seems clumsy to say, and would be easy > to confuse with BEAM if said too quickly. > > Cheers, > Louis > > On 28 June 2016 at 13:42, Oliver Korpilla wrote: >> Great to see such a worthy project. >> >> I also wholeheartedly support "BeaML" as a title. >> >> Cheers, >> Oliver >> >> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >> Von: "Joshua Barney" >> An: "Louis Pilfold" >> Cc: erlang-questions >> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >> >> This seems like a very fun project. >> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >> >> Sent from my iPhone >> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >> >> Hello! >> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >> I also find the name quite unwieldy. >> Cheers, >> Louis >> >> On 28 Jun 2016 06:44, "Alin Popa" wrote: >> Well done; I can't wait to see how it goes. >> >> (indeed, something's weird about that name). >> >> Alin >> >> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >> >> >> super keen! >> >> (scary name, tho.) >> >> ATS supports erlang, somehow, apparently: >> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED[erlang-questions@REDACTED] >> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > > > De: Oliver Korpilla > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 13:43:02 BRT > Para: Louis Pilfold > Cc: erlang-questions > > > Beam-el. > > On June 28, 2016 6:10:35 PM CEST, Louis Pilfold wrote: > Hey > > How would pronounce BeaML? It seems clumsy to say, and would be easy > to confuse with BEAM if said too quickly. > > Cheers, > Louis > > On 28 June 2016 at 13:42, Oliver Korpilla wrote: > Great to see such a worthy project. > > I also wholeheartedly support "BeaML" as a title. > > Cheers, > Oliver > > Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr > Von: "Joshua Barney" > An: "Louis Pilfold" > Cc: erlang-questions > Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM > > This seems like a very fun project. > Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > > Sent from my iPhone > On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: > > Hello! > How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > I also find the name quite unwieldy. > Cheers, > Louis > > On 28 Jun 2016 06:44, "Alin Popa" wrote: > Well done; I can't wait to see how it goes. > > (indeed, something's weird about that name). > > Alin > > On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > > > super keen! > > (scary name, tho.) > > ATS supports erlang, somehow, apparently: > https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl [https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl ] > > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > > > erlang-questions mailing list > erlang-questions@REDACTED[erlang-questions@REDACTED] > http://erlang.org/mailman/listinfo/erlang-questions > erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions [http://erlang.org/mailman/listinfo/erlang-questions ] > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > De: Joshua Barney > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 14:00:34 BRT > Para: Louis Pilfold > Cc: erlang-questions > > > Yeah I can see that being a concern, but I think "bee-mul" or "beem-el" with two syllables is clearly different from beam. > > Do you have a preferred pronunciation of MLFE? > > Sent from my iPhone > >> On Jun 28, 2016, at 12:10 PM, Louis Pilfold wrote: >> >> Hey >> >> How would pronounce BeaML? It seems clumsy to say, and would be easy >> to confuse with BEAM if said too quickly. >> >> Cheers, >> Louis >> >>> On 28 June 2016 at 13:42, Oliver Korpilla wrote: >>> Great to see such a worthy project. >>> >>> I also wholeheartedly support "BeaML" as a title. >>> >>> Cheers, >>> Oliver >>> >>> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >>> Von: "Joshua Barney" >>> An: "Louis Pilfold" >>> Cc: erlang-questions >>> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >>> >>> This seems like a very fun project. >>> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >>> >>> Sent from my iPhone >>> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >>> >>> Hello! >>> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >>> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >>> I also find the name quite unwieldy. >>> Cheers, >>> Louis >>> >>> On 28 Jun 2016 06:44, "Alin Popa" wrote: >>> Well done; I can't wait to see how it goes. >>> >>> (indeed, something's weird about that name). >>> >>> Alin >>> >>> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >>> >>> >>> super keen! >>> >>> (scary name, tho.) >>> >>> ATS supports erlang, somehow, apparently: >>> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > > > De: Rickard Andersson > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 13:22:03 BRT > Para: erlang-questions@REDACTED > > > Simply like "BML", no? Something like "BeeML", only spelled "BeaML". > > > On 28-Jun-16 19:10, Louis Pilfold wrote: >> Hey >> >> How would pronounce BeaML? It seems clumsy to say, and would be easy >> to confuse with BEAM if said too quickly. >> >> Cheers, >> Louis >> >> On 28 June 2016 at 13:42, Oliver Korpilla wrote: >>> Great to see such a worthy project. >>> >>> I also wholeheartedly support "BeaML" as a title. >>> >>> Cheers, >>> Oliver >>> >>> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >>> Von: "Joshua Barney" >>> An: "Louis Pilfold" >>> Cc: erlang-questions >>> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >>> >>> This seems like a very fun project. >>> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >>> >>> Sent from my iPhone >>> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >>> >>> Hello! >>> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >>> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >>> I also find the name quite unwieldy. >>> Cheers, >>> Louis >>> >>> On 28 Jun 2016 06:44, "Alin Popa" wrote: >>> Well done; I can't wait to see how it goes. >>> >>> (indeed, something's weird about that name). >>> >>> Alin >>> >>> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >>> >>> >>> super keen! >>> >>> (scary name, tho.) >>> >>> ATS supports erlang, somehow, apparently: >>> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > De: Felix Gallo > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 14:25:32 BRT > Para: Rickard Andersson > Cc: "erlang-questions@REDACTED" > > > MLFE is clearly a tribute to Dr. Melfi, Tony Soprano's therapist (http://sopranos.wikia.com/wiki/Jennifer_Melfi ), and would have to be pronounced that way... > > F. > > On Tue, Jun 28, 2016 at 9:22 AM, Rickard Andersson > wrote: > Simply like "BML", no? Something like "BeeML", only spelled "BeaML". > > > > On 28-Jun-16 19:10, Louis Pilfold wrote: > Hey > > How would pronounce BeaML? It seems clumsy to say, and would be easy > to confuse with BEAM if said too quickly. > > Cheers, > Louis > > On 28 June 2016 at 13:42, Oliver Korpilla > wrote: > Great to see such a worthy project. > > I also wholeheartedly support "BeaML" as a title. > > Cheers, > Oliver > > Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr > Von: "Joshua Barney" > > An: "Louis Pilfold" > > Cc: erlang-questions > > Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM > > This seems like a very fun project. > Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) > > Sent from my iPhone > On Jun 28, 2016, at 3:28 AM, Louis Pilfold > wrote: > > Hello! > How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! > Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? > I also find the name quite unwieldy. > Cheers, > Louis > > On 28 Jun 2016 06:44, "Alin Popa" [alin.popa@REDACTED ]> wrote: > Well done; I can't wait to see how it goes. > > (indeed, something's weird about that name). > > Alin > > On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke [raould@REDACTED ]> wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. > > > super keen! > > (scary name, tho.) > > ATS supports erlang, somehow, apparently: > https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED [erlang-questions@REDACTED ] > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED [erlang-questions@REDACTED ] > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED [erlang-questions@REDACTED ] > http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > De: Tristan Sloughter > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 14:34:31 BRT > Para: Rickard Andersson , erlang-questions@REDACTED > > > I like BML, it is similar to BMO from Adventure Time :) -- which gives > the opening for a mascot! > > -- > Tristan Sloughter > t@REDACTED > > On Tue, Jun 28, 2016, at 11:22 AM, Rickard Andersson wrote: >> Simply like "BML", no? Something like "BeeML", only spelled "BeaML". >> >> >> On 28-Jun-16 19:10, Louis Pilfold wrote: >>> Hey >>> >>> How would pronounce BeaML? It seems clumsy to say, and would be easy >>> to confuse with BEAM if said too quickly. >>> >>> Cheers, >>> Louis >>> >>> On 28 June 2016 at 13:42, Oliver Korpilla wrote: >>>> Great to see such a worthy project. >>>> >>>> I also wholeheartedly support "BeaML" as a title. >>>> >>>> Cheers, >>>> Oliver >>>> >>>> Gesendet: Dienstag, 28. Juni 2016 um 13:54 Uhr >>>> Von: "Joshua Barney" >>>> An: "Louis Pilfold" >>>> Cc: erlang-questions >>>> Betreff: Re: [erlang-questions] ML shaped language for the Erlang VM >>>> >>>> This seems like a very fun project. >>>> Might I suggest the name BeaML? (Current I can't stop seeing "milf" :-/) >>>> >>>> Sent from my iPhone >>>> On Jun 28, 2016, at 3:28 AM, Louis Pilfold wrote: >>>> >>>> Hello! >>>> How exciting, I've been toying with creation of something like this myself, but my compiler knowledge isn't quite there yet. Looking forward to learning from your work! >>>> Are there ways in which you would like people to contribute or get involved? Or anything you would like feedback on? >>>> I also find the name quite unwieldy. >>>> Cheers, >>>> Louis >>>> >>>> On 28 Jun 2016 06:44, "Alin Popa" wrote: >>>> Well done; I can't wait to see how it goes. >>>> >>>> (indeed, something's weird about that name). >>>> >>>> Alin >>>> >>>> On Tue, Jun 28, 2016 at 6:39 AM, Raoul Duke wrote:> I've been hacking away at an ML-shaped language for the BEAM for the past few months and just opened it up. I figure a few people on the list might be interested in poking at it and may have thoughts on improvements. >>>> >>>> >>>> super keen! >>>> >>>> (scary name, tho.) >>>> >>>> ATS supports erlang, somehow, apparently: >>>> https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl[https://github.com/githwxi/ATS-Postiats-contrib/tree/master/projects/MEDIUM/CATS-atsccomp/CATS-atscc2erl] >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED[erlang-questions@REDACTED] >>>> http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > De: "Richard A. O'Keefe" > Assunto: Re: [erlang-questions] ML shaped language for the Erlang VM > Data: 28 de junho de 2016 20:39:57 BRT > Para: > > > > > On 29/06/16 4:10 AM, Louis Pilfold wrote: >> How would pronounce BeaML? > > bee-mill, presumably. > > I note that Elixir is sort of Ruby-Flavour Erlang, > but it's not called RFE. > > Since it's CAML-flavoured rather than (S)ML-flavoured, > perhaps CARMEL (CAml-Resembling Modified Erlang Language) > might work as a name. > > It's an excellent thing that Jeremy Pierre is doing. > > I have never been able to stomach CAML syntax despite > my great respect for the work of Xavier Leroy and > awed admiration for some of the things that have been > done with it. F# now has a "lite" syntax that comes > closer to something I could bear to use. There is > some very interesting data mining work being done in > F#, and that might also serve as an inspiration. > > I got quite excited at the thought of something like > ML running in Erlang, and was bitterly disappointed > to find it was no such thing. If you put "ML" in the > name, you *will* raise and dash just such hopes. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.14159@REDACTED Thu Jun 30 04:21:58 2016 From: j.14159@REDACTED (Jeremy Pierre) Date: Thu, 30 Jun 2016 02:21:58 +0000 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: <908D5277-8476-41AE-AB62-108DC0EB6C35@gmail.com> References: <908D5277-8476-41AE-AB62-108DC0EB6C35@gmail.com> Message-ID: It wasn't too bad once I found out where to look :) Truth be told I still ended up needing to read the compiler code a few times to figure out some of my early mistakes. Jeremy On Wed, Jun 29, 2016, 03:31 Anthony Ramine wrote: > I'm sorry for the documentation pain you encountered by not having the > documentation for the cerl module. > > Unfortunately the OTP team decided it was more important to not have > utilitarian modules listed first in the documentation, rather than having > documentation at all. > > https://github.com/erlang/otp/pull/363#issuecomment-99829062 > > Le 28 juin 2016 ? 01:23, Jeremy Pierre a ?crit : > > Hi all, > > I've been hacking away at an ML-shaped language for the BEAM for the past > few months and just opened it up. I figure a few people on the list might > be interested in poking at it and may have thoughts on improvements. > > Blog post: http://noisycode.com/blog/2016/06/27/introducing-mlfe/ > Repo: https://github.com/j14159/mlfe > > TLDR: > - written in Erlang with leex, yecc, and cerl > - all inferenced, typer works (albeit lots of testing still to do, no > binaries nor maps) > - PIDs are typed, as are message flows (but not if you're calling from > Erlang) > - Apache 2.0 > > Jeremy > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Wed Jun 29 23:51:46 2016 From: igor.clark@REDACTED (Igor Clark) Date: Wed, 29 Jun 2016 14:51:46 -0700 Subject: [erlang-questions] Static callback in NIF Message-ID: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> Hi folks, I've got a NIF that uses some library code to talk to specific hardware. It's a hobby project with only one user (me) and no real performance concerns, so what I've got works well at the moment, but I think I'm doing some sneaky/dirty stuff and would like to know the best way to do what I need. Sending messages outwards from erlang->C->HW is easy, very quick, and works fine. I return a pointer to the HW reference back to erlang using the enif_*_resource functions, and manage keeping track of everything on the erlang side, which feels pretty natural. Coming the other way works fine too, but relies on a C function callback which gets called when the hardware has a message for me. Right now I just have a static function in the NIF C code which I pass to the library. I create a static ErlNifEnv on NIF load() which I keep around and use in the callback to send messages to a specified erlang Pid, passed in via enif_get_local_pid() in another NIF function and also stored statically. This works a treat, but I'm feeling some pretty strong dread that it's very much the wrong way to do things, and asking for scheduler headaches/explosions. I'm planning to try storing the various resources in priv_data at load() time, on the theory that that way the memory would at least be managed by the NIF system rather than just as enif_alloc()'ing static pointers, but I'm not sure if that would make any diffrence if code external to the scheduler calls back into it. I've looked into running this part as a C node or a port that sends messages with the HW data in a callback in its own process, and the communication seems straightforward enough, but it also seems like I immediately need to start designing mechanisms to deal with working out where to send received messages, almost a protocol in itself. Whereas with the NIF+callback method I have a lot of the work done for me - except, of course, for the synchronisation and memory management, which is the bit I'm worried about. FWIW the callback code doesn't modify any of the static data structures directly, it just calls library code which uses the stored references to work out which hardware device & channel to send the message to. What's the best practice here? Is a callback in a NIF OK if it's stored in priv_data, or is it never OK? What's the best way to do this if not? Would appreciate any tips! Cheers, Igor From ok@REDACTED Thu Jun 30 06:14:18 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 30 Jun 2016 16:14:18 +1200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: <908D5277-8476-41AE-AB62-108DC0EB6C35@gmail.com> References: <908D5277-8476-41AE-AB62-108DC0EB6C35@gmail.com> Message-ID: <88bd0449-2414-7198-0a58-1ab0985a2621@cs.otago.ac.nz> On 29/06/16 10:31 PM, Anthony Ramine wrote: > I'm sorry for the documentation pain you encountered by not having the > documentation for the cerl module. > > Unfortunately the OTP team decided it was more important to not have > utilitarian modules listed first in the documentation, rather than > having documentation at all. > > https://github.com/erlang/otp/pull/363#issuecomment-99829062 Thank you for that link, which puzzles me mightily. It seems to me obvious that there are three intrinsically unrelated things: - exposition order: the order in which a beginner needs to read a series of documents - reference frequency order: the order which puts the most often needed documents at the top and the least often needed documents at the bottom - alphabetic order: the dictionary order of the names of things. Anyone else remember the two-dimensional table of contents for "An Informal Introduction to Algol 68"? How about the textbooks with a reading-order digraph at the beginning? Of these three orders (not that these are the only three) the *least* important one is alphabetic order. You cannot use it unless you already know the name of the thing you are looking for. From magnus.ottenklinger@REDACTED Thu Jun 30 10:58:11 2016 From: magnus.ottenklinger@REDACTED (Magnus Ottenklinger) Date: Thu, 30 Jun 2016 08:58:11 +0000 Subject: [erlang-questions] A question on EINVAL in active-mode TCP Message-ID: Hi List. We stumbled over a peculiarity in the usage of ports in a TCP connection (active mode). Specifically, we encountered `einval` when writing to a socket which was closed just before. Our expectation was that the resulting error should be `{error,closed}`. The following module reproduces this on my machine (OTP 19.0). I ran the test module with `erlc test.erl && erl -s test run -noshell` as well as with `erl -smp disable +A 0 -s test run -noshell`. Both executions triggered `einval`. -module(test). -export([run/0]). run() -> run(0). run(I) -> io:format("Run ~p~n", [I]), run1(), run(I+1). run1() -> {ok,LSock} = gen_tcp:listen(0, []), {ok,Port} = inet:port(LSock), spawn_link( fun() -> {ok,Sock} = gen_tcp:accept(LSock), gen_tcp:close(LSock), gen_tcp:close(Sock) end), {ok,Sock} = gen_tcp:connect("localhost", Port, []), case gen_tcp:send(Sock, "Hello") of ok -> ok; {error,closed} -> ok end, gen_tcp:close(Sock). Here, the execution stops after about 3800 runs with `{case_clause,{error,einval}}`. It seems weird that the test module usually encounters `{error,closed}`, and only after some time hits `{error,einval}`. This hints at a race condition, but I am unsure if this is instead expected behaviour. So, finally to the actual question: Should we expect `einval` on active-mode TCP and interpret it as `closed`? Or is this a fixable race condition? Cheers, Magnus From sergej.jurecko@REDACTED Thu Jun 30 11:04:41 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Thu, 30 Jun 2016 11:04:41 +0200 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> Message-ID: > On 29 Jun 2016, at 23:51, Igor Clark wrote: > > Coming the other way works fine too, but relies on a C function callback which gets called when the hardware has a message for me. Right now I just have a static function in the NIF C code which I pass to the library. I create a static ErlNifEnv on NIF load() which I keep around and use in the callback to send messages to a specified erlang Pid, passed in via enif_get_local_pid() in another NIF function and also stored statically. This works a treat, but I'm feeling some pretty strong dread that it's very much the wrong way to do things, and asking for scheduler headaches/explosions. If there is a better way of doing this I haven't found it and I've spent a lot of time in NIF's. An issue might be if you end up using the same environment variable across multiple threads. Sergej -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Thu Jun 30 10:59:17 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Thu, 30 Jun 2016 10:59:17 +0200 Subject: [erlang-questions] ML shaped language for the Erlang VM In-Reply-To: <953a300f-3b35-4d54-4ece-1eb3e3558bcd@cs.otago.ac.nz> References: <74E141AB-566F-45D9-B52B-6F472206AAC1@icloud.com> <953a300f-3b35-4d54-4ece-1eb3e3558bcd@cs.otago.ac.nz> Message-ID: <5774e71d.c90f190a.b0854.620a@mx.google.com> "Richard A. O'Keefe" writes: > On 29/06/16 4:10 AM, Louis Pilfold wrote: >> How would pronounce BeaML? > > bee-mill, presumably. > > I note that Elixir is sort of Ruby-Flavour Erlang, > but it's not called RFE. > > Since it's CAML-flavoured rather than (S)ML-flavoured, > perhaps CARMEL (CAml-Resembling Modified Erlang Language) > might work as a name. > > It's an excellent thing that Jeremy Pierre is doing. > > I have never been able to stomach CAML syntax despite > my great respect for the work of Xavier Leroy and > awed admiration for some of the things that have been > done with it. F# now has a "lite" syntax that comes > closer to something I could bear to use. There is > some very interesting data mining work being done in > F#, and that might also serve as an inspiration. > I agree - Caml is not as nice as SML. > I got quite excited at the thought of something like > ML running in Erlang, and was bitterly disappointed > to find it was no such thing. If you put "ML" in the > name, you *will* raise and dash just such hopes. > Equally excited - have a number of notes on the topic, but never found the time to turn it into something real. And putting ML in the name will induce expecations. Name suggestion: sial Static Inferred Actor Language Sial actually means the upper crust of the Earth's crust [1] - that would be a suiting analogy as it will sit on top of the BEAM. Cheers, Torben [1] https://en.wikipedia.org/wiki/Sial > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Torben Hoffmann Chief Architect, basho.com M: +45 25 14 05 38 From roger@REDACTED Thu Jun 30 13:01:52 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 30 Jun 2016 12:01:52 +0100 Subject: [erlang-questions] Static callback in NIF In-Reply-To: References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> Message-ID: As I understand it, if you're using enif_send from a background thread, you should allocate a fresh environment for that case, using enif_alloc_env: ErlNifEnv *msg_env = enif_alloc_env(); ERL_NIF_TERM a_moo = enif_make_atom(msg_env, "moo"); enif_send(NULL, &m_parentPid, msg_env, a_moo); enif_free_env(msg_env); You can keep the pid around between threads, though: ErlNifPid parentPid; if (!enif_get_local_pid(env, argv[0], &m_parentPid)) return enif_make_badarg(env); On 30 June 2016 at 10:04, Sergej Jure?ko wrote: > > On 29 Jun 2016, at 23:51, Igor Clark wrote: > > Coming the other way works fine too, but relies on a C function callback > which gets called when the hardware has a message for me. Right now I just > have a static function in the NIF C code which I pass to the library. I > create a static ErlNifEnv on NIF load() which I keep around and use in the > callback to send messages to a specified erlang Pid, passed in via > enif_get_local_pid() in another NIF function and also stored statically. > This works a treat, but I'm feeling some pretty strong dread that it's very > much the wrong way to do things, and asking for scheduler > headaches/explosions. > > > If there is a better way of doing this I haven't found it and I've spent a > lot of time in NIF's. An issue might be if you end up using the same > environment variable across multiple threads. > > Sergej > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roger@REDACTED Thu Jun 30 13:43:09 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 30 Jun 2016 12:43:09 +0100 Subject: [erlang-questions] type record() undefined Message-ID: I have the following: -spec parse_json(JSON :: undefined | binary(), Record :: record(), Parser :: function()) -> record(). With Erlang 18.3, I get "type record() undefined". With Erlang 17.x, I don't. Was this always wrong, and the compiler's got stricter? From kostis@REDACTED Thu Jun 30 13:45:02 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 30 Jun 2016 14:45:02 +0300 Subject: [erlang-questions] type record() undefined In-Reply-To: References: Message-ID: <108be64b-c7c5-3a75-b6bb-e60c36f5c435@cs.ntua.gr> On 06/30/16 14:43, Roger Lipscombe wrote: > I have the following: > > -spec parse_json(JSON :: undefined | binary(), Record :: record(), > Parser :: function()) -> record(). > > With Erlang 18.3, I get "type record() undefined". With Erlang 17.x, I don't. > > Was this always wrong, and the compiler's got stricter? Yes. Kostis From hans.r.nilsson@REDACTED Thu Jun 30 15:31:49 2016 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Thu, 30 Jun 2016 15:31:49 +0200 Subject: [erlang-questions] Patch package OTP 19.0.1 released Message-ID: <57751F45.3030409@ericsson.com> Patch Package: OTP 19.0.1 Git Tag: OTP-19.0.1 Date: 2016-06-30 Trouble Report Id: OTP-13674, OTP-13702, OTP-13708, OTP-13709, OTP-13712, OTP-13715, OTP-13716 Seq num: seq13143 System: OTP Release: 19 Application: dialyzer-3.0.1, erts-8.0.1, inets-6.3.1, observer-2.2.1, ssh-4.3.1, tools-2.8.5 Predecessor: OTP 19.0 Check out the git tag OTP-19.0.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- dialyzer-3.0.1 -------------------------------------------------- --------------------------------------------------------------------- The dialyzer-3.0.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13709 Application(s): dialyzer Related Id(s): ERL-177, PR-1115 Fix a map related bug. Full runtime dependencies of dialyzer-3.0.1: compiler-7.0, erts-8.0, hipe-3.15.1, kernel-5.0, stdlib-3.0, syntax_tools-2.0, wx-1.2 --------------------------------------------------------------------- --- erts-8.0.1 ------------------------------------------------------ --------------------------------------------------------------------- The erts-8.0.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13716 Application(s): erts A memory allocation bug in group_leader/2 could cause an emulator crash when garbage collecting a process that had been assigned a remote group leader. This bug was introduced in ERTS version 8.0. Full runtime dependencies of erts-8.0.1: kernel-5.0, sasl-3.0, stdlib-3.0 --------------------------------------------------------------------- --- inets-6.3.1 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.3.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13712 Application(s): inets Related Id(s): seq13143 A debug message was accidently left enabled in the ftp client. Full runtime dependencies of inets-6.3.1: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- observer-2.2.1 -------------------------------------------------- --------------------------------------------------------------------- The observer-2.2.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13702 Application(s): observer Related Id(s): ERL-171 Fixed a crash happening when observing another node, who have a different number of schedulers than the current one. Full runtime dependencies of observer-2.2.1: erts-7.0, et-1.5, inets-5.10, kernel-3.0, runtime_tools-1.8.14, stdlib-2.0, wx-1.2 --------------------------------------------------------------------- --- ssh-4.3.1 ------------------------------------------------------- --------------------------------------------------------------------- The ssh-4.3.1 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13674 Application(s): ssh Related Id(s): TR-HU92273 SSH client does not any longer retry a bad password given as option to ssh:connect et al. OTP-13715 Application(s): ssh Removed possible hanging risk for a certain timing sequence when communicating client and server executes on the same node. Full runtime dependencies of ssh-4.3.1: crypto-3.3, erts-6.0, kernel-3.0, public_key-1.1, stdlib-3.0 --------------------------------------------------------------------- --- tools-2.8.5 ----------------------------------------------------- --------------------------------------------------------------------- The tools-2.8.5 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-13708 Application(s): tools Related Id(s): ERL-173 Correct a bug when adding multiple modules to an Xref server. The bug was introduced in OTP-19.0. Full runtime dependencies of tools-2.8.5: compiler-5.0, erts-7.0, inets-5.10, kernel-3.0, runtime_tools-1.8.14, stdlib-2.5 --------------------------------------------------------------------- --- Thanks to ------------------------------------------------------- --------------------------------------------------------------------- Magnus L?ng --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4101 bytes Desc: S/MIME Cryptographic Signature URL: From jesper.louis.andersen@REDACTED Thu Jun 30 15:58:59 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 30 Jun 2016 15:58:59 +0200 Subject: [erlang-questions] A question on EINVAL in active-mode TCP In-Reply-To: References: Message-ID: On Thu, Jun 30, 2016 at 10:58 AM, Magnus Ottenklinger < magnus.ottenklinger@REDACTED> wrote: > We stumbled over a peculiarity in the usage of ports in a TCP connection > (active mode). Specifically, we encountered `einval` when writing to a > socket which was closed just before. Our expectation was that the resulting > error should be `{error,closed}`. The following module reproduces this on > my machine (OTP 19.0). I ran the test module with `erlc test.erl && erl -s > test run -noshell` as well as with `erl -smp disable +A 0 -s test run > -noshell`. Both executions triggered `einval`. Knowing what operating system it is could help. Low level TCP isn't entirely the same thing in its response codes, even though POSIX exists. There are some things down here which are up to "interpretation" for some levels of interpretation. It does, from your failing program, look like it is a race however. The question is if anything can be done about it. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Thu Jun 30 16:45:56 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 30 Jun 2016 14:45:56 +0000 Subject: [erlang-questions] Static callback in NIF In-Reply-To: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> Message-ID: Static vs priv_data are functionally the same here so it doesn't matter which way you go. I can empathize with your sense of dread; there are a lot of rules to keep track of for keeping Undefined Behavior at bay. But you seem to have a good handle on things. On Thu, Jun 30, 2016 at 6:02 AM Igor Clark wrote: > Hi folks, > > I've got a NIF that uses some library code to talk to specific hardware. > It's a hobby project with only one user (me) and no real performance > concerns, so what I've got works well at the moment, but I think I'm > doing some sneaky/dirty stuff and would like to know the best way to do > what I need. > > Sending messages outwards from erlang->C->HW is easy, very quick, and > works fine. I return a pointer to the HW reference back to erlang using > the enif_*_resource functions, and manage keeping track of everything on > the erlang side, which feels pretty natural. > > Coming the other way works fine too, but relies on a C function callback > which gets called when the hardware has a message for me. Right now I > just have a static function in the NIF C code which I pass to the > library. I create a static ErlNifEnv on NIF load() which I keep around > and use in the callback to send messages to a specified erlang Pid, > passed in via enif_get_local_pid() in another NIF function and also > stored statically. This works a treat, but I'm feeling some pretty > strong dread that it's very much the wrong way to do things, and asking > for scheduler headaches/explosions. > > I'm planning to try storing the various resources in priv_data at load() > time, on the theory that that way the memory would at least be managed > by the NIF system rather than just as enif_alloc()'ing static pointers, > but I'm not sure if that would make any diffrence if code external to > the scheduler calls back into it. > > I've looked into running this part as a C node or a port that sends > messages with the HW data in a callback in its own process, and the > communication seems straightforward enough, but it also seems like I > immediately need to start designing mechanisms to deal with working out > where to send received messages, almost a protocol in itself. Whereas > with the NIF+callback method I have a lot of the work done for me - > except, of course, for the synchronisation and memory management, which > is the bit I'm worried about. > > FWIW the callback code doesn't modify any of the static data structures > directly, it just calls library code which uses the stored references to > work out which hardware device & channel to send the message to. > > What's the best practice here? Is a callback in a NIF OK if it's stored > in priv_data, or is it never OK? What's the best way to do this if not? > > Would appreciate any tips! > > Cheers, > Igor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Thu Jun 30 16:58:41 2016 From: james@REDACTED (James Fish) Date: Thu, 30 Jun 2016 15:58:41 +0100 Subject: [erlang-questions] A question on EINVAL in active-mode TCP In-Reply-To: References: Message-ID: I believe the race condition occurring is that the gen_tcp port will exit after it sends a {tcp_closed, Socket} message when in active mode. When calling a gen_tcp function the module for the Socket (Erlang port) is looked up using a BIF (for ipv4 and ipv6 are different modules internally). If the port doesn't exist when the lookup occurs then you get {error, closed}. Otherwise the module is returned and that module is used to communicate with Socket (Erlang port). It is possible for the port to exit in between the lookup of the module and the actual call. This situation leads to {error, einval} as that is the error returned by the lower level inet port calls. You can see this occur when using inet:setopts/2 as the module isn't looked up (IIRC) and {error, einval} is always returned on a "dead" Erlang port. It is possible to stop the gen_tcp socket exiting on {tcp_closed, Socket} by using the {exit_on_close, false} inet option. However then you will need to call gen_tcp:close(Socket) if the controlling process does not exit when the connection closes. On 30 June 2016 at 14:58, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Thu, Jun 30, 2016 at 10:58 AM, Magnus Ottenklinger < > magnus.ottenklinger@REDACTED> wrote: > >> We stumbled over a peculiarity in the usage of ports in a TCP connection >> (active mode). Specifically, we encountered `einval` when writing to a >> socket which was closed just before. Our expectation was that the resulting >> error should be `{error,closed}`. The following module reproduces this on >> my machine (OTP 19.0). I ran the test module with `erlc test.erl && erl -s >> test run -noshell` as well as with `erl -smp disable +A 0 -s test run >> -noshell`. Both executions triggered `einval`. > > > Knowing what operating system it is could help. Low level TCP isn't > entirely the same thing in its response codes, even though POSIX exists. > There are some things down here which are up to "interpretation" for some > levels of interpretation. > > It does, from your failing program, look like it is a race however. The > question is if anything can be done about it. > > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jun 30 17:08:36 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 30 Jun 2016 17:08:36 +0200 Subject: [erlang-questions] Patch package OTP 19.0.1 released In-Reply-To: <57751F45.3030409@ericsson.com> References: <57751F45.3030409@ericsson.com> Message-ID: <577535F4.5030108@ninenines.eu> On 06/30/2016 03:31 PM, Hans Nilsson R wrote: > --------------------------------------------------------------------- > --- dialyzer-3.0.1 -------------------------------------------------- > --------------------------------------------------------------------- > > The dialyzer-3.0.1 application can be applied independently of other > applications on a full OTP 19 installation. > > --- Fixed Bugs and Malfunctions --- > > OTP-13709 Application(s): dialyzer > Related Id(s): ERL-177, PR-1115 > > Fix a map related bug. Much appreciated. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From igor.clark@REDACTED Thu Jun 30 20:31:50 2016 From: igor.clark@REDACTED (Igor Clark) Date: Thu, 30 Jun 2016 11:31:50 -0700 Subject: [erlang-questions] Static callback in NIF In-Reply-To: References: <7549f4ba-467c-a16a-ba48-5455e54d1015@gmail.com> Message-ID: <508348d9-30ee-c5ce-cc80-a4ce5d530759@gmail.com> Thanks Daniel, good to hear. Thanks also Roger & Sergej for your replies. I'll try out enif_alloc_env()'ing a new ErlNifEnv each time the callback uses enif_send(), rather than just leaving it static. Cheers! Igor On 30/06/2016 07:45, Daniel Goertzen wrote: > Static vs priv_data are functionally the same here so it doesn't > matter which way you go. I can empathize with your sense of dread; > there are a lot of rules to keep track of for keeping Undefined > Behavior at bay. But you seem to have a good handle on things. > > On Thu, Jun 30, 2016 at 6:02 AM Igor Clark > wrote: > > Hi folks, > > I've got a NIF that uses some library code to talk to specific > hardware. > It's a hobby project with only one user (me) and no real performance > concerns, so what I've got works well at the moment, but I think I'm > doing some sneaky/dirty stuff and would like to know the best way > to do > what I need. > > Sending messages outwards from erlang->C->HW is easy, very quick, and > works fine. I return a pointer to the HW reference back to erlang > using > the enif_*_resource functions, and manage keeping track of > everything on > the erlang side, which feels pretty natural. > > Coming the other way works fine too, but relies on a C function > callback > which gets called when the hardware has a message for me. Right now I > just have a static function in the NIF C code which I pass to the > library. I create a static ErlNifEnv on NIF load() which I keep around > and use in the callback to send messages to a specified erlang Pid, > passed in via enif_get_local_pid() in another NIF function and also > stored statically. This works a treat, but I'm feeling some pretty > strong dread that it's very much the wrong way to do things, and > asking > for scheduler headaches/explosions. > > I'm planning to try storing the various resources in priv_data at > load() > time, on the theory that that way the memory would at least be managed > by the NIF system rather than just as enif_alloc()'ing static > pointers, > but I'm not sure if that would make any diffrence if code external to > the scheduler calls back into it. > > I've looked into running this part as a C node or a port that sends > messages with the HW data in a callback in its own process, and the > communication seems straightforward enough, but it also seems like I > immediately need to start designing mechanisms to deal with > working out > where to send received messages, almost a protocol in itself. Whereas > with the NIF+callback method I have a lot of the work done for me - > except, of course, for the synchronisation and memory management, > which > is the bit I'm worried about. > > FWIW the callback code doesn't modify any of the static data > structures > directly, it just calls library code which uses the stored > references to > work out which hardware device & channel to send the message to. > > What's the best practice here? Is a callback in a NIF OK if it's > stored > in priv_data, or is it never OK? What's the best way to do this if > not? > > Would appreciate any tips! > > Cheers, > Igor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: