[erlang-questions] On OTP rand module difference between OTP 19 and OTP 20

[Raimo Niskanen]

On Tue, Aug 29, 2017 at 10:45:51AM +0200, Raimo Niskanen wrote:
> On Tue, Aug 29, 2017 at 10:35:28AM +0200, Raimo Niskanen wrote:
> > Thank you Kenji for reviewing the changes and summarizing the implications.
> > 
> > Sorry about not getting you into the loop during the rewrite!
> > 
> > As you guessed I was focused on plugging in 'crypto', fixing the flaws,
> > incorporating the new algorithm from Prof. Vignia and keeping it
> > as backwards compatible as possible, so I forgot about you...
> > 
> > 
> > 
> > Regarding the changed uniform float behaviour: it is the functions
> > rand:uniform/0 and rand:uniform_s/1 this concerns.  They were previously
> > (OTP-19.3) documented to output a value 0.0 < X < 1.0 and are now
> > (OTP-20.0) documented to return 0.0 =< X < 1.0.
> > 
> > Previously they _could_ return exactly 0.0, even though not stated in the
> > documentation.  But the probability for exactly 0.0 was about 2048 times
> > less than it is now.  This is because the float generation has been fixed
> > to generate equidistant floats on the form K * 2^-53 so exactly 0.0 has now
> > got the same probability as all other possible float values.
> > 
> > I'd rather say that the documentation of rand:uniform/0 and rand:uniform_s/1
> > has been corrected to match their behaviour, and that the probability for
> > exactly 0.0 has increased from about 1/64 to about 1/53.
> 
> Sorry!  That should be:     from about 1/2^64 to about 1/2^53.

And that was for the 64-bit generators.  For the _default_ 58-bit generator
the probability for exactly 0.0 has changed from about 1/2^58 to about
1/2^53 i.e increased with a factor 32.

/ Raimo


> 
> / Raimo
> 
> 
> > 
> > Despite this the distribution of generated numbers has actually not changed
> > - it is still uniform over the range 0.0 =< X < 1.0.
> > 
> > That is my view of the float changes, of which I am fairly certain. :-)
> > 
> > 
> > / Raimo
> > 
> > 
> > 
> > On Tue, Aug 29, 2017 at 10:53:17AM +0900, Kenji Rikitake wrote:
> > > I finally reviewed OTP rand module change between 19.3 and 20.0.
> > > While most of users won't recognize the changes,
> > > the details are considerably different.
> > > 
> > > A good news is that if you're not aware of the PRNG algorithm
> > > you can keep the code untouched. The new default algorithm
> > > works ok and even better than the OTP 19 with less bugs.
> > > 
> > > On the other hand, if you explicitly specify the algorithm in rand:seed/1,
> > > you are suggested to use a newly-available algorithms
> > > described in the rand module manual of OTP 20.0. Actually
> > > all the old (OTP 18 and 19) algorithms are now deprecated.
> > > See rand module manual of OTP 20. Use exrop (default of OTP 20) in most
> > > cases;
> > > if you use exs1024 that should be changed to exs1024s;
> > > if you stick with exsplus (default until OTP 19), use exsp instead.
> > > If you use other algorithms, consider converting to exrop.
> > > 
> > > Also, if your code depends on the output range of
> > > rand:uniform/1 or rand:uniform_s/1, it has been changed as follows:
> > > OTP 18 and 19: 0.0 < X < 1.0
> > > OTP 20: 0.0 =< X < 1.0 (note the =< operator)
> > > where X is the output of the functions.
> > > In short, since OTP 20, the functions may output 0.0.
> > > 
> > > I noticed the changes shortly after OTP 20.0 release.
> > > These changes were in the last minute (19-MAR-2017 - 26-APR-2017)
> > > just before the OTP 20 release
> > > and after my Erlang and Elixir Factory 2017 trip so I didn't notice.
> > > I was not notified at all by Raimo Niskanen and other contributors
> > > about these changed either,
> > > so I had to take time and chance to review the code again.
> > > I know OTP is open source software and the OTP Team can modify
> > > the code for better features and removing bugs without notifying to anyone,
> > > so nobody is to blame on these changes.
> > > I'm assured that Raimo and the other contributors have done a great job
> > > on changing the details while maintaining the compatibility
> > > and fixing the trivial bugs which I left in the code.
> > > 
> > > I would also like to note that crypto module also utilizes
> > > the rand module API to simplify access to cryptographically strong
> > > random number module since OTP 20. See crypto:rand_seed/{0,1}
> > > and crypto:rand_uniform/2. This is a good example to make use of
> > > rand module's extensible plugins.
> > > 
> > > Thanks to Tuncer Ayaz for giving me a chance to review this.
> > > 
> > > And always remember: if you are still dependent on random module,
> > > migrate to rand module now.
> > > 
> > > Regards,
> > > Kenji Rikitake
> > 
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://erlang.org/mailman/listinfo/erlang-questions
> > 
> > 
> > -- 
> > 
> > / Raimo Niskanen, Erlang/OTP, Ericsson AB
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> 
> -- 
> 
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB