[erlang-questions] casting float to integer

[Bob Ippolito]

Floating point math happened. The float precision of the Erlang shell
is very misleading.

1> io:format("~.21g~n", [(math:pow(2,52) - 1) / math:pow(2,53)]).
0.499999999999999888978
ok
2> io:format("~.21g~n", [((math:pow(2,52) - 1) / math:pow(2,53)) + 1/2]).
0.999999999999999888978
ok
3> io:format("~.21g~n", [(math:pow(2,53) - 1) / math:pow(2,54)]).
0.499999999999999944489
ok
4> io:format("~.21g~n", [((math:pow(2,53) - 1) / math:pow(2,54)) + 1/2]).
1.00000000000000000000
ok

This is what you should expect from any language that uses floating point math.

Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import division
>>> (2**52 - 1) / (2**53)
0.49999999999999989
>>> (2**53 - 1) / (2**54)
0.49999999999999994
>>> ((2**52 - 1) / (2**53)) + (1/2)
0.99999999999999989
>>> ((2**53 - 1) / (2**54)) + (1/2)
1.0

You might want to read up on how IEEE-754 doubles (64-bit floating
point) work. It's the de facto standard representation for a "float"
type in most languages (except the ones where you can choose "float"
or "double", where "float" is a 32-bit single).

-bob

On 5/14/07, Nitin Verma <nitin.matrix@REDACTED> wrote:
> Thanx for all the replies, appreciate it. I am very new to functional
> programming, and to Erlang. This all helped me a lot.
>
> I saw something very weird.
>
> ( 2^(n-1) – 1 ) / 2^n + 1/2 < 1
> ---
>
> 1> Tc=(math:pow(2,52) - 1) / math:pow(2,53).
> 0.500000
> 2> Td= Tc + 1/2.
> 1.000000
> 3> trunc(Td).
> 0
>
> This is correct.
> ---
>
> 1> Tc=(math:pow(2,53) - 1) / math:pow(2,54).
> 0.500000
> 2> Td=Tc + 1/2.
> 1.00000
> 3> trunc(Td).
> 1
>
> eeh, what happened here?
>
>
>
> On 5/12/07, Per Hedeland <per@REDACTED> wrote:
> > Robert Virding <robert.virding@REDACTED> wrote:
> > >
> > >Can't you instead just have:
> > >
> > >floor(V) when V >= 0 -> trunc(V);
> > >floor(V) -> - trunc(-V) - 1.
> >
> > No, try floor(-1.0).
> >
> > --Per
> >
> > >Robert
> > >
> > >Per Hedeland wrote:
> > >> Håkan Stenholm <hokan.stenholm@REDACTED> wrote:
> > >>> Nitin Verma wrote:
> > >>>> Hi All,
> > >>>>
> > >>>> I am looking for a function/BIF that can help me.
> > >>>>
> > >>>> Problem: I need a float number's floored integer. That is if:
> > >>>> A = f(5/2), then A = 2 and
> > >>>> A = f(1000/501), then A = 1
> > >>>>
> > >>>> I found 'round/1' but it just rounds-off .
> > >>
> > >>> see erlang.erl documentation:
> > >>>
> > >>> -----------------------------------------------------------------------
> > >>>
> > >>> *|trunc(Number) -> int()|*
> > >>
> > >> But trunc is not the same as floor - though maybe Nitin actually wants
> > >> trunc, or even div (which "truncates" the same way as trunc), as
> > >> indicated by the examples. When I actually needed floor/1, I came up
> > >> with this somewhat bizarre-looking definition:
> > >>
> > >> floor(V) when V >= 0 -> trunc(V);
> > >> floor(V) -> trunc(V - trunc(V) + 1) + trunc(V) - 1.
> > >>
> > >> There may be a better way to do it...
> > >>
> > >> --Per Hedeland
> >
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>