[erlang-questions] Debugging float math in geohashing module

[Bob Ippolito]

I'm pretty sure your problem is the precision of io_lib:fwrite("~f",
[N]). Don't use that, it only prints 6 digits of precision.

On Mon, Jun 2, 2008 at 10:52 PM, Nick Gerakines <nick@REDACTED> wrote:
> I've taken an interest in the concept of the geohash [1] and wrote a simple
> module [2] to encode/decode latitude/longitude pairs into these geohashes.
> The modules is loosely based on Geo::Hash [3] but with a number of obvious
> changes.
>
> So far encoding works perfectly and the same test cases from the Geo::Hash
> perl module pass in my module for encode operations.
>
> The decode functionality is different however. When I attempt to decode a
> geohash into a latitude/longitude pair the end result is slightly* off. When
> I say slightly off I mean that the float precision is incorrect from the
> latitude/longitude that was initially given. I think it has something to do
> with the way the floats are being divided and rounded in the process. I'm
> really having a rough time working around this, any help is welcome. I've
> tried (unsuccessfully) to work around it using big int tuples ({N, E}) but
> that didn't go so well.
>
> [1] http://en.wikipedia.org/wiki/Geohash
> [2] http://github.com/ngerakines/erlang_geohash/tree/master/geohash.erl
> [3] http://search.cpan.org/~andya/Geo-Hash/lib/Geo/Hash.pm
>
> # Nick Gerakines
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>