[erlang-questions] pre-load large data files when the application start

[Benoit Chesneau]

On Wed, Mar 30, 2016 at 3:16 AM Richard A. O'Keefe <ok@REDACTED>
wrote:

> I notice a lot of lines like
>
> ,"0001" => { "0", "", ""}
> ,"0002" => { "0", "", ""}
> ,"0003" => { "0", "", ""}
> ,"0004" => { "0", "", ""}
> ,"0005" => { "0", "", ""}
> ,"0006" => { "0", "", ""}
> ,"0007" => { "0", "", ""}
>
> I just wonder whether factoring {"0","",""} out
> would help at all.
>
> I also notice that the lines are all
>      "<hex>" => {"<dec>", "<stuff>", "<hex>"}
> and wonder whether having three maps
>     <int> => "<dec>"
>     <int> => "<stuff>"
>     <int> => "<hex>"
>

So I changed the code to use integers everywhere.  It makes the processing
a lot faster:
https://github.com/benoitc/erlang-idna/commit/a40fedb1ac2933a4d668b4813bf2cfab408ef6a4

For now the code is creating one module with pattern matching and still has
the tuple. Since the size of the file is smaller, it compile a little
faster and use less memory.



> and then
>      lookup(Codepoint) ->
>          N = list_to_integer(Codepoint, 16),
>          A = maps:get(N, ?BY_CODE_TO_DEC,   false),
>          B = maps:get(N, ?BY_CODE_TO_STUFF, false),
>          C = maps:get(N, ?BY_CODE_TO_HEX,   false),
>          if A == false -> false
>           ; true -> {A, B, C}
>          end.
> might give the compiler less trouble.
>
> I mean, it seems a bit odd to do
>
> lookup(Codepoint) ->
>      idna_unicode_data:lookup(hex(Codepoint)).
>
> when you could do
>
> lookup(Codepoint) ->
>      idna_unicode_data:lookup(Codepoint).
>
> In fact, since different functions in idna_unicode.erl
> are interested in different bits of the triples, I
> don't understand why these weren't separate maps to
> begin with.
>
>
Do you mean having different functions in the same module using a map each?
What is the cost of using a map compared to use function pattern-matching
if the code is called by different processes?

- benoit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160331/319eb82e/attachment.htm>