[erlang-questions] pattern matching vs proplists vs dict performance

[David Mercer]

On Thursday, May 05, 2011, Mikhail Sobolev wrote:

> We have 10s of thousands of documents that require this kind of
> conversion and this conversion has to be done fast.  So I decided to
> write a little benchmark[1], which gave the definitive answer: pattern
> matching is much faster (times on the test machine):
>   * proplists -- ~4.9s
>   * dicts -- ~3.0s
>   * case inside the function -- ~0.86s
>   * case as a separate function -- ~0.89s
> 
> We have discussed the results and started to wonder:
> 
> 1) are we doing something wrong? (we are all newbies in here)

I don't think so.  I'd expect pattern matching in a case or function to be
faster than iterating through a data structure, regardless of whether the
data structure is a proplist or dict.

> 2) why there's such a big difference in time? (proplists module
> provides BIFs, right?)

Again, I don't think so.  I consider proplists to be Erlang-only, and I'd be
surprised if it were implemented in C.  Regardless, I'd still expect pattern
matching to be faster.

> 3) are there any other ways to do what we want?

You could look them up in ETS tables.  ETS *is* in written in C, so it
*might* be faster than proplists and dicts, but I still wouldn't expect it
to be faster than pattern-matching.  Benchmark to prove me right or wrong.

> The current inclination is to generate a function based on the mapping
> and then utilize the produced function to do the job.

If 3-5s is unacceptable, but 0.9s is OK, that's the way I'd go, so long as
you're not changing the atom-to-string mapping very often (requiring
regeneration/recompilation of the function).  If 3-5s is OK, however, I'd
concentrate on other problems.

Cheers,

DBM