[erlang-questions] Scaling bcrypt

[Richard Jonas]

Hey guys,

last time I wrote about some problems how bcrypt (at least current erlang
bcrypt implementations) cannot be scale. The original problem what that
bcrypt driver (github chef/erlang-bcrypt) is using port and nif
implementation. The nif implementation behind the scenes it uses a
gen_server which serializes all the requests are coming. The port
implementation uses a port pool which can be configured.

Our problem comes from the fact that a good hashing algorithm should be
slow by design. bcrypt uses a round parameter (3-16) with which one can say
that bcrypt should use 2^(3-16) cycles during computing. According to
security guidelines a good hashing algorithm should take 0.1-0.5 seconds to
be computed, so now bcrypt should run with round 10-12 depending on the
hardware. It needs to be slow in order that one cannot find a
hash-collision easily.

Now you can imagine the problem. With load tests we easily made message
queues and the problem is that the computing has sometimes very very big
latencies (50-60 seconds depending on the number of concurrent users). The
nif implementation spoils one scheduler thread but if I have 32 cores I
cannot scale the computing.

The problem I wanted to solve is to split the bcrypt_hashpw nif into 3
parts: initialization, computing some predefined number of rounds,
encryption. I used enif_schedule_nif to schedule the computing of the next
task.

https://github.com/jonasrichard/erlang-bcrypt/blob/master/c_src/bcrypt_nif.c

The implementation is based on chef/erlang-bcrypt and vinoski/bitwise. The
first loadtests are promising since latencies are much more even. With 10
concurrent users (basho bench + bcrypt driver) the latency was (500ms
1100ms / mean median), with 50 users (2800ms ~6000ms) which 5x increase,
let us say.

My questions are:
- how can I measure if my nif "slices" took no longer than 1ms
- how can I implement some kind of adaptive computing? My bcrypt_compute
computes 64 rounds which sometimes below 1ms, sometimes above 1ms.

https://github.com/jonasrichard/erlang-bcrypt/blob/bb96149332c8c6b6135698dfbc21e19e554bc066/c_src/bcrypt_nif.c#L133

If I use some kernel gettime function, can it cause blocking?

- how can I be sure that no garbage left in memory. I try to watch out
putting release functions in correct places but it would be good to measure.

Thanks,
Richard

-- 
Richard Jonas
Erlang Solutions Hungary Kft

Address:
  Riverpark Office K.32
  Közraktár street 32. 3/1.
  1093 Budapest
  Hungary
Phone/fax:
  +36-1-7000-654
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160706/e809b70c/attachment.htm>