Zeroization of sensitive data in Erlang

Sun Oct 27 23:42:42 CET 2019

Hi Amit,

Regarding the sensitive flag:

   - I like the idea of only zeroing memory of sensitive processes, it
   sounds like a sensible thing.
   - However, beware that this flag is not used widely enough. For example,
   I found no usage of it in the OTP source code (except for some test cases
   verifying it actually works), although applications like ssl could probably
   make good use of it. Similarly, on GitHub there are only 313 hits
   <https://github.com/search?l=Erlang&q=process_flag+sensitive&type=Code>
   if you search for the flag (with a lot of false positives). So you'd
   definitely have to audit your libraries, such as your webserver, on whether
   they mark all processes with sensitive data as sensitive.
   - For the above reason, I wouldn't worry that much about performance
   impact on existing applications. But just to be on the safe side, you could
   add a command line switch for turning the new feature on, leaving it off by
   default.
   - What you should worry about are things like shared, reference-counted
   binaries. They may be shared between sensitive and non-sensitive processes,
   so do they have to be zeroed or not? What about the memory allocated by
   NIF-s in sensitive processes? Like the openssl data structures where crypto
   will copy the keys? I have a feeling that blindly zeroing every freed up
   memory is inefficient, but doable, while extending the sensitive concept to
   cover every aspect of the BEAM VM may not be feasible at all.

Regarding more fine grained control via marking terms as sensitive instead
of entire processes: I don't think this would be a good idea in practice.

   - The BEAM VM reserves a few low bits in each word of memory for type
   tags <https://blog.stenmans.org/theBeamBook/#_the_tagging_scheme>. To
   mark any term as sensitive, you'd need an unused tag bit, which you don't
   have. There is one unused tag bit combination for boxed types left, so you
   could in theory introduce a new, "sensitive data" term type, but that's not
   feasible to implement. For boxed types on x64, you could also use the top
   16 bits of the pointer for meta-data, as currently the physical addresses
   are only 48 bit wide on this platform. But this also seems unreasonably
   complicated and isn't even portable to other architectures.
   - Even if you could tag each sensitive term, this wouldn't make the GC
   faster. The GC only visits (and copies) live terms. I'm pretty sure that
   reading in every dead term, checking whether they're sensitive and then
   selectively zeroing parts of the memory would be much-much slower in
   practice than just blindly zeroing over the entire memory area. We are
   talking about out of order CPU-s with precious, limited cache space here.
   - What may work instead is to maintain a list of sensitive terms on the
   heap for each process. This is probably not impossible to do (e.g.
   processes already maintain a list of shared, reference-counted binaries
   they use), but it seems impractically complicated.

Daniel

On Sun, 27 Oct 2019 at 17:34, Amit K <klg.amit@REDACTED> wrote:

> Hi Eric,
> I like your suggestion, but would it be a reasonable performance overhead,
> especially for those who are already using the sensitive flag for certain
> processes? If the answer is likely yes I may well try to implement this and
> possibly create a PR for that :)
>
> On Sun, Oct 27, 2019 at 9:21 AM Eric Pailleau <eric.pailleau@REDACTED>
> wrote:
>
>> Hi,
>>
>> Note that there is a sensitive flag for processes.
>>
>> Would be neat that variables used by sensitive processes get their memory
>> zeroed.
>>
>> Envoyé depuis mon mobile
>>
>>
>> ---- Amit K a écrit ----
>>
>> Hi Daniel,
>>
>> Good to know about that workaround, thank you. However as you already
>> pointed out, I really do want to be able to handle such "sensitive"
>> variables in Erlang code and use the standard "crypto" library of the OTP.
>>
>> With regards to GC - that's a great direction I think, and perhaps a more
>> finer grained approach would be to add a new "sensitive" flag for typed
>> variables that the GC can recognize. Then only those variables get zeroized
>> when the GC runs.
>> Would probably be better for performance than always zeroizing everything
>> that we free...
>>
>> BTW - so far I didn't find any FP language that can do something like
>> that, and please correct me if I'm wrong. It would be cool if Erlang would
>> be the first to the gold here (again!) :)
>>
>> On Sun, Oct 27, 2019 at 1:42 AM Dániel Szoboszlay <dszoboszlay@REDACTED>
>> wrote:
>>
>>> Hi Amit,
>>>
>>> A NIF could use a resource object to implement a zeroable memory
>>> location. The Erlang program will only see a handle to the resource, which
>>> is immutable. But the contents of the object can be changed by the NIF
>>> (it's better explained in the docs
>>> <http://erlang.org/doc/man/erl_nif.html>). So you could create an API
>>> that looks like this for the Erlang program:
>>>
>>> Handle = create_secret(Some, Input, Values),
>>> ok = use_secret(Handle),
>>> ok = destroy_secret(Handle), % The NIF zeroes out the memory in the
>>> resource object
>>> {error, destroyed} = use_secret(Handle). % The secret is no more at this
>>> point
>>>
>>> The big problem is that your secret shall never leave the native
>>> library. E.g. you cannot hide a cryptographic key behind a zeroable
>>> resource object and then pass it to the crypto library. As soon as you
>>> convert it into an Erlang term, it is copied onto the non-zeroed stack or
>>> heap or a process. Considering this, you may just as well use a port
>>> program to hold and work with the secret outside of the Erlang VM.
>>>
>>> Probably the only real solution within the VM would be to patch the GC
>>> to zero any reclaimed memory. Or maybe not even the GC, but the memory
>>> allocators, in case a secret is copied to some non-GC-d memory area, such
>>> as an ETS table.
>>>
>>> Cheers,
>>> Daniel
>>>
>>> On Sat, 26 Oct 2019 at 22:39, Amit K <klg.amit@REDACTED> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Concerning the security practice of Zeroizing sensitive data from
>>>> memory after you're finished with it, such as Cryptographic keys,
>>>> passwords, etc, I wanted to ask if any of you ever had to implement such a
>>>> thing in Erlang, and if so, how? :)
>>>>
>>>> Also note that often this is a requirement that you must fulfill if you
>>>> want your product to pass a security standard evaluation such as the
>>>> "Common Criteria" one (As an example requirement see FCS_CKM_EXT.4 here:
>>>> https://www.commoncriteriaportal.org/files/ppfiles/pp_nd_v1.0.pdf).
>>>>
>>>> The problem of course is that in Erlang (and I suppose - FP in general)
>>>> once a variable gets assigned, you are not allowed to change its value, and
>>>> in particular of course overwrite it with zero bytes.
>>>>
>>>> One option I thought about is doing it with a NIF, but from my
>>>> understanding a NIF isn't supposed to change its input values, it should
>>>> treat them as constants, which understandable as well (basic FP language
>>>> property).
>>>>
>>>> Any feedback can be helpful :)
>>>>
>>>> Regards,
>>>>
>>>> Amit
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20191027/28eadbfb/attachment.htm>