[erlang-bugs] a crypto function call crashes VM

[Michael Santos]

On Thu, Aug 25, 2011 at 06:58:21PM +0200, Erik Søe Sørensen wrote:
> Simpler version:
>   crypto:sha_update(list_to_binary(lists:seq(1,96)).

Maybe even simpler:

1> crypto:sha_update(<<0:(95*8), 0>>, "x").
<<0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,
  120,...>>

2> crypto:sha_update(<<0:(95*8), 1>>, "x").
Segmentation fault 

The crypto NIF is crashing when doing a SHA_update() (crypto.c:502).

Looks like an OpenSSL bug.  "1" is set in the num field of the SHA_CTX
struct.

This segfaults too:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/sha.h>

int
main (int argc, char *argv[])
{
    SHA_CTX ctx;
    u_char zero[sizeof(ctx)];
    u_char buf[sizeof(ctx)];

    (void)memset(&ctx, 0, sizeof(ctx));
    (void)memset(zero, 0, sizeof(zero));

    SHA_Init(&ctx);
    (void)memset(&ctx, 1, sizeof (ctx));
    SHA_Update(&ctx, zero, sizeof(zero));
    SHA_Final(buf, &ctx);

    return (0);
}


> Boundary:
> 6> crypto:sha_update(list_to_binary([0 || _ <- lists:seq(1,96)]), "x").    
> <<0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,
>   120,...>>
> 7> crypto:sha_update(list_to_binary([1 || _ <- lists:seq(1,96)]), "x").
> Segmentation fault
> 
> (Probably part of the binary is interpreted as a pointer, except when NULL.)
> 
> ________________________________________
> Fra: erlang-bugs-bounces@REDACTED [erlang-bugs-bounces@REDACTED] På vegne af Kenji Rikitake [kenji.rikitake@REDACTED]
> Sendt: 25. august 2011 18:37
> Til: erlang-bugs@REDACTED
> Emne: [erlang-bugs] a crypto function call crashes VM
> 
> Enter the following code to erl shell:
> 
> crypto:sha_update(
>   list_to_binary(lists:reverse(binary_to_list(crypto:sha_init()))), "hello").
> 
> Segmentation fault occurs and VM crashed with the core dump.
> 
> This is originally reported at
> https://twitter.com/#!/szktty/status/106761685284564992
> by Tetsuya Suzuki (Twitter: @szktty)
> (Tweet in Japanese but the code is the same as shown here)
> 
> Verified on
> R14B03 running on FreeBSD 8.2-RELEASE x86
> and R14B03 running on RHEL v4 x86_64
> for Kyoto University Supercomputer Thin Cluster.
> 
> Kenji Rikitake
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs