[erlang-patches] Win64 memory corruption fix

[Sverker Eriksson]

We have to reject this patch in its current state.

Type Uint is not same size as a pointer on halfword emulator. You should 
have used type UWord instead for a pointer-sized integer.

And to printf pointer variables I think we should use %p and pass the 
pointers without the need of any integer type casts.

/Sverker, Erlang/OTP


Fredrik wrote:
> Hello,
> I will put it in the 'master-pu' branch for building and testing!
>
> BR Fredrik Gustafsson (Terran player)
> Erlang OTP Team
> On 02/06/2013 04:27 AM, Blaine Whittle wrote:
>>
>> git fetch git://github.com/bwhittle/otp.git win-64-pointer-fix
>>
>> https://github.com/bwhittle/otp/commit/1d21ce8d0287a0a50b2e42631d361f43ce14e23e 
>>
>>
>> This patch should fix a number of memory corruption issues and / or 
>> crashes on Win64 that can potential occur when the Erlang VM exceeds 
>> 4 GB of ram.    The problem stems from casting pointers to unsigned 
>> long and assuming long is type that is always large enough to hold a 
>> pointer.  This assumption holds up for all platforms except windows.
>>
>> Nix 32 (unsigned long) -> 32 bit                  (pointer size = 
>> unsigned long)
>>
>> Nix 64 (unsigned long) -> 64 bit                  (pointer size = 
>> unsigned long)
>>
>> Win 32 (unsigned long) -> 32 bit                (pointer size = 
>> unsigned long)
>>
>> Win 64 (unsigned long) -> 32 bit                (pointer size != 
>> unsigned long)
>>
>> To compound the problem these casts can appear to be fine on Win64 as 
>> only those pointers that reference memory above the 32 bit address 
>> space will lead to issues.    Which means you need Erlang to allocate 
>> ~ 4 GB of memory before you even have a chance or running into 
>> problems.     The issue is if you have a pointer that reference 
>> memory above the 32 bit address space on Win64 and then type cast it 
>> to a long (i.e. 32 bits) and then turn around and use that type cast 
>> value as a pointer then you'll be referencing a different memory 
>> location.   Most of the time the incorrect pointer will still 
>> reference a valid location as memory is allocated bottom up which can 
>> lead to memory corruption.
>>
>> This patch has been tested heavily and has been used on production 
>> systems.  I made the changes a year ago when the Win64 Erlang VM was 
>> released (just didn't mean to wait so long to submit it.)
>>
>> The patch submission page recommends that I create new test cases 
>> which I have not done.    However I have a small registry change that 
>> should be applied on any systems that execute Erlang Win64 smoke and 
>> / or unit tests.
>>
>> The registry change instructs Windows to allocate memory from top 
>> down, meaning that any valid memory pointers will require a 64 bit 
>> value and any attempt to cast them to a 32 bit value followed by a 
>> dereference will produce an access violation.
>>
>> http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=vs.85).aspx 
>>
>>
>> To apply the registry setting just copy and paste the following 
>> section and place it into a <some name>.reg file and import it on 
>> each test machines followed by a reboot.
>>
>> =============================================================
>>
>> Windows Registry Editor Version 5.00
>>
>> [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session 
>> Manager\Memory Management]
>>
>> "AllocationPreference"=dword:00100000.
>>
>> =============================================================
>>
>> With this registry change Erlang's existing unit tests should be able 
>> to catch any incorrect pointer casts by causing the VM to crash.   
>> Every pointer will reference memory above the 32 value so type 
>> casting it to a 32 bit value and then dereferencing causes an access 
>> violation.
>>
>> One potential issue with using this registry setting is that if your 
>> test machines rely on 3rd party Win64 apps it's possible they may 
>> crash on startup (that is if they contain similar type casting bugs.)
>>
>>
>>
>> _______________________________________________
>> erlang-patches mailing list
>> erlang-patches@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-patches
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches
>