[erlang-bugs] R16B02: Dialyzer consumes too much, if not infinite, amount of memory for some apps

[Masatake Daimon]

Hello,

I'm using R16B02 on 64-bit Linux. When building PLT, Dialyzer consumes 
too much memory that even reaches to 1.5 GiB for some applications:
> % (ulimit -v 1572864; dialyzer --build_plt --output_plt syntax_tools.plt --statistics --apps syntax_tools)
>   Creating PLT syntax_tools.plt ...
>     compile    (+0.02s):   1.10s (   9 modules)
>     clean      (+0.00s):   0.02s
>     remote     (+0.05s):   1.56s
>     order      (+0.01s):   0.06s
>     typesig    (+0.00s):
> Crash dump was written to: erl_crash.dump
> eheap_alloc: Cannot allocate 123210440 bytes of memory (of type "heap").

Prologue of the crash dump:
> =erl_crash_dump:0.2
> Fri Nov 15 17:07:48 2013
> Slogan: eheap_alloc: Cannot allocate 123210440 bytes of memory (of type "heap").
> System version: Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]
> Compiled: Thu Nov 14 19:14:45 2013
> Taints:
> Atoms: 12021
> =memory
> total: 722308232
> processes: 498144488
> processes_used: 497536304
> system: 224163744
> atom: 339441
> atom_used: 322213
> binary: 607984
> code: 6600866
> ets: 213161040

The process with the largest heap (259 MiB):
> =proc:<0.283.0>
> State: Garbing
> Spawned as: erlang:apply/2
> Last scheduled in for: erl_types:t_has_var/1
> Spawned by: <0.31.0>
> Started: Fri Nov 15 17:07:38 2013
> Message queue length: 0
> Number of heap fragments: 0
> Heap fragment data: 0
> Link list: [<0.31.0>]
> Reductions: 55265641
> Stack+heap: 15401305
> OldHeap: 18481566
> Heap unused: 220
> OldHeap unused: 18481566
> Memory: 271063808
> Program counter: 0x00002b0ab308ba08 (erl_types:t_has_var/1 + 8)
> CP: 0x0000000000000000 (invalid)

Second-largest (192 MiB):
> =proc:<0.347.0>
> State: Scheduled
> Spawned as: erlang:apply/2
> Current call: erl_types:'-t_limit_k/2-lc$^0/1-2-'/2
> Spawned by: <0.31.0>
> Started: Fri Nov 15 17:07:38 2013
> Message queue length: 0
> Number of heap fragments: 0
> Heap fragment data: 0
> Link list: [<0.31.0>]
> Reductions: 33155719
> Stack+heap: 2984878
> OldHeap: 22177879
> Heap unused: 2071396
> OldHeap unused: 8568663
> Memory: 201302904
> Program counter: 0x00002b0ab30b7e78 (erl_types:'-t_limit_k/2-lc$^0/1-2-'/2 + 8)
> CP: 0x00002b0ab30b7f90 (erl_types:'-t_limit_k/2-lc$^0/1-2-'/2 + 288)
> arity = 2

Third-largest (Only 10 MiB)
> =proc:<0.31.0>
> State: Waiting
> Spawned as: erlang:apply/2
> Spawned by: <0.29.0>
> Started: Fri Nov 15 17:07:36 2013
> Message queue length: 10512
> Number of heap fragments: 0
> Heap fragment data: 0
> Link list: [<0.29.0>, <0.32.0>, <0.246.0>, <0.248.0>, <0.249.0>,<0.247.0>, <0.52.0>, <0.283.0>, <0.284.0>, <0.347.0>, <0.349.0>, <0.348.0>, <0.285.0>, <0.256.0>, <0.383.0>, <0.414.0>, <0.400.0>, <0.384.0>, <0.534.0>, <0.537.0>, <0.535.0>, <0.549.0>, <0.551.0>, <0.550.0>, <0.538.0>, <0.444.0>, <0.350.0>, <0.588.0>, <0.592.0>, <0.589.0>, <0.595.0>, <0.608.0>, <0.597.0>, <0.593.0>, <0.613.0>, <0.618.0>, <0.624.0>, <0.623.0>, <0.614.0>, <0.609.0>, <0.626.0>, <0.628.0>, <0.627.0>, <0.630.0>, <0.633.0>, <0.636.0>, <0.631.0>, <0.629.0>, <0.667.0>, <0.666.0>, <0.678.0>, <0.673.0>, <0.668.0>, <0.655.0>, <0.625.0>, <0.552.0>, <0.693.0>, <0.699.0>, <0.695.0>, <0.711.0>, <0.709.0>, <0.703.0>, <0.714.0>, <0.716.0>, <0.715.0>, <0.720.0>, <0.719.0>, <0.717.0>, <0.712.0>, <0.722.0>, <0.726.0>, <0.727.0>, <0.725.0>, <0.732.0>, <0.746.0>, <0.735.0>, <0.729.0>, <0.721.0>, <0.756.0>, <0.755.0>, <0.758.0>, <0.759.0>, <0.757.0>, <0.772.0>, <0.770.0>, <0.776.0>, <0.774.0>, <0.773.0>, <0.768.0>, <0.779.0>,
 <0.784.0>, <0.781.0>, <0.790.0>, <0.795.0>, <0.793.0>, <0.785.0>, <0.777.0>, <0.830.0>, <0.822.0>, <0.835.0>, <0.833.0>,<0.856.0>, <0.868.0>, <0.866.0>, <0.865.0>, <0.874.0>, <0.873.0>, <0.869.0>, <0.841.0>, <0.816.0>, <0.747.0>, <0.680.0>]
> Reductions: 5876487
> Stack+heap: 318187
> OldHeap: 999631
> Heap unused: 172595
> OldHeap unused: 722724
> Memory: 10884200
> Program counter: 0x00002b0aaf561be0 (dialyzer_coordinator:collect_result/1 + 208)
> CP: 0x0000000000000000 (invalid)
> arity = 0

So I suspected either erl_types:t_has_var/1 or erl_types:t_limit_k/2 (or 
both) are entering into an infinite recursion but I couldn't find a 
small example code to reproduce the problem. Any ideas?

Thanks,
Masatake Daimon