[erlang-questions] dialyzer maybe_improper_list usage and possible internal error

[Tobias Lindahl]

Hi Bill,

Foolish Ewe wrote:
...

> -spec(compute_generator/3 :: (Q :: pos_integer(), P ::
> maybe_improper_list(pos_integer(), pos_integer()), T ::
> non_neg_integer())-> pos_integer()).
> 
> 
> compute_generator( Q, P, T ) when (is_integer(Q) and is_list(P) and is_integer(T)) andalso ((Q > 0) and (T >= 0)) ->
> 
> Such a spec crashes the dialyzer, (I was using R12B3.22.1  on OpenSuse 11.0 at the time):

...

> 
> dialyzer: Internal problems were encountered in the analysis.
> As per the discussion by the very helpful Lestat and Pergu (actually Pergu gave this fix for the problem) on the forum, the spec should have had [] as an alternative in the second parameter to maybe_improper_list, so the spec should have read.
> -spec(compute_generator/3 :: (Q :: pos_integer(), P ::
> maybe_improper_list(pos_integer(), pos_integer() | []), T ::
> non_neg_integer())-> pos_integer()).
> 
> 
> 
> compute_generator( Q, P, T ) when (is_integer(Q) and is_list(P) and is_integer(T)) andalso ((Q > 0) and (T >= 0)) ->
> I've also sent copies to the folks I think wrote/maintain the dialyzer, so please forgive me if you get this message twice.
> 
> Having given it a bit of consideration, I think there are a few issues to consider:
> 1) The dialyzer should complain about the lame spec I made, but should not crash.

I agree. it should be handled more gracefully. What happened is that an 
assert kicked in because of a malformed type. See below for a more 
precise explanation.

> 2) However, the real problem is why did the dialyzer think that
> function used an improper_list (or maybe_improper_list)?  I'd like to
> figure out why, perhaps I have some error I can't see myself.

You use is_list(P) which only checks that P is either [] or [_|_] 
without looking at the termination. Dialyzer cannot say anything about 
the termination and gives the type maybe_improper_list().

The lists are a bit obscure in the internal representation, and some of 
the types are not really intended to be used in specs.

As Pergu pointed out to you, the termination part of a 
maybe_improper_list(X, Y) must be a supertype of [], or it would be an 
improper_list(X, Y). If the termination does not contain [] then the 
list cannot be a proper list, so the maybe_ is dropped.

Of course, this is nothing that a spec-writer should have to worry 
about, or at least get a friendly warning about rather than a crash.

> 3) I suspect my code might be stressing the dialyzer and causing it
> to have unexpected behavior (although I'm pretty reluctant to claim
> my code is anywhere near bug free).  In general, figuring out what
> the dialyzer messages really mean is sometimes tricky, and in some
> cases, I suspect the dialyzer messages do not indicate what is really
> going on.  Some static analysis tools give the values that govern
> flow of control and show just how an unexpected state can be entered,
> perhaps the dialyzer could do that as well.  Perhaps I'll post some
> examples in separate e-mails if there is sufficient interest in the
> mailing list (or if the authors are willing to take a look, perhaps
> we can see if my code is stressing the dialyzer in unexpected ways or
> perhaps my code is indeed wrong.

I am aware that sometimes the cause of the warnings are very hard to 
find. In rare cases you even need to know intricate details of the 
analysis to decipher them ;-)

If you have doubts about the warnings, just send a (preferably minimal) 
test case to me and Kostis and we can help you.

> 4) To increase the impact of a nice tool like the dialyzer, we might
> be able to author a sort of howto guide (or a wiki) to give users
> guidance in how to interpret and fix problems reported by the
> dialyzer.  E.g. I've noticed is that if a "low level" function is
> invoked by  higher level functions and they all have dialyzer errors,
> fixing the low level function will correct/change errors reported in
> thei higher level functions, so fixing those first makes sense.

Yes, this is the way to do it. Start at leaf cases in the code and fix 
the warnings about those functions, and then work your way upwards to 
the interface functions.

Indeed, there should be a howto on interpreting warnings. Unfortunately, 
I have little time to spend on such things nowadays.


> 5) If I understand, the dialyzer generates a data flow graph.  Could
> the dialyzer emit a dot version of the data flow graph suitable for
> importation into graphviz, it might be interesting to visualize that.

If you're willing to hack a bit, look at the end of the file 
lib/dialyzer/src/dialyzer_callgraph.erl where there is a debug hack to 
output the callgraph in dot format. There might be something more 
official in the future, but don't hold your breath...

Best,
Tobias

> 
> 
> Regards:
> 
> Bill M.
> 
> _________________________________________________________________
> Stay up to date on your PC, the Web, and your mobile phone with Windows Live
> http://clk.atdmt.com/MRT/go/119462413/direct/01/