[erlang-questions] Lack of warning for overlapping clauses with

[Torben Hoffmann]

On Wed, Jul 21, 2010 at 01:43, Richard O'Keefe <ok@REDACTED> wrote:

>
> On Jul 21, 2010, at 10:24 AM, Torben Hoffmann wrote:
>
>>
>>
>> Well, I actually ran into a problem with this in a real program and it was
>> by sheer coincidence that I noticed the overlapping clauses, so I would have
>> liked the compiler to spot this.
>>
>
> It would be really really good to see the actual code, if possible.

Not possible, but the general problem was the same only with matching on
fields of the incoming records. I tried the changes in my simplified example
on the real code and the results were exactly the same.


>
>
>> Given the Haskel evidence it seems that the problem is even harder than I
>> imagined at first, so maybe I am expecting too much... expecting too much
>> becomes so natural when you work with Erlang.
>>
>> This means that there are two or more actions to be taken on this:
>> 1. Ask a bright person who is into type checking to look at this in his
>> thesis project.
>>
>
> There is no type problem here; everything is well typed.
>

Well, typing and semantic analysis blends together in my head every now and
then, so the correct assignment would be to explore which practical
extensions one can make to the analysis of function clauses with the
requirement that it should not overly complicate matters.

<nice examples removed from reply>

>
>  2. Change my style of programming so that it stays within the bounds of
>> what the compiler will warn me about.
>>
>
> I have a bad feeling about the solvability of "do these two
> binary patterns have a common instance", so you had better
> avoid binary patterns.  (:-)
>

I wouldn't go as far as that, but in the real code the when-clause was used
to avoid duplicating code since two different values for a specific field
had the same implementation. One could extract the common code into a
function and see how well the code reads after that. I will do that
experiment and see if I like the result.
But you are right about the continued need for programmers - not everything
can be handled by the compilers.


>
> You can't expect a compiler to warn you about every conceivable
> problem.  As a trivial example,
>
>        f() -> m:g(1).
>
> If m:g/1 is not available to the compiler, who knows what it might
> do?  How could the compiler warn you "this function will always
> crash" when it hasn't seen that function?  Worse, suppose the
> compiler *can* see a definition of m:g/1, but at run time it is
> replaced by a different one.  Is the compiler to warn you about
> every remote call
>        Warning: you are calling a function that might be
>        replaced at run time by another definition that might
>        do all sorts of horrible things.
>

I can live with that since it clearly places responsibility outside the
current module - quite different from the original problem.


>
> I suppose we could call this a full employment theorem for
> testers and debuggers:  in any nontrivial programming language
> there will be bugs expressible that no algorithm can
> infallibly detect.  (Note: the Dialyzer does NOT claim to
> infallibly detect bugs.  It makes the weaker claim that the
> things it detects are bugs; it will certainly miss some.)
>
> Keep your style as clear as you can so that other people can
> understand it.  Write specifications and test cases.  (Ulrich
> Neumerkel -- hope I've spelled that right -- has worked on a
> teaching environment for Prolog called GUPU that requires
> students to write test cases first, and tests the tests.  A
> neat idea.)  When you find something that the compiler didn't,
> ask if the compiler can be improved to detect it.  It's an
> economic trade-off:  how likely is the bug, what are the
> expected consequences of not finding it, how costly will it
> be to improve the compiler to that level.
>

Exactly and that is why the thesis proposal above should have this sort of
trade-offs included in the problem statement.
Given the large amount of code written in Erlang, Haskel and the likes there
is plenty of code where the improved static analysis could be tested on to
see how many potential problems it finds compared to the original algorithm.


But you did hit a sore spot here - the code was written without a
comprehensive test suite due to familiar problems such as incomplete
knowledge of one of the protocols and lack of experience with Erlang and in
particular testing (eunit and QuickCheck) when we started the project. It is
always expensive to be poor...


>
>  3. [Optional] Put some explanations/warnings in the Erlang documentation
>> so that people are not led astray like me.
>>
>
> It's not clear precisely what you want people to be warned
> against.  "Expect the compiler to be right but dumb" is about
> all I can think of here.
>

Well, I think that it would be okay to ask for a few examples of how
function clauses using when clauses work together with regards to utilising
one of the greatest strengths of Erlang (and other FP languages): pattern
matching.
I have gazed through the documentation and I have not found anything on the
warnings that will be emitted by the compiler and what it will not detect.
The things not detected should then be on the check list used when doing a
code review since those things is where we still have a chance to make money
as programmers.

Again, this the effort required for this has to be weighed against the
occurrence of the problem and I cannot quite figure out if others have big
enough pains to warrant an investment.

Cheers,
Torben
-- 
http://www.linkedin.com/in/torbenhoffmann