[erlang-questions] Understanding dialyzer errors better

Jesper Louis Andersen jesper.louis.andersen@REDACTED
Thu Mar 2 23:04:44 CET 2017


The dialyzer often reports an error which really occurs somewhere else.
Hence, if given an error which you don't know how to solve, it is often
good to look for other errors which you do know how to solve. In turn,
fixing those errors can clear up the call chain such that other errors
become clearer or get resolved as no-error-in-the-first-place.

The reason it is often confusing is that it is never wrong. Rather, the
dialyzer tends to find a path in your code you never thought of. So you are
thrown off by the structure of your own code which you think you know how
runs. The trick is to throw away assumptions and go chase call chains. And
fix those errors which you know how to fix early on. Once they are done,
you can turn your eyes on the other errors.

Case in point: I did a fix for a single dialyzer warning today. The error
count went from about 43 to 24 on that project. The code looked like:

foo(_, _, null) -> ok;
foo(_, _, X) -> ...CBody...

The dialyzer said that the function wouldn't work because my use differed
from foo's success typing: (any(), any(), null). What the dialyzer is
saying is "there is a bug in CBody, so the function only works if being
passed null". Since foo/3 had many uses all over the code base, they were
reported with the same error.

On Thu, Mar 2, 2017 at 6:39 PM Brujo Benavides <
fernando.benavides@REDACTED> wrote:

Hi Raghav,

I have a private gig at my company: I dialyze projects for beer (true
story).
Let me show you an example of my mental process based on your commit. Let’s
tackle like 197…

It reads:
_ = bitcask_fileops:close_for_writing(WriteFile),
And dialyzer is saying that the call to that function will never return
since *it differs* the first *(and only)* argument can’t be *fresh* nor
*undefined*.
There are 3 possibilities here:
1. bitcask_fileops:close_for_writing/1 actually accepts other kinds of
arguments, but the clauses where those types are accepted are also
questioned by dialyzer so, in dialyzer’s eyes, they can never happen.
2. WriteFile can actually be *fresh* or *undefined*, but dialyzer is fooled
by a misplaced spec somewhere.
3. The code is actually broken (i.e. bitcask_fileops:close_for_writing/1
expects fresh or undefined and WriteFile can never be any of those)

Let’s discard #3… I bet your app is tested with 100% coverage, so that
can’t happen.
Let’s aim for #1. We have to check the
bitcask_fileops:close_for_writing/1’s code. …and *bingo!*
bitcask_fileops:close_for_writing/1
does have a third clause that, instead of *fresh* or *undefined* can
receive a *#filestate{}* record (with some fixed params).

Dialyzer must have produced a warning for that one, too… for sure. We check
your list and… we found no warning in these lines (236-238).
That’s bad, but check it out, that function is calling *close_hintfile/1* and
we do have warnings for that one, specifically:

 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _,
_, _} tagged with a record name violates the declared type of
#filestate{mode::'read_only' | 'read_write' |
'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' |
'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
violates the declared type of field hintfd::port()

Ok, so the function has no local return, that explains all! (o_O)
Anyway, let’s check the last 2, shall we?
On line 240, header of the first clause of the function, we’re using
*undefined* in a record field that doesn’t accept undefined. The warning
can be improved, but it can be roughly translated in this way:
- Matching of pattern {‘filestate’, …} tagged with a record name… => The
pattern with record filestate that you are using…
- violates the declared type of #… => can never match an actual #filestate
record because it’s using at least one sub-pattern that doesn’t match the
type of the corresponding field. Here is the record definition:…

On line 251, where we’re building a filestate record, the warning is much
clearer: Dialyzer says we’re using the wrong value for field *hintfd *which
is defined as *port()*.

So, it’s pretty clear that our intention was to allow hintfd to be
*undefined*.

My next step here would be to fix that (as you did) and run rebar3 dialyzer
again.

We started from line 197… but we could’ve started this process from like
219 as well and reached the same conclusion, right?

I’m not sure if this is what you were actually looking for, but maybe it
helps anyway.

Happy dialyzing! :)


On Mar 2, 2017, at 06:39, Raghav Karol <raghav.karol@REDACTED> wrote:

I spent a bit of time yesterday on some dialyzer warnings resulting from
the singleton type 'undefined' no longer automatically to record fields
types OTP-19. What threw me off were several related but hard to decipher
warnings.

Using OTP-19, `rebar3 dialyzer` on this commit
https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677
produces:

```
src/bitcask.erl
 197: The call
bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' |
'read_write' | 'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
 219: The call
bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' |
'read_write' | 'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
 520: The created fun has no local return
 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will
never return since it differs in the 1st argument from the success typing
arguments: ('fresh' | 'undefined')
 717: The call
bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined'
| integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
 722: The call
bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined'
| integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' |
'read_write' | 'undefined',filename::[any()],tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
1224: The call
bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined'
| port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return
since it differs in the 1st argument from the success typing arguments:
('fresh' | 'undefined')
1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the
type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' |
#filestate{mode::'read_only' | 'read_write' |
'undefined',filename::[any()],tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined'
| reference(),read_files::'undefined' |
[{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' |
integer(),opts::'undefined' | [any()],key_transform::'undefined' |
fun(),keydir::reference(),read_write_p::'undefined' |
integer(),tombstone_version::0 | 2},100,'undefined'>
1870: Function wrap_write_file/1 has no local return
1924: Record construction
#filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0}
violates the declared type of field hintfd::port()

src/bitcask_fileops.erl
 166: Record construction #filestate{mode::'read_only',filename::string() |
#filestate{mode::'read_only' | 'read_write' |
'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0}
violates the declared type of field hintfd::port()
 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _,
_, _} tagged with a record name violates the declared type of
#filestate{mode::'read_only' | 'read_write' |
'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' |
'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
violates the declared type of field hintfd::port()
 314: The pattern 'undefined' can never match the type port()

src/bitcask_merge_delete.erl
 151: Record construction
#filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0}
violates the declared type of field hintfd::port()
===> Warnings written to
/Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings
===> Warnings occured running dialyzer: 17
```

Particularly confusing, are errors like the first one

```
 197: The call
bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' |
'read_write' | 'undefined',filename::string(),tstamp::'undefined' |
integer(),fd::'undefined' |
port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' |
non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()})
will never return since it differs in the 1st argument from the success
typing arguments: ('fresh' | 'undefined')
```

I _think_ dialyzer is saying that the call
`bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not
possible because of type violations when creating `#filestate{}` lower down
the call stack.

Would appreciate if someone could help understand this better and also
suggest how one separates noise from real warnings with dialyzer.

Best,
Raghav

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170302/b67e02f4/attachment.htm>


More information about the erlang-questions mailing list