[erlang-questions] Bit syntax matching gotchas
Richard A. O'Keefe
Wed Feb 10 06:54:50 CET 2016
On 3/02/16 7:17 pm, Björn Gustavsson wrote:
> There are some gotchas in the bit syntax that comes up now and then on
> the mailing list, and also as a bug report at the end of last year:
I am sorry to be so late replying to this, but things have been (a)
> BACKGROUND ABOUT BIT SYNTAX CONSTRUCTION When constructing binaries,
> there is an implicit masking of the values. All of the following
> constructions give the same result: <<255>> <<16#FF>> <<16#FFFF>> <<-1>>
That always felt so WRONG to me.
If I *want* masking, I can write <<X band 16#FF>>.
I really do not understand why this kind of thing was ever
3> <<X>> = <<X>>.
** exception error: no match of right hand side value <<"ÿ">>
If you stop doing the masking (except where there is an explicit
'band' licensing it) then
(a) some modules will break
(b) the fix will be easy
(c) the fixed code will be backwards portable
> There have been complaints about the implicit masking behaviour, but
> there is a lot of code that depends on it, so it would be unwise to
> change it.
I don't buy that, sorry.
Make a new option:
allowed once per module. dangerous is the current behaviour,
safe raises an exception for overflow.
In R19, introduce it, with dangerous as the default.
In R20, make the compiler issue a warning if a module has a bit syntax
form that would be ambiguous and doesn't state the desired behaviour.
In R21, make safe the default.
In R22, make the compiler issue a warning if a module has
In R23, remove the option.
This process *FIXES* the problem.
Instead of safe/dangerous, consistent/inconsistent might be better.
> POSSIBLE SOLUTION #1 The most obvious solution is probably to let the
> compiler warn for the above cases. The matching would still fail. The
> developer will need to fix their code. For example: <<-1/signed>> =
> <<-1>> POSSIBLE SOLUTION #2 There is one problem with the solution #1.
> It is not possible to produce a warning for the following example:
> f(Val) -> <<Val:8>> = <<Val:8>>, Val.
Then solution #1 is not a solution. But why isn't it possible?
The problem arises whenever you have <<...X:N...>> and
the compiler has no proof that X fits in N bits. This is just
such a case.
> The proble So in addition to warning when possible, another solution
> is to mask values also when matching. Internally, the compiler could
> rewrite the function to something like: f(Val) -> <<NewVar:8>> =
> <<Val:8>>, Val = NewVar band 16#FF, Val. Similar rewriting should be
> done for literal integer, so the following expression would now match:
> <<-1>> = <<-1>>
What exactly is the rewriting?
0> <<NuVar>> = <<-1>>,
10> -1 = NuVar band 255.
** exception error: no match of right hand side value 255
That's not a match.
> Solution #1 would point all cases when literal integers could not
> possibly match and force the developer to fix them. Therefore I choose
> solution #1.
The problem is not a problem with literal integers.
It is a problem with ANYTHING that doesn't fit being quietly
truncated in one context but not the other.
Anything that leaves matching and building contexts
inconsistent is not a solution. If a match cannot
*return* a value in a certain place, the corresponding
build should not *accept* that value in that place.
The idea that this can't be fixed because there is code
depending on the old behaviour reminds me of two famous
(C) C's switch statement is a botch, with 'break' being
used both for "exit this loop" and "exit this switch".
This was fixed in C's grandparent BCPL where 'break'
was used for "exit this loop" only and 'endcase' was
used for "exit this switch only". C's designers were
aware of that correction to BCPL, but felt they could
not adopt it because they "already had 100 users".
(X) Excel notoriously gets its leap year calculations wrong
but Microsoft have announced that this feature will not
be changed in case there are users who depend on the bug.
I'm also reminded of a bug that was squashed.
(B) Burroughs Algol has pointers. That made it possible for
you to write
EBCDIC POINTER P;
EBCDIC ARRAY A[0:79];
P := A;
and then use P. They didn't notice the problem for a while.
They then made a temporary fix for the operating system so
that when you exited a block that declared an array, the OS
would sweep your stack looking for pointers to it. They also
made an addition to the language, so that you could write
EBCDIC ARRAY B[0:131];
EBCDIC POINTER P FOR B;
EBCDIC ARRAY A[0:79];
P := A; %% This is a syntax error!
Customers were given about a year to fix their programs,
then warnings were turned into hard errors and the OS patch
was removed, and presto chango, absence of dangling pointers
proved by the compilers. Burroughs Algol is still available
-- I have a personal copy of the MCP and software that runs
in a VM on a Windows box; now all I need is a Windows box --
and for 30 years that class of bugs has been GONE.
people are *still* making 'break' mistakes, all because people chose
to perpetuate old bugs rather than eliminate them.
The longer any match/build inconsistency is allowed to remain
in bit syntax, the more pain it will cause.
More information about the erlang-questions