[erlang-questions] The Erlang Rationale

[Richard O'Keefe]

On 2 Oct 2008, at 7:55 pm, Edwin Fine wrote:

> Sorry, but I did not agree.

You agreed that macros *CAN* make maintenance harder.
That's enough agreement for me.

>   You just read it that way. What I perhaps should have written is  
> that you bluntly asserted that macros make maintenance harder. I am  
> bluntly asserting that it is not necessarily so.

My claim is that macros *always* makes it harder
to build intelligent maintenance support tools.
>
> just like gotos *can* create spaghetti code. Neither of them are  
> intrinsically bad, merely easy to misuse.

The existence of essentially uncontrolled gotos in a language
makes optimising compilation harder; the task of building a
compiler for such a language is more difficult *whether the
feature is used or not*.  In the same way, the *possibility*
of macros makes it harder to build support tools, whether they
are used or not, no matter what degree of discipline might be
involved.

Of course Erlang still isn't as weird as C, where
	/??\
	* this *??\
	/// was a comment

As it happens, I straddle the fence on gotos.  I have a compiler
for a language with no gotos (and in which I never miss them);
the compiler generates C with gotos (where they are used in a
particular stylised way to avoid code duplication).  But I
maintain a version of AWK, and one of the things I did in that
was to hunt down the gotos one after another.  In fact, I'm
still doing it.  While writing this message I just zapped five
more.  (They were there for a micro-optimisation that made
sense 30 years ago, but is now routinely done by compilers.
The code is *definitely* easier to understand without them.)

> I don't think this is a particularly compelling argument. I've been  
> involved in maintenance of multi-million line systems and had much  
> bigger fish to fry than macro-related difficulties with cross- 
> reference tools. Maybe your experience has been different.

I have repeatedly encountered serious difficulties in navigating
other people's macro-heavy code.

> Used with care and discipline, they both arguably have a place in  
> good programming practice.

Remember, MY "care and discipline" is YOUR "spaghetti code".
[I was going to say it the other way around, but decided that
might sound insulting.]

I have chunks of C code generated by AWK scripts.
(There's a Haskell compiler written in C where large chunks
were generated by Perl scripts.)
I happen to think this is not only cool, but counts as
"care and discipline".  Someone whose IDE hasn't the slightest
clue and can't really cope with such stuff (XCode certainly
isn't much help with colouring C code inside AWK strings (:-))
will not be impressed.
>
>
> As a matter of fact, I still use M4 on Java code.
> (Remember, Java 1.5 generics do *not* accept primitive types
> as arguments.  If you want to do that without the very heavy
> overhead of Java boxing and unboxing, M4 is the only game in town.)
>
> So macros are NOT that evil :)

Macros are indeed like gotos:  when you aren't given the
*right* tools for the job (something rather like Haskell/Clean/
Mercury/BitC typeclasses), macros are better than nothing.
But something built into the language (counting the Dialyzer,
which can read and check -spec declarations for functions but
not for ?macros), is generally better.
>
> But what if, instead of or in addition to ?MODULE and ?LINE,
> the system provided you with ?HERE, expanding to
> {Module, Line, {Function, Arity}}.  Then
>
>        iutil_log:flog(?HERE, "Received ~p from ~p~n", [Msg,Socket])
>
> doesn't seem _that_ horrible.
>
> Sorry to belabor the point, but what if it did? It doesn't. What if  
> we all lived in peace and harmony and respected each other's rights?

Two answers.  First, adding ?HERE to 'epp' is no big deal.
Second, we don't actually need it.

In your debug.hrl file, have

	-compile({inline, [{debug,0},{flog,3}]}).

	debug() -> true.		% or false, whatever.

	flog(L, F, A) ->
	    case debug()
	      of true  -> iutil_log:log_debug(?MODULE, L, F, A)
	       ; false -> ok
	    end.

And now all you need to get what you had before is

	flog(?LINE, "Received ~p from ~p~n", [Msg,Socket])

where the ?MODULE part is supplied automatically.

> Suppose we had top-level variables instead.  So
>
> But we don't. I wish we did.

Agreement again.  One of the reasons we *don't* have
top level variables is that we were given the preprocessor
*instead*.  Another strike against it.  However, thanks
to inlining, we don't actually need them for this problem.

>
> True - but I was working towards my ultimate point that having to  
> recompile is a pain and wanted the ability to change run-time log  
> levels.

Which I dealt with later in the message.

> And, using the above code, won't you get copious compiler or  
> Dialyzer warnings complaining that only one branch of the case will  
> ever be reached? How would you suppress those? Or are you going to  
> invent a suitable -pragma() for Erlang :) ?

You're right that Dialyzer complains here.
This is precisely as annoying, and precisely as (un)helpfl,
as the way Lint started to warning about "constant conditional"
in the do { ... } while (0) construction.
If a case or if has *no* matching arms, that's a failure,
but if *some* arms don't match, that's not a failure,
it's just constant propagation.

What's really annoying is that adding the declaration
-spec(debug/0 :: () -> bool()).
-- which I expected to tell Dialyzer that although *this*
version of debug/0 can't return 'false', another version might --
didn't help at all.  Shouldn't this *be* the pragma we need here?
(Dialyzer v1.8.1)
>
> Ah, the old I've-seen-crap-code trick. Yes, absolutely true, and  
> anyone worth their salt will never put important code into log  
> statements that won't get evaluated.

Which means that the argument for using a macro here
completely disappears.
>
>        -inline([flog/3]).
>        flog(Where, Format, Arguments) when Debug -> %%% Huh??
>
>            case iutil_log:ok_to_log(debug)
>              of true ->
>                 {Module, Line, _} = Where,
>                 iutil:log_debug(Module, Line, Format, Arguments)
>               ; false ->
>                 ok
>            end;
>        flog(_, _, _) ->
>            ok.
>
>        ... flog(?HERE, "......", [.....]) ...
>
> would push the evaluation of the format and arguments into the one
> case branch that uses them.  If it doesn't, we have far worse
> performance issues to worry about than this one.
>
> Where does Debug come from? Is it a (non-existent) top-level variable?

Please.  We are picking nits here.  So I edited the wrong version.

	flog(Line, Format, Arguments) ->
	    case debug()
	      of false -> ok
	       ; true->
                  case iutil_log:ok_to_log(debug)
                    of false -> ok
                     ; true ->
		      iutil_log:log_debug(?MODULE, Line, Format, Arguments)
		 end
	    end.
>
> In this context, I find BitC's distinction between pure and impure
> functions _in the type system_ interesting.
>
> That would be very useful - does that  mean the compiler  
> automatically flags functions as pure or impure based on their side- 
> effects?

Yes.  The draft manual gives the example of inferring
that 	map :: %e fn (%e fn a -> b) list(a) -> list(b)
-- I can't quite recall the syntax -- a call to map(f, xs)
has side effects if and only if a call to f has.
>
>
> That's also its strength (to be able to do ANYTHING and you don't  
> have to change the code). To find the definition using find and grep  
> isn't *that* hard, is it? I have searched the entire Erlang source  
> code base for things of interest in under a minute with find/grep.  
> Maybe it's slow using that 500MHz UltraSparc you are saddled with :(

It's actually a 2.16 Intel Core 2 Duo intel-Mac that I've been
running Erlang on recently.  Grep is nice, but it doesn't understand
about comments, it doesn't understand the difference between
f(X) and f(X, Y), and a whole bunch of other things which explain
why we have an Erlang cross-reference program.

> If only we had ?HERE,
> But we don't. I have to work with the real world. If ?HERE was here  
> I would use it!

A prototype implementation of ?HERE,
where it expands to {?MODULE,?LINE,undefined},
is just this much of a change to epp.erl:

<     Ms30 = dict:store({atom,'HERE'}, {none,[
<               {'{',1},{'?',1},{atom,1,'MODULE'},
<                 {',',1},{'?',1},{atom,1,'LINE'},
<                 {',',1},{atom,1,undefined},{'}',1}]}, Ms3),
<     Ms31 = dict:store({atom,'MODULE_STRING'}, undefined, Ms30),
---
 >     Ms31 = dict:store({atom,'MODULE_STRING'}, undefined, Ms3),

Making it add ?LINE in as well would be trivial.
To get function information requires parser support,
so I'd have the preprocessor expand it to a
	{location, Line_Number, File_Name, Module}
token, and then have the parser recognise this and
generate the appropriate data structure.  I suspect
that I would have the parser turn it into a new {location...}
AST form, and then another pass would deal with it.

The issues are
(1) finishing the design.  Exactly what information should we
     put in a location?  EEP time, except I don't have time today.
(2) writing the code.  Given my unfamiliarity with the compiler,
     this would probably take me two days to code and test.
(3) documenting it.  Hunting down all the files that need
     changing would be the tricky bit.
(4) fixing all the code that currently thinks it knows exactly
     what the lexical analyser and parser can produce.
     This is the really hard bit.


> The one important thing that I perhaps didn't understand from your  
> discussion is how you proposed to change the logging level at run  
> time, given the *existing* features of Erlang.

With plain ordinary run-time code, how else?
Didn't you see the "case iutil_log:ok_to_log(debug)" line?
>
> Understand that I gave the LOG_DBG macro as an example. As it  
> happens, I have LOG_TRACE, LOG_INFO, LOG_WARN, and LOG_ERROR as well.

Which can presumably all be done the same way.
To someone writing this code from scratch today,
using functions is _less_ effort (because better
understood by editing and debugging tools) than
using macros.  With *no* additions to the language.
(Except for getting Dialyzer to shut up.)

> Let me tell you what I have done, and this will probably give you  
> nightmares because it's a terrible hack, but it works well.

Actually, no.  It was the obvious thing to do.
While I too suspect that it's probably faster than
using the process dictionary, it might be a good idea
to measure it.

Ironically, you have actually provided a good example
of why functions are *better* than macros for this kind
of thing.  You *can* hot-load a replacement for an
imported function, but you *can't* hot-load a replacement
for an included macro.  (Nor can you set breakpoints on
macros.)

What I mean by a "top-level variable" is not something
mutable.  It's simply an Erlang variable bound at the
top level of a module.  A named constant, in other words.
Although it would be nice to export and import top level
variables; your hot-loading trick would still apply (and
maybe even be a touch faster).