Conditional compilation (was: Erlang/OTP R10B-10 has been released)

[David Hopwood]

Richard A. O'Keefe wrote:
> David Hopwood <david.nospam.hopwood@REDACTED> replied
> to my observations about conditional compilation.
> I always read anything of his that I see with interest and respect,
> so it was a little alarming to see that we didn't appear to agree.
> 	My experience (mainly in C, but not specific to C) is that this
> 	is not sufficient in practice.  For example, in a recent
> 	embedded system, I have:
> 	
> 	#define ENABLE_CONTROL            1
> 	#define ENABLE_HEATER             0
> 	#define ENABLE_HEATER_ALARM       0
> 	#define ENABLE_UV                 1
> 	#define ENABLE_ANTISTATIC         1
> 	
> 	and so on, for ~89 features.

(Incidentally, it is ~61 features. I should not have counted 28 macros
controlling debugging and logging that only rely on dead code elimination,
not conditional compilation.)
	
> 	This does not mean that there are 2^89 combinations of features
> 	that need to be tested.  The final program is going to have
> 	*all* remaining features enabled, after any that don't work or
> 	are redundant have been stripped out.  A test with some features
> 	disabled is only intended to be a partial test.
> 	
> Fortunately, he and I are not talking about exactly the same thing.
> 
> I was talking about feature tests IN CODE AS DELIVERED so that it is
> entirely possible that no two customers will be running the same code.
> The particular example program I was talking about has now been reduced
> to a possible 100,000 variants, and we have determined that most of them
> do NOT in fact work.  (That's because just a few of the features interact
> badly.)
> 
> David Hopwood is talking about temporarily snipping stuff out
> DURING DEVELOPMENT AND TESTING.
> 
> 	There are two main reasons to disable a feature:
> 	
> 	 - because it doesn't work. Sometimes this is because the associated
> 	   hardware doesn't work yet, and sometimes the code doesn't compile or has
> 	   known bugs that would interfere with testing the rest of the system.
> 	
> 	 - because we want to replace it with a mock/stub implementation for testing
> 	   on a development system rather than on the actual hardware. In some cases the
> 	   code would not compile on the development system because a needed library
> 	   is not implemented there.
> 	
> 	Note that as well as stubbing out the code that implements it, disabling a
> 	feature sometimes needs to change code that would use that feature in other
> 	modules.
> 	
> 	While I like the general idea of feature pseudo-functions, I
> 	think that to be useful as a tool for testing parts of programs
> 	during development (which is the main thing I use conditional
> 	compilation for), there must be support for excluding code that
> 	doesn't compile.
> 	
> There's a big difference here between C and Erlang.
> In C, the presence or absence of a declaration in one place
> can affect whether or not a later function can be compiled or not.

Right; I see now that dynamic typing makes a big difference here. In C, it's
often necessary to set up types and function prototypes before other code can
compile. (The fact that typedefs share the same namespace as other identifiers
is one reason why this is trickier even than in some other statically typed
languages.)

In Erlang, I think the only corresponding category of declarations that can
affect the validity of other code, in a "development mode" as you describe below,
are record declarations.

> As long as you DON'T use the preprocessor, that's not true in Erlang.
> In delivery mode, you want missing functions to be errors.
> But in development mode, it may be enough if the compiler *warns* about
> functions that are used or exported but not defined and automatically
> plants stubs that raise an exception if called.

Well, it is true in Erlang as it stands now, because calls to missing functions
are errors.

But yes, if there were a mode in which they are warnings, this approach would
probably be sufficient to replace my use of conditional compilation, and is
consistent with the pragmatics of a dynamically typed language.

In addition to having the stub throw an exception, it might be a good idea to
have a construct that tests for the presence of a function, returning a boolean.
Note that this *cannot* be simulated in all cases by trying to call the function
and catching the exception, since we may want to test for its presence *without*
calling it.

I just had a look at several more C/C++ programs (unfortunately, I don't have
enough programs in Erlang, or other languages with conditional compilation, to
draw any conclusions from them on this point).

Excluding things that are totally C/C++-specific, all uses of conditional
compilation fell into the following three categories:

1. Alternative implementations depending on platform or configuration.
   This includes replacing functions that are standard, but missing on a
   given platform.

2. Sanity checks on constant expressions, including invalid combinations of
   features ('#if !check  #error ... #endif').

3. Workarounds for differences between language or compiler versions (e.g.
   '#if __STDC_VERSION__ >= 199901L', '#if _MSC_VER > 1000', '#if __cplusplus').

I'm quite prepared to believe that category 3 is much less common in Erlang.
Categories 1 and 2 would be better handled in a configuration language.
(Category 2 checks could also be done at runtime on start-up, which has the
advantage of allowing some conditions that are not compile-time constants to
be tested.)

> Taking the list of reasons in turn:
> 
>  * because it doesn't work
> 
>     That's a reason for not *calling* code, but not for not *compiling* it.
>
>    * because the hardware doesn't work yet
> 
>       Agreed that you don't want to call code that uses hardware that
>       doesn't work yet, but it's not clear to me why you wouldn't want
>       to *compile* it.  In fact, it might be *essential* to compile it
>       to make sure that it doesn't interfere with something else.
> 
>    * because the code doesn't compile
> 
>       We are not talking about the desirability of #if in C, but in Erlang.
>       It's much harder to write code that doesn't compile.  If it
>       doesn't compile, I don't want it processed until it *does*, and that
>       means that I *don't* want it controlled by an "ENABLE" switch that
>       might accidentally be set.

I suppose so, but note that Erlang doesn't (AFAIK) have nestable multiline
comments, so if you do have code that you want to disable temporarily because
it doesn't compile, the only way to do that without the preprocessor is to add
'%' to the start of every line. Perhaps there should be an explicit construct for
this, equivalent to C's '#if 0' idiom.

>    * because the code has known bugs
> 
>      This is a reason not to *call* the code, not a reason not to *compile*
>      it.  You want to select test cases using some kind of rule-based thing
>      that leaves out things that test bugs you know about until you are
>      ready to test that those bugs have gone.
> 
>   * because we want to replace it with a mock/stub implementation
> 
>     This is the classic multiple implementations of a single interface
>     issue.  This is where you want child modules and a configuration
>     management system that says "in this configuration use this child,
>     in that configuration use that one."
> 
>     In effect, the preprocessor does this kind of stuff *BELOW* the
>     language level using a very clunky and limited language.  I'm saying
>     that it should be done *ABOVE* the language level using some
>     reasonable rule-based language.  (Datalog?)
> 
>    * because the code would not compile on one system because
>      a needed library isn't there.
> 
> So we see that
> 
>  - test case selection needs to be informed by which tests exercise
>    what, so that we don't (deliberately) *run* code that we don't
>    intend to, but that doesn't have to mean we don't *compile* it
> 
>  - installing the software in different environments may require different
>    implementations for some functions &c, which is very little different
>    from the features for installation problem.  And this can be done
>    by child modules (which we don't have, but could do with) and some
>    kind of declarative configuration language.

You've convinced me. (I didn't need very much convincing -- I have thought
for a long time that declarative configuration languages are the Right Thing
for building any nontrivial program.)

I had some concern that, in cases where the variation between different
versions of a module is small, that requiring it to be abstracted into a call
to a child module rather than "inlined" might make the code less clear, and
possibly result in some code duplication (with the maintenance problems usually
associated with duplication). This would be most disruptive in cases where the
conditional code uses variables from its lexically enclosing scope, which would
have to be turned into function parameters.

But I see from the "exclude function calls" thread that the current Erlang
preprocessor can't actually be used to express conditional code within a
function, anyway. And after looking through the C/C++ code mentioned above,
I think the needed restructuring would be an improvement in almost all cases,
and not at all onerous when writing new programs.

I have some concrete suggestions for the build/configuration system, but
I'll leave those to another post.

-- 
David Hopwood <david.nospam.hopwood@REDACTED>