Metaprogramming

[Jay Nelson]

When I first saw the smerl post my thinking went like this:

   - Normal programs execute and return a value
   - When I program, I generate source code and run the compiler
   - smerl is just doing normal programming

This is a step up from executing, but it is programming.

Then I looked on wikipedia and they call Generative Programming a 
metaprogramming technique, so a for loop that spits out source code is 
metaprogramming.  I guess it is a question of what your background is in 
making a determination like that.

I progressed through programming languages in the following order:

1) Basic
2) APL
3) RPG
4) Prolog
5) Lisp / Flavors (Symbolics) / Common Lisp Object System (CLOS)
6) C then C++
7) Perl
8) erlang
9) C#

I guess I learned some SNOBOL between 5 & 6 and REFAL around the same 
time.  5 was a progression of 3 different eras.

I can't recall ever hearing the term 'quasi-quote' or 
'quasi-expression'.  Macros and backquotes were just part of the Lisp 
language.

When I got to C is when I got confused.  It seemed like the language 
wasn't finished.  Here's what happened when I first start using it:

Me: "What is all this malloc / free nonsense?  I've never had to do this 
before."
Coworker (CW): "What do you mean?  All languages require memory management."
Me: "I haven't done this in 15 years of programming. Ever."

Me: "Ok, show me how to figure out what is wrong."
CW: "Fire up the debugger."
Me: "Good, I know what that is...   Hmm, what's the value of this thing 
all in caps?"
CW: "It can't tell you, that's a macro.  It's not part of the program."
Me: "??  How can that be a macro if it never executes?  It sounds like 
an editor expansion that somehow escaped from from its environment."
Me: "Ok, let me create a new variable and evaluate the expression it 
represents."
CW: "You can't create a new variable silly."
Me: "?? Ok, step into the function.  Wait, I see... the incoming arg is 
wrong.  Let's step back and change it."
CW: "No can do."
Me: "??  Ok, mentally I'll assume it is changed and...  oh I see the 
problem.  Let's change this function and continue!"
CW: "What planet did you come from?  This is just a debugger!"
Me: "Isn't a debugger the thing you use to write code?  You type it in 
to the listener and you modify it in the debugger.  When you're done you 
dump it in an editor using the pretty printer.  How do you guys write code?"

And that was 15 years ago.  Now it is possible to use Visual programming 
tools... although you still can't program in the debugger.


To me metaprogramming is writing code that dynamically changes the rules 
of the language.  Writing source code and calling the compiler isn't 
even generative programming because it feels like using a tool chain to 
script a series of things a human does at the keyboard.  Generating code 
(typically attributes, like when receiving a new XML stream with new 
attributes) that was not there when the compiler first ran is standard 
fare for a dynamic, runtime language.  Although I've not yet seen one 
code example in a commercial environment with XML where the 
implementation has to be recompiled to accept the new attributes.  Why?  
Because they use static languages and adding a new atom or string is 
runtime wizardry equated with automatic programming.

If you want to see how metaprogramming really works, read "Art of the 
MetaObject Protocol" by Gregor Kiczales and Jim de Rivieres.  This book 
is as eye opening as reading a first book on how neutron stars become 
black holes.  It will either blow your mind or prevent you from having 
fun writing normal programs again.  Every programmer should read this book.

[For those without the time, the book explains how OO is implemented in 
Lisp using StandardObject, StandardClass and StandardMethod and then 
shows how you can subclass those instances to **change the behavior of 
the object system** for example using a different implementation for 
storing a class or changing the semantics of inheritance or invoking a 
method.]

On further reflection, the metaprogramming in the book is modifying the 
object-oriented system not Lisp itself.  At some point all things bottom 
out or else there are no assumptions the system makes.  The reason Lisp 
was so great was because data and code were one (in the greatest zen 
sense) so that you could manipulate code with the same ease as data.  
You didn't need to use a different language or API and then call a 
compiler and some other arcane incantations.


Just some ramblings.  Not to detract from smerl which may prove to be a 
useful tool for erlang. Judging from wikipedia I am definitely in the 
minority on the classification of metaprogramming.

It's just that erlang is structured with a VM and internal 
representations and weak text-substitution macros, so this area of 
research has rather limited potential IMHO.

jay