OO, FP and XP training (extremely long)
Jay Nelson
jay@REDACTED
Fri Dec 9 19:41:20 CET 2005
[The meat about Functional Programming is at the bottom, you need to
read the set up to understand how I arrived at my conclusions.]
At work we are mainly a C++ / C# shop now with most development being
geared towards .NET. We previously were a UNIX / C++ shop with
excellent scaleability, reliability and redundancy. Needless to say,
scale is being challenged by the move to interactive GUI-based
applications on Windows XP coming from UNIX servers.
As part of this multi-year transition we have embraced the principles of
eXtreme Programming (XP) and agile development. Our team has used an
external consultant in the last few weeks to coach us, although we have
been using the Test Driven Development (TDD) approach for a year now.
One of the demos that they gave was a live process of developing a
bowling game scoring application to show how refactoring works in
practical terms. The lecturer ran through the typical OO design anyone
would come up with which involved 4 or 5 classes and inheritance, and
then held it to the side while he developed the code with prompting from
an audience using TDD. The end result was a for loop with two if
statements and no classes. This caused a little discussion as to why he
ended up with the result he did (TDD forced him to write only the
necessary code) and his reflection on why "the power of OO was not
needed in this case". After being challenged on this in the past, he
wrote the full OO implementation and ended up with over 400 lines of
code, confirming his suspicion that no OO was a simpler implementation.
However, still convinced that OO was a good thing, he believed for more
complicated programs it was warranted and aided in development*.
In our actual pairs programming with the coach observing, he often
wanted to simplify the code we have built up over the last year. One of
the things that surprised me was that he wanted to eliminate Getters and
Setters and just use public member variables. Another difference was
that all member functions be declared public since that made them much
easier to test without proxy methods. He also advocated removing
inheritance in many cases and replacing it with object factories,
cooperating classes and interfaces that exposed only the required public
methods.
-----
*My immediate question was, "If this is so obvious with a toy problem,
why isn't the complexity of the code multiplied many times when scaled
up to a larger problem?".
-----
It was enlightening to see the change that TDD is bringing about from
the standpoint of the experts in the field (our coaching organization
has been a long advocate and publisher of some seminal texts on the
subject). Here is my take from the exercise:
1) IDEs are very important in changing the way you program
- Verbosity is great, type a few letters and allow completion
- Refactoring, renaming is much easier
- Don't ever worry about overhead, avoid all duplicate code
- Rely heavily on the editor to point out errors without compiling
- Declare types so that the IDE signatures are more readable, rather
than type safety
2) TDD changes the way you code
- Test first means most functions are best made public
- Constant refactoring makes the code units smaller
- Make sure you refactor the tests just as diligently
- Freely modify anything, the tests protect you
- Classes are essential to override external behavior
3) Interfaces are much more important than classes
- Use object factories and only access via interfaces
- Use interfaces as replaced facades when testing
- Mock objects allows dynamic swapping of functionality
4) All meetings and planning are evil
In general I shuddered at a lot of the "best practices" that violated my
old training in OO about protection of data and encapsulation. This was
after all one of the big advantages of OO. I came to the following
general conclusions (feel free to challenge or augment them):
A) TDD subverts staticly typed languages
The main purpose of static typing is to allow the compiler to notify the
programmer of errors before allowing the execution of code. If you have
a bunch of tests that break, it can be more thorough than just a static
type check of variables and signatures. This equates in my mind to
preference for run-time validation with real data.
The second advantage of static typing is compiler optimization, but with
XP, programmer optimization is more important. Performance is only
considered if the acceptance test that specifies a performance level
does not pass.
B) Engineering principles and experience hinder development
TDD and XP are all about making the team more effective -- faster with
fewer software errors. The methods used boost the worst performers a
lot and hinder the best performers to a lesser degree. (I suspect the
very best programmers are slowed down by more than the boost of the
worst, but there are so few world-class programmers that most companies
have never met one.) Any anticipation based on previous experience is
eschewed for fear it results in over-engineering when a simple solution
works. Start with the simplest case that passes the test and gradually
refine it -- the process of refinement incurs refactoring which will
produce the desired solution with the simplest design.
While this works in practice for most problems -- it is very easy to
refactor code to speed it up -- architectural problems do not fall out
so simply. Refactoring a data flow that involves multiple systems which
did not consider performance (or, say, hot code loading) cannot just be
refactored to introduce a solution. Deep thought and insight are needed
to produce an innovative solution. But a fast, reproducible, straight
forward approach gets you into the marketplace competing with
like-minded companies quickly.
C) No one understands the system
Everyone works in pairs on every part of the code. Writing code is a
flurry of making a test case fail, writing a simple solution, increasing
the test case and refactoring. The tools simplify this so that extra
arguments can be added to a function in one place and it will be updated
everywhere -- even if you don't know where those changes happen. I
stepped away after hours of coding several times without understanding
anything we coded, but knowing that it passed the tests and therefore
was functionally correct.
Everyone is interchangeable. Mindless code generation and copious unit
test studying produce a localized understanding quickly that leads to
enhancements and bug fixes being done rapidly. Ask anyone a deep
question about the philosophy of the system, the general performance
characteristics or whether there is a clean approach to expanding its
range of scope and you will be met with blank stares and requests for
specific test descriptions.
D) XP developers are coding Functionally, not using OO
Gradually the constraints of class encapsulation are being removed. All
methods and member variables become public to ease testing, interfaces
are used to make the IDE work again rather than give a meaningless list
of every data item in the system (not to restrict code access, since the
unit tests prove which functions are accessed). Global variables and
global functions harken back to the days of APL, one of the first
functional languages. The use of cooperating classes rather than
inheritance is concurrency oriented and relies on function modules
(interfaces) to access the ghost classes.
When XP developers believe they are taking advantage of Mock Objects or
proxy classes, they are really doing late binding of function
interfaces. When unit tests run, they are simulating the dynamic scope
of execution and use run-time validation. When acceptance tests are
executing they have built an entire scaffolding that should be part of
the language and always enforced during execution. UBF-B is a general
solution that doesn't require all the test instances to be written.
My bold predictions:
1) Agile development will lead to OO languages being replaced by FP
languages.
2) Agile development will hit a wall that will require a "new innovative
development process" when average programmers use it for concurrent
programming without a language like erlang.
3) Businesses will quickly compete in markets and with a level of
software complexity that they previously couldn't, but will find
themselves stuck with a product and user base that is not understood
well enough for strategic planning purposes.
4) "Lone wolf" teams and programmers will be more innovative, but they
will be copied more quickly by average teams. The sheer volume of code
needed to compete will overwhelm the small organizations and force
innovation out of the market.
5) Big paradigm shifts will be necessary to rebalance industry
competition; mundane evolution of versions will be possible by all
companies.
6) IDEs will hide the complexity and level of insight equally, making
"hand coded" programming a specialized black art that is equated to
hotrod enthusiast engine tinkerers.
7) Big businesses will continue to make money touting large complex
systems worked by a team of programmers; startups will continue to find
niches that they can subvert with new approaches that they sell to the
big businesses.
jay
More information about the erlang-questions
mailing list