Algorithmic lists

[Ulf Wiger]

On Tue, 17 Oct 2000, Robert Virding wrote:

rv>Because of all this I don't really see that much use of lazy
rv>lists apart form as a fun programming exercise.  Find me a real
rv>application and try to convert me.  Personally I think fifthly
rv>is the real clincher.

Not to say that it wasn't fun, but we started this as an approach to
be able to change data access patterns in existing code without
changing too much code in the process.

We see quite often that people solve their problems by handling large
lists of data. If the data is stored in mnesia or ets, they call
ets:tab2list/1 and then traverse the data as a list. This is a very
bad idea if the tables are large, since Erlang will allocate about
three times as much memory as is needed to actually represent the data
on the process heap.

Our approach, when we come across these patterns, is to rewrite the
iteration so that they step through a table instead of building a
list and then stepping through it. But with thousands of lines of
list-processing code, we'd have to rewrite a ton of function clauses
if we depart from the list syntax. Often, we end up doing an almost
complete rewrite for this reason (*).

We feel that this is a simple extension that gives us much more
flexibility, and allows us to experiment with physical data
representations by changing only a few lines of code -- and without
changing the fundamental nature of what we're doing, which is
iterating over a set of data objects (**).

/Uffe

(*) Of course, these rewrites are usually warranted for other reasons,
but timing is always an issue in production projects. If we can, we'd
like to schedule complete rewrites to some other time than during the
system test phase -- which is a fairly common place to find these
problems.

(**) Also, this is not so easy to address by simply trying to educate
programmers about how to write code, so that we can easily change it;
people tend to use the most intuitive constructs, and we should
encourage this as much as possible. Operating on a list is one of the
most intuitive concepts, so it is used very often.

-- 
Ulf Wiger                                    tfn: +46  8 719 81 95
Strategic Product & System Management        mob: +46 70 519 81 95
Ericsson Telecom AB,              Datacom Networks and IP Services
Varuvägen 9, Älvsjö,                    S-126 25 Stockholm, Sweden