[erlang-questions] Idiomatic iterators

[Frédéric Trottier-Hébert]

There are two different reasons for iterators there. The first one is to avoid producing the elements in the first place, and the second one is to just provide an API to let you work on them one at a time.

In the first case, you want to only fetch a few elements at a time: they might be in a file, a DB, or even just bits of information you can generate mathematically. In this case, you can look at the APIs of ETS and Mnesia, which use the form Mod:next/2 and Mod:prev/2 (http://erldocs.com/R14B01/mnesia/mnesia.html#next/2). This type of API is rather flexible because it allows random access to many readers based off the keys. If you're able to hold your data set in one of these databases, you'd benefit from iterators right there.

If what you want is the iterator so you can work on each element one at a time, but don't really mind holding them all in memory at once you might be fine using something like the queue module and popping elements off the queue one at a time. Otherwise, there's the generalisation of that queue called a 'zipper'. I give a quick explanation on my blog (http://ferd.ca/yet-another-article-on-zippers.html) and have a little library with them (https://bitbucket.org/ferd/zippers/src) that can be installed with agner (http://erlagner.org) too. Zippers have the advantage of being bi-directional (you can go forward or backwards) and can insert at the current point you're in the structure, while the queues in the standard library allow you to push element on the queue and then remove them in O(1) amortised time. 

Last of all, you have gb_trees and gb_sets, which provide an iterator to the data structure that supports a single operation, next/1. gb_trees is used for key-value stores while sets are for sets (duh) of unique elements. These are mostly useful because they allow to amortise the conversion to a list over each element as opposed to at the beginning.

--
Fred Hébert
http://www.erlang-solutions.com