[erlang-questions] Blog article on our reasons for moving to erlang.

[vijay chakravarthy]

We wrote our own search engine in Erlang. It didnt need to have all the
features of lucene, since in our problem space,
we only needed to do simple boolean search.

Writing a  inverted index based search engine in Erlang is quite
straightforward, mainly due to two reasons:
1. Mnesia and dets can easily handle "the last n additions" problem. This
allows you to structure a search engine as a combination
of merging responses from two queries -- the last n additions to the index
which havent been merged into the file based index,
and the results from the file based inverted index.

2. For file based inverted indexes (like the segments in lucene), the bit
syntax allows you to write very efficient (Golomb coded) structures, while
the standard library has some really nice modules like file_sorter which are
useful..

For us, the problem was even simpler, since typically PowerPoint documents
do not contain large amounts of text, so a simple
mnesia based inverted index with some clever optimizations suffices. It also
solves the problem of making the search engine highly available.

Regards,
Vijay


On 4/19/07, Christian S <chsu79@REDACTED> wrote:
>
> What did you replace lucene with in the new heterogene erlang version?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070419/5260f9c5/attachment.htm>