[erlang-questions] What I dislike about Erlang

[Evan Miller]

A few more comments specific to TinyMQ:

On Fri, Aug 31, 2012 at 1:20 AM, Richard O'Keefe <ok@REDACTED> wrote:
>   Another documentation failure is that we fail to document what
>   is not there.  In TinyMQ, a channel automatically comes into
>   existence when you try to use it.  Perhaps as a consequence of
>   this, there is no way to shut a channel down.  In TinyMQ, old
>   messages are not removed from a channel when they expire, but
>   the next time someone does a 'subscribe' (waves hands) or a 'poll'
>   or a 'push' *after* they expire.  So if processes stop sending
>   and requesting messages to some channel, the last few messages,
>   no matter how large, may hang around forever.  I'm sure there
>   is a reason, but because it's a reason for something *not* being
>   there, there's no obvious place to hang the comment, and there
>   isn't one.  (Except for the dead 'expire' clause mentioned above.)
>
> IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE.

This probably proves your point, but just to be clear old messages
will only "hang around forever" if 1. there are no new messages on a
channel and 2. the channel continues to receive requests. If there is
no activity on a channel for max_age, the gen_server timeout is
invoked here:

https://github.com/evanmiller/tinymq/blob/master/src/tinymq_channel_controller.erl#L76

That calls "expire" (erasing the reference to the channel) and then
exits the supervisor (eliminating the channel and all of its
messages). So in the absence of channel activity, the longest a
message will hang around is 2 * max_age, e.g. when there is a push
\epsilon seconds before an old message is set to expire.

>
>   The central fact about TinyMQ is that it holds the messages of
>   a channel in a simple list of {Message, Timestamp} pairs.  As
>   a result, every operation on the data takes time linear in the
>   current size.
>
>   This is not stated anywhere in any comments nor in the README.
>   You have to read the code in detail to discover this.  And it
>   is a rather nasty surprise.  If a channel holds N messages,
>   the operations *can* be done in O(log(N)) time.  (I believe it
>   is possible to do even better.)  Some sliding window applications
>   have a bound on the number of elements in the window.  This one
>   has a bound on the age of elements, but they could arrive at a
>   very high rate, so N *could* get large.

I have added some notes about TinyMQ's run-time characteristics to the README:

https://github.com/evanmiller/tinymq/commit/568bc7ce94ebaa42e3ce047c375c73323042853a

>
>   It is very easy to implement the necessary operations using lists,
>   so much so that they are present in several copies.  Revising the
>   TinyMQ implementation to work better with long queues would be
>   harder than necessary because of this.  And this goes un-noticed
>   because there is so much glue code for the guts to get lost in.
>
>   Given that Evan Miller took the trouble to use library components
>   for structuring this application, why didn't he take the next step,
>   and use the existing 'sliding window' library data structure?
>
>         Because there is none!

This would be a useful addition!

Evan

>
>   Yet sliding windows of one sort or another have come up before in
>   this mailing list.  Perhaps we should have a Wiki page on
>   trapexit to gather requirements for one or more sliding window
>   libraries.  Or perhaps not.  "true religion jeans for women" --
>   what has that or "Cheap Nike Shoes" to do with Erlang/OTP
>   (http://www.trapexit.org/forum/viewforum.php?f=20)?
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



-- 
Evan Miller
http://www.evanmiller.org/