[erlang-questions] My frustration with Erlang

[Bhasker Kode]

Hey Joel,

>4 Mnesia overloaded error reports
> ** WARNING ** Mnesia is overloaded: {dump_log,write_threshold}

> e.g. the reply  takes longer than 5 seconds. The game bombs out at this point.
> I could increase the timeout or make the call just wait forever. I  don't want to do this, though, as this timeout serves as a performance indicator.

I've had the same exact same two reports causing all kinds of
unexpected behaviour.

While setting infinity with gen_server call's  worked with me, i think
setting a higher timeout value (than the default 5 seconds ) is your
best best, and thats what worked with us. We had some python<->erlang
logic happening in one of the gen_server calls, and used to
consistently timeout until i came to know of the 5 second default
value.

IMHO , perhaps you can also move the 'performance analysis' to another
part of the poker server, rather than mixign it with the decks, poker
logic. In that case , having gen_server calls as infinite may not seem
a bad idea.

Regarding the mnesia dump_threshold calls, we did two things :

#1 If latency is'nt an issue take  a whole bunch of writes , write
them to a file, then on a trial-and-error basis configure that after
every N entries, do a mnesia:load_textfile(Dump). This way all those N
entries get written in one transaction. One thing to look at though is
that at the time of loading, permissions to access mnesia . In our
case since we access mnesia only through Yaws , so no rwx permission
errors throw up , and also made sure that mnesia already knows where
to distribute the newly loaded data ( in our case a 4-node cluster
currently). If latency is critical, then why do you need mnesia in the
first place, just message pass (but i suppose that you still need to
archive them even if it behaves like a messaging system)

#2 creating user specific tables. i know most normalization/db guru's
out there might raise an eyebrow . As much as we're used to pattern
matching all rows from a table where user matches X, we just created
seperate tables for each user, and know exactly to look. Now we can
take maybe a much higher factor of the load into a table , without
even considering Fragmentation. btw, Thanks for pointing out the
default max ets tables, will have to look into that.

jut FYI, the third biggest error we used to get on our logs was the
{error,open} message "yaws failed to bind socket error", and we spent
a long time figuring out if it was related to the timeout. turned out
that yaws.conf has a little property that sets "Fail completely when
yaws fails to bind socket = true " by default. Switched it to false,
and that's the last time we saw those error,open reports 8 )

~
Keep Clicking,
Bhasker V Kode | bosky101
hover.in