Poker bots and nested state machines /Structured Network Programming reloaded/

[Joel Reymont]

I went through Ulf Wiger's EUC presentation on Structured Network  
Programming and realized that I had to tackle something similar in my  
recent project. I would like to share my design and solicit feedback.

I find my approach very easy to implement and use and Uffe's a bit  
complicated. I would appreciate comments on the merits of the our  
approaches among other things.

The goal of my project is to be able to thoroughly test a poker  
server using poker bots. Each poker bot is to be scripted and  
excercise different parts of the server by talking the poker protocol  
consisting of 150+ binary messages. The poker server itself is  
written in C++ and runs on Windows.

Easy scripting is an essential requirement since customer's QA techs  
are not programmers but need to be able to write the bots. Another  
key requirement is to be able to launch at least 4,000 poker bots  
from a single machine.

It's worth noting that I started with Haskell and by the time I gave  
up 10-11 weeks later I was still troubleshooting core issues in my  
application. It took me 1 week to rewrite the app in Erlang. It's the  
end of that week and I'm  already way past the state of the Haskell  
version. The Erlang code is about 1/3 of the Haskell code. I used the  
same approach to event handling in both apps.

Each poker bot is a state machine. I managed to bang out wrappers to  
the 150+ packets rather quickly and elegantly (see http:// 
www.erlang.org/ml-archive/erlang-questions/200512/msg00279.html) and  
it took me about 2 days. Implementing the state machine proved to be  
more troublesome.

My initial thinking was to give the customer a library of packet  
wrappers and the means to send and receive them. This failed  
miserably. Writing a robust poker bot state machine proved too  
complicated for the QA techs. I was then asked to provide a library  
of "code snippets" or LEGO blocks to assemble a poker bot with.

This is a sample poker bot script that joins the lobby. Each poker  
bot script is a module with a single exported function. The module  
name is meant to be given to the controller which calls script/0 and  
passes it the user id, password and any other required arguments.

-module(gotolobby).

-export([script/3]).

-include("records.hrl").

script(Bot, _, connected)
   when is_record(Bot, bot) ->
     %% common script initialization
     Bot1 = snippets:start_script(Bot),
     Bot2 = snippets:go_to_lobby(Bot1, [...]),
     %% set trace level
     Bot3 = bot:traceat(Bot2, 100),
     bot:trace(Bot3, 10, "Kicking off"),
     %% mark event as processed
     {eat, Bot3};

script(Bot, _, {joined_table, 0}) ->
     bot:trace(Bot, 10, "We are in the lobby"),
     timer:sleep(3000), % milliseconds
     %% stop the bot
     stop;

script(Bot, _, Event) ->
     bot:trace(Bot, 100, "Skipping: ~w", [Event]),
     {skip, Bot}.

I control the event loop and dispatch events by calling the script  
fun. Each script fun is a dispatcher or state machine with 3 possible  
return values: {eat, State}, {skip, State} and stop. The controller  
keeps a stack of dispatchers and events are propagated from the top  
of the stack to the bottom.

If a dispatcher in the stack returns {skip, State} then the event is  
propagated to the next dispatcher and the new bot state is assumed to  
be State. On {eat, State} the dispatch sequence is complete and the  
controller goes on to the next iteration of the event loop. On stop  
the bot is stopped. If no dispatcher has eaten the event by the time  
the bottom of the stack is reached then the controller throws out the  
event.

Dispatchers can be pushed on to the stack and popped from it. The  
first dispatcher that's pushed onto the stack is the bot script  
itself. It always stays there. I do not try to guard against extra  
stack pops, my assumption is that a qualified person will be writing  
the snippets and will match each push with a pop. Bot writers do not  
need to use push/pop at all.

%%% Push a handler. Desc describes the handler (string, atom, etc.)
%%% and is only used for debugging.

push(Bot, Desc, Fun, Args)
   when is_record(Bot, bot),
        is_function(Fun),
        is_list(Args) ->
     Dispatchers = Bot#bot.dispatchers,
     Bot1 = Bot#bot {
	     dispatchers = [{Desc, Fun, Args}|Dispatchers]
	    },
     Bot1.

%%% Pop a handler

pop(Bot)
   when is_record(Bot, bot) ->
     %% drop the head
     [_|Dispatchers] = Bot#bot.dispatchers,
     Bot1 = Bot#bot {
	     dispatchers = Dispatchers
	    },
     Bot1.

The reusable blocks of code (snippets, LEGO blocks) are implemented  
as dispatchers that are pushed onto the stack. By pushing snippets to  
the top of the stack they get to look at each event before the bot  
and decide whether to propagate the event or eat it.

The controller event loop looks like this:

%%% Dispatch event

run(_, {keep_going, Bot})
   when is_record(Bot, bot) ->
     receive
	{tcp, _, <<Packet/binary>>} ->
	    Event = unpickle(Bot, Packet),
	    run(Bot, handle(Bot, Event));
	{script, Event} ->
	    run(Bot, handle(Bot, Event));
	Any ->
	    run(Bot, handle(Bot, Any))
     end;

%%% Handle event

handle(Bot, Event)
   when is_record(Bot, bot) ->
     %% let dispatchers in the stack take a look
     handle(Bot, Bot#bot.dispatchers, Event).

handle(Bot, [], _)
   when is_record(Bot, bot) ->
     %% bottom of the dispatcher stack reached,
     %% throw out the event and keep going
     {keep_going, Bot};

handle(Bot, [{_, Fun, Args}|Rest], Event)
   when is_record(Bot, bot) ->
     case Fun(Bot, Args, Event) of
	{skip, Bot1} ->
	    handle(Bot1, Rest, Event);
	{eat, Bot1 } ->
	    {keep_going, Bot1};
	stop ->
	    stop;
	Other ->
	    trace(Bot, 85, "handle: Unknown event: ~p~n", [Other]),
	    erlang:error(unknown_event)
     end.

Consider this section of the example bot code:

script(Bot, _, connected)
   when is_record(Bot, bot) ->
     Bot1 = snippets:start_script(Bot),
     Bot2 = snippets:go_to_lobby(Bot1, [...]),
     Bot3 = bot:traceat(Bot2, 100),
     bot:trace(Bot3, 10, "Kicking off"),
     {eat, Bot3};

snippets:start_script/1 above pushes another handler onto the stack  
that handles common events that are used in all poker bots such as  
updating the number of players at the table. snippets:go_to_lobby/2  
is a good snippet to look at.

%%% Go to lobby. The lobby is table# 0

go_to_lobby(Bot, AffId) ->
     go_to_table(Bot, 0, AffId).

%%% Join table

go_to_table(Bot, 0, AffId)
   when is_record(Bot, bot),
        is_list(AffId) ->
     Bot1 = bot:push(Bot, "go_to_table",
		    fun go_to_table/3, [0, AffId]),
     Cmd = #cl_connect_game {
      ...
      },
     bot:send(Bot1, lobby, Cmd),
     Bot1;

The fun above is meant to be invoked from poker bot scripts and  
pushes the go_to_table/3 dispatcher to the top of the stack. It also  
sends the "connect to table" packet to the poker server.

go_to_table(Bot, Table, AffId)
   when is_record(Bot, bot),
        is_number(Table),
        is_list(AffId) ->
     Bot1 = bot:push(Bot, "go_to_table",
		    fun go_to_table/3, [Table, AffId]),
     bot:reconnect(Bot1, Table);

Joining a particular table is more complicated since separate  
connections to the poker server need to be maintained for the lobby  
and for each each table. Not my design, so just take it for  
granted ;-). bot:reconnect/2 closes the existing table connection and  
opens a new one.

go_to_table(Bot, [Table, AffId], table_connected) ->
     Cmd = #cl_connect_game {
      ...
      },
     bot:send(Bot, table, Cmd),
     {eat, Bot};

The table_connected event is posted once the table connection has  
been established, i.e. the custom SSL handshake has been completed.  
More on that below. The lobby connection goes through the encryption  
handshake when the bot starts so we only need to send the "connect to  
game" packet and there's no waiting for the table_connected event.

go_to_table(Bot, [Table, _], #srv_connect_game_ok{}) ->
     bot:post(Bot, {joined_table, Table}),
     Bot1 = bot:pop(Bot),
     {eat, Bot1};

The table is considered connected when "srv_connect_game_ok" is  
received. That's when I post the {joined_table, Table} event and pop  
myself from the dispatcher stack.

Take a look at the sample script at the beginning of the message. My  
nested "go to lobby" state machine is completely transparent to the  
user. It even looks like a synchronous call /Bot2 =  
snippets:go_to_lobby(Bot1, [...]/.

go_to_table(Bot, _, _) ->
     {skip, Bot}.

If I don't recognize the event then I just skip it which lets  
dispatchers further down in the stack take a look at it. This is a  
crucial ability. Two server connections (lobby and table) can be open  
at the same time so I can be receiving lobby messages while trying to  
connect to a table.

Opening a new table connection requires going through a custom SSL  
handshake with the server. This eventually boils down to the  
following piece of code

connect(Bot, Args) ->
     %% connect to the script server to get a client handle
     UtilSock = Bot#bot.util_sock,
     ?match(ok, gen_udp:send(UtilSock, ?SRV_IP, ?SRV_PORT, <<? 
SRV_CONNECT>>)),
     %% push handler
     Bot1 = bot:push(Bot, "handshake", fun handshake/3, Args),
     %% start script server connection timer
     bot:start_timer(Bot1, script_server_connect, 5000).

I'm using a separate UDP server to keep track of SSL descriptors  
since I could not figure out a way to do this with the built-in  
Erlang SSL. I need to capture each packet in the standard SSL  
handshake and add my own header before sending it out. This lets me  
selectively use SSL encryption on the packets that require it and  
lessens the load on the server and network connections.

Notice the bot:push/4 above. The state machine (dispatcher) stack is  
3-deep at this point. The encryption handshake state machine is at  
the top, followed by the "connect to game" state machine and the  
poker bot state machine (gotolobby:script/3).

Finally, I start a timer above to make sure that the encryption state  
machine does not run forever.

	Thanks, Joel

--
http://wagerlabs.com/