[erlang-questions] TCP flow control problem

[Nav]

Hi Bengt and Alain,

Many thanks for pointing out my logical error.

As I am now rewriting my whole network layer, what would be the best
recommended design.

My requirement is that the Network Layer should just read data, and
pass it on to other OTP component using handle_cast.

Network Layer Logic (gen_fsm or gen_server) psuedo code
-------------------------------------------------------------------------------------

- Socket in {active, once} mode  {to keep flow under control with remote server}
- Read available data using handle_info()
- Parse message-data using marker, and store rest of message-data
(=DataRest) inside State variable of OTP
- Set socket to {active, once} mode
- Go back to start, read data and add after DataRest, and parse
message, and repeat above steps.


Since I am aiming to handle very high amount of traffic, would this
design suffice?

For e.g.,
- Could handle_cast queue run out of memory, if network layer is
pumping messages at faster rate?

Any better design to make server faster and be able to handle high capacity?


Thanks again,

Nav


On Mon, Jan 31, 2011 at 7:27 PM, Alain O'Dea <alain.odea@REDACTED> wrote:
> Bengt is right.  I suggest using a gen_server to collect the data and look for message boundaries.  As it collects data it would prepend it to an iodata() in its own State structure.  When it encounters a message boundary it would cast the data in its State plus the data up to and including the message boundary onto a delegate gen_server for processing.  The data in the packet after the message boundary would then become the new partial message data in it's State.
>
> Below is pseudo-code for this:
>
> -record(state, {partial_message :: iodata(), delegate :: pid()}).
>
> handle_info({tcp, {data, Data}}, _From, #state{partial_message = Partial} = State) ->
>    NextPartial = case boundary_split(Data) of
>        {MsgEnd, NextMsg} ->
>            tcp_message_handler:handle_message(State#state.delegate, [MsgEnd | Partial]),
>            NextMsg;
>        Data ->
>            [Data | Partial]
>    end,
>    {noreply, State#state{partial_message = NextPartial}.
>
>
> On 2011-01-31, at 4:19, Bengt Kleberg <bengt.kleberg@REDACTED> wrote:
>
>> Greetings,
>>
>> TCP is a stream protocol. It does not preserve message boundaries. You
>> have to handle that yourself. Sleeping one second is one way. When you
>> do not sleep, how do you separate the messages?
>>
>>
>> bengt
>>
>> On Sun, 2011-01-30 at 23:38 +0100, Nav wrote:
>>> I have implemented two TCP servers using gen_fsm, and gen_server, respectively.
>>> I am using {active, once} option, after getting each message.
>>>
>>> When I try to send data from one side to another in a loop, the data
>>> gets corrupted (first 1 or 2 message arrive ok, but following gets
>>> corrupted) when it arrives at other side.
>>>
>>> However, if I put a timer:sleep(1000) before sending each message,
>>> then everything seems to go fast.
>>>
>>> What seems to go wrong by sending data too fast? Do I need to set some
>>> more socket options?
>>>
>>> Thanks, Nav
>>>
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>
>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>