Curious behaviour of gen_tcp

[Sean Hinde]

Hi,

I don't know the answer to your question about whether it is possible to
receive a tcp_closed before all the data is received from a socket, but
another observation may be of interest.

In R7B when you do concat_binary it will create a completely new binary.
This means that every time you go round your loop and append a new bit of
data you create a new copy of the entire buffer plus the new bit. This can
very quickly chew up all your memory..

It is generally better for large amounts of data if you maintain a list of
binaries and concatenate them in one go at the end. You can also use size/1
to get the size of a binary:

 loop_busy(Queue, Socket, Buffer) ->
   receive
     {tcp, Socket, Data} ->
       loop_busy(Socket, [Buffer, Data]);
     {tcp_closed, Socket} ->
       io:fwrite("Finished ~w", [size(concat_binary(Buffer))]),
       loop_idle(Queue);
     Any ->
       io:fwrite("Received ~w~n", [Any])
   end.

I'm not sure whether this behaviour changes in R8 with more support for
segmented binaries...

- Sean

> -----Original Message-----
> From: Alexis Le-Quoc [mailto:alexis@REDACTED]
> Sent: 30 January 2001 22:54
> To: erlang-questions@REDACTED
> Cc: prakash@REDACTED
> Subject: Curious behaviour of gen_tcp
> 
> 
> Hello everyone,
> 
> I'm using the following program structure to have an erlang process
> communicate with a socket based server (written in perl). The code is
> simplified on purpose.
> 
> start() ->
>   loop_idle().
> 
> loop_idle(Queue)
>   case not_empty(Queue) of
>     true ->
>       send_and_loop(Queue);
>     _ ->
>       do_nothing
>   end;
> 
>   receive
>     {send, Data} ->
>       send_and_loop(Data);
>     _ ->
>       log_crap
>   end.
> 
> send_and_loop(Queue) ->
>   {ok, Socket} = gen_tcp:connect("localhost", 1, [binary, 
> {packet, 0}]),
>   gen_tcp:send(Socket, Queue:pop()),
>   loop_busy(Queue, Socket, <<>>).
> 
> loop_busy(Queue, Socket, Buffer) ->
>   receive
>     {tcp, Socket, Data} ->
>       loop_busy(Socket, concat_binary([Buffer, Data]);
>     {tcp_closed, Socket} ->
>       io:fwrite("Finished ~w", [length(binary_to_list(Buffer)]),
>       loop_idle(Queue);
>     Any ->
>       io:fwrite("Received ~w~n", [Any])
>   end.
> 
> %%%%%%%%%%%%%
> 
> This works great except that for responses bigger than 70 KB, 
> I don't get
> consistent results, i.e. sometimes, the response received by erlang is
> about the right size (compared to what was sent by the 
> server, always the
> same size), sometimes, it's half or a quarter of the size.
> 
> Am I correct in assuming that the {tcp_closed..} pattern does not
> necessarily match after {tcp, Socket, Data}? I.e. the process 
> will match
> some {tcp, Socket, Data} and then, if {tcp_closed...} has 
> been received,
> it *can* match it and exit the loop *while* there are still 
> {tcp, Socket,
> Data} messages in its mailbox.
> 
> If this is correct, does it mean that I need to spawn a 
> listener process
> which will do gen_tcp:recv() and notify the parent that it 
> has received
> all data?
> 
> I have to say I'm a bit boggled by this one.
> 
> Alexis
> 



NOTICE AND DISCLAIMER:
This email (including attachments) is confidential.  If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents.  We cannot accept liability for any breaches of
confidence arising through use of email.  Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions.  We will not accept responsibility for any commitments
made by our employees outside the scope of our business.  We do not warrant
the accuracy or completeness of such information.