<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 08/27/2013 05:27 PM, Peter Membrey
wrote:<br>
</div>
<blockquote
cite="mid:1396147141.1692.1377617279511.JavaMail.zimbra@membrey.hk"
type="cite">
<pre wrap="">Hi Serge,
Thank you very much for clarifying this! At least I know where everything stands now.
Does the scenario I'm talking about make sense? Could a heartbeat in the current design end up interleaved with other data? I suspect it would be pretty rare, but we pump 10 million or more messages through this app per day, so even if the chance was a million to one, we'd end up seeing it ten times a day on average...
</pre>
</blockquote>
<small>Yes, it make sense. The ei_send/receive API is not thread
safe for the same file descriptor.<br>
</small><br>
<blockquote
cite="mid:1396147141.1692.1377617279511.JavaMail.zimbra@membrey.hk"
type="cite">
<pre wrap="">
I'm thinking for now that I'll give Robert's suggestion a try. If I call ei_xreceive() with a timeout and wrap a mutex around it, would that then protect the socket? I'm assuming (probably a bad thing) that it doesn't reply in a different thread or something...
I guess it would be a bit wasteful to effectively poll the socket every millisecond or so (I can't really block the other threads from writing longer than that) but it seems to be a potential way to resolve this issue in the short term while I come up with a better fix.
Does that sound like a reasonable plan?
</pre>
</blockquote>
<small>You could use select() to wait for a message to arrive on the
socket.<br>
Then lock your mutex and call ei_xreceive().<br>
<br>
It might not be such a good idea to poll by repeated calls to
ei_xreceive() with a short timeout. You can get situations when
the timeout triggers after receiving half a message. That will
make all following calls to ei_xreceive() get out of synch and
probably fail. I guess the probability of this scenario depends on
the message sizes and the speed and reliability of your
connection.<br>
<br>
Calling ei_xreceive() without a timeout and my suggestion with
select() on the other hand could make a hickup on the net to cause
all your sending threads to freeze waiting for the mutex.<br>
<br>
/Sverker<br>
<br>
<br>
<br>
</small>
<blockquote
cite="mid:1396147141.1692.1377617279511.JavaMail.zimbra@membrey.hk"
type="cite">
<pre wrap=""><small>
</small>Thanks again!
Kind Regards,
Peter Membrey
----- Original Message -----
From: "Serge Aleynikov" <a class="moz-txt-link-rfc2396E" href="mailto:serge@aleynikov.org"><serge@aleynikov.org></a>
To: "Peter Membrey" <a class="moz-txt-link-rfc2396E" href="mailto:peter@membrey.hk"><peter@membrey.hk></a>
Cc: "Erlang Questions" <a class="moz-txt-link-rfc2396E" href="mailto:erlang-questions@erlang.org"><erlang-questions@erlang.org></a>
Sent: Tuesday, August 27, 2013 10:59:27 PM
Subject: Re: [erlang-questions] Is ei_xreceive_msg() thread safe?
e
i_send/receive family of functions are not thread safe, and receive functions do handle heartbeats internally. The way the ei_{connect,send,receive} functions are written it's not possible to do any non-blocking I/O with them or use them in multi-threaded code in the manner you described without modifying the functions.
You can take a look at the alternative C++ library that doesn't have such limitations:
<a class="moz-txt-link-freetext" href="https://github.com/saleyn/eixx">https://github.com/saleyn/eixx</a>
, and offers almost all functionality that ei has.
Regards,
Serge
On Tue, Aug 27, 2013 at 3:30 AM, Peter Membrey < <a class="moz-txt-link-abbreviated" href="mailto:peter@membrey.hk">peter@membrey.hk</a> > wrote:
Hi guys,
I've got a fairly basic C Node set up where I have the main thread running in a loop with ei_xreceive_msg() and a number of "callback" threads that execute functions and write data using ei_send() to the shared socket (connecting to the Erlang node).
Originally I had a lot of data corruption (the Erlang node crashing due to corrupt data) because of incorrect locking on socket writes. I added mutexes to the ei_send() calls and this problem seemed to go away.
However I've had a couple of occasions where the system has been quiet and then suddenly become busy where corrupt data has still been sent to the Erlang node. I'm positive all the places where I do ei_send() are protected, but that got me wondering about ei_xreceive_msg().
>From what I can find, ei_xreceive_msg() automatically handles heartbeats for you and I guess that means it will send some sort of reply on that socket. If the heart beat is being sent at the same time as some other process tries to write to the socket, is it possible that the two could get interleaved or something? I would honestly have thought it unlikely but I'm running out of ideas.
Assuming it's possible, how could I add a mutex in this case? The call itself blocks, so I can't wrap the whole call in the mutex else nothing else will be able to send data, and there's no way to pass a mutex into the call itself. So as far as I can tell, there's no way to protect these writes and prevent them from getting mixed up with other writes on that socket.
Does anyone have any ideas? I'm quite willing to accept I could be doing something pretty stupid, but I'm really out of ideas as to what that might be...
Thanks in advance!
Kind Regards,
Peter Membrey
_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
<br>
</body>
</html>