<div dir="ltr">Just noticed that my phone email decided not to reply-all.<br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 27, 2013 at 1:32 PM, Robert Raschke <span dir="ltr"><<a href="mailto:rtrlists@googlemail.com" target="_blank">rtrlists@googlemail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Peter,<br><br></div>that sounds OK, I guess. It's been ages since I spent any time in the deep down details of how the receive/send works, so It's difficult for me to say for certain that a mutex around each explicit send is enough. Your theory of the receive doing a send internally without the protection of a mutex sounds reasonable.<br>
<br></div>I don't think you can handle the TICK messages yourself, but you might dig in to see if that's possible.<br><br></div>Another way that comes to mind is to wrap both receive and send in the same mutex and have receive be "non-blocking" via the timeout version of the function.<br>
<br></div>HTH,<br>Robby<br><br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Aug 27, 2013 at 9:50 AM, Peter Membrey <span dir="ltr"><<a href="mailto:peter@membrey.hk" target="_blank">peter@membrey.hk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif"><div>Hi Robert,<br></div><div><br></div>
<div>I have a thread pool that receives data from another C library and generates messages for sending to Erlang. Each message is atomic and self contained, so it's not a data stream as such, just a bunch of messages. These can be received and handled in any order at the Erlang side so that should be okay I'd have thought. The mutex protects ei_send() with the intention of ensuring that only one thread can send a message at a time. Of course this assumes that when ei_send() returns, the message has actually been sent...</div>
<div><br></div><div>I get data corruption on the Erlang node where it seems as though messages have become mixed up or inter-mingled. All the ei_send() that I do are protected by mutex, and the the main thread of the application simply blocks on ei_xreceive() - so in theory it could reply to a heart beat at any time which maybe gets interspersed with sends from the call back threads because it is not protected by the mutex.</div>
<div><br></div><div>I can't think of any other way to do this, so I'd greatly appreciate any guidance you can give me...</div><div><br></div><div>Kind regards,</div><div><br></div><div>Peter Membrey</div><div><br>
</div><hr><div style="font-size:12pt;font-style:normal;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal"><b>From: </b>"Robert Raschke" <<a href="mailto:rtrlists@googlemail.com" target="_blank">rtrlists@googlemail.com</a>><br>
<b>To: </b>"Peter Membrey" <<a href="mailto:peter@membrey.hk" target="_blank">peter@membrey.hk</a>><br><b>Sent: </b>Tuesday, 27 August, 2013 4:39:01 PM<br><b>Subject: </b>Re:[erlang-questions] Is ei_xreceive_msg() thread safe?<div>
<div><br><div><br></div><p>You need to call ei_xreceive_msg() regularly, it only handles the pings while it is running. Is it possible that you are losing the connection to Erlang because of that (see the discussion about net_ticktime in the kernel app documentation) ?</p>
<p>But I also note you say you are starting threads to handle work. Is each one of them simply sending data to Erlang? Wouldn't that mean you're sending randomly interspersed pieces of data? I think you may have to investigate handling the comms back to Erlang in more detail. This may depend a lot on your use case. But simply mutexing the sends doesn't sound right to me.</p>
<p>Robby<br>
</p>
<div class="gmail_quote">On Aug 27, 2013 8:26 AM, "Peter Membrey" <<a href="mailto:peter@membrey.hk" target="_blank">peter@membrey.hk</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi guys,<br>
<br>
I've got a fairly basic C Node set up where I have the main thread running in a loop with ei_xreceive_msg() and a number of "callback" threads that execute functions and write data using ei_send() to the shared socket (connecting to the Erlang node).<br>
<br>
Originally I had a lot of data corruption (the Erlang node crashing due to corrupt data) because of incorrect locking on socket writes. I added mutexes to the ei_send() calls and this problem seemed to go away.<br>
<br>
However I've had a couple of occasions where the system has been quiet and then suddenly become busy where corrupt data has still been sent to the Erlang node. I'm positive all the places where I do ei_send() are protected, but that got me wondering about ei_xreceive_msg().<br>
<br>
>From what I can find, ei_xreceive_msg() automatically handles heartbeats for you and I guess that means it will send some sort of reply on that socket. If the heart beat is being sent at the same time as some other process tries to write to the socket, is it possible that the two could get interleaved or something? I would honestly have thought it unlikely but I'm running out of ideas.<br>
<br>
Assuming it's possible, how could I add a mutex in this case? The call itself blocks, so I can't wrap the whole call in the mutex else nothing else will be able to send data, and there's no way to pass a mutex into the call itself. So as far as I can tell, there's no way to protect these writes and prevent them from getting mixed up with other writes on that socket.<br>
<br>
Does anyone have any ideas? I'm quite willing to accept I could be doing something pretty stupid, but I'm really out of ideas as to what that might be...<br>
<br>
Thanks in advance!<br>
<br>
Kind Regards,<br>
<br>
Peter Membrey<br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote></div>
</div></div></div><div><br></div></div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>