<p>Five minutes certainly sounds like a perplexingly high delay.<br>
Why do you convert your binaries to lists, though? As opposed to either building a binary instead, or an iolist?</p>
<div class="gmail_quote">Den 21/11/2012 19.19 skrev "Richard Evans" <<a href="mailto:richardprideauxevans@gmail.com">richardprideauxevans@gmail.com</a>>:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks Joe. This is very helpful.<div><br></div><div>Now that I have set the packet-size as 4 bytes, this does work. I can happily send the 2 meg binary block from erlang to c.</div><div><br></div><div>But there is a perplexing slow-down which I do not understand. </div>
<div><br></div><div>I am encoding the binary block like this:</div><div><br></div><div>
<p style="text-align:left"><span style="font-family:arial,sans-serif;font-size:x-small">encode({load_instance_from_binary, GameIndex, BinaryBlock}) -></span><span style="font-family:arial,sans-serif;font-size:x-small"> </span><br>
<span style="font-family:arial,sans-serif;font-size:x-small"> </span><span style="font-family:arial,sans-serif;font-size:x-small">Result = lists:append([[?SERVER_INSTRUCTION_LOAD_INSTANCE_FROM_BINARY | convertToFourBytes(GameIndex)], convertToFourBytes(byte_size(BinaryBlock)), binary:bin_to_list(BinaryBlock)]),</span><br>
<span style="font-family:arial,sans-serif;font-size:x-small"> </span><span style="font-family:arial,sans-serif;font-size:x-small">Result.</span><br></p><p style="text-align:left">
</p><p><span style="font-size:x-small">convertToFourBytes(X) -></span><br><span style="font-size:x-small"> </span><span style="font-size:x-small"><<B1,B2,B3,B4>> = <<X:32>>,</span><br><span style="font-size:x-small"> </span><span style="font-size:x-small">[B1,B2,B3,B4].</span><br>
</p>
<p></p>
<p style="text-align:left">This converts the binary block to list format before appending it to the rest of the list. This bin_to_list does not seem to cause any slow-down.</p><p style="text-align:left">The slow-down seems to happen when sending the 2 meg message from erlang to c. In this function, taken from Joe's book:</p>
<p style="text-align:left">
</p><p><font size="1">handle_call(Msg, _From, #state{port=Port}=State) -><br> EncodedMsg = encode(Msg),<br> validate_message(EncodedMsg),<br> Port ! {self(), {command, EncodedMsg}},<br> receive<br> {Port, {data, Data}} -><br>
DecodedData = decode(Data)<br> end,<br> {reply, DecodedData, State};</font><br></p>
<p></p><p style="text-align:left">I have a breakpoint on the c function that corresponds to the message. </p><p style="text-align:left">What perplexes me is that there is a 5 minute delay between the message being sent to the c-port (<span style="font-size:x-small">Port ! {self(), {command, EncodedMsg}}, </span>and the c-function actually getting called!</p>
<p style="text-align:left">Any advice much appreciated.</p><p style="text-align:left">thanks,</p><p style="text-align:left">Richard</p><p><br></p><br>On Saturday, November 10, 2012 1:45:35 PM UTC, Joe Armstrong wrote:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">
The only answer is "try it and see" - it all depends. The computer I'm typing<div>this mail on has 4GB or RAM - compared to that 1M is "nothing".</div><div><br></div><div>Are you running on a 24GB monster or 5MB embedded system? Do you have</div>
<div>one parallel process, sending one 1 MB message, or a thousand? Do you send</div><div>your 1 MB messages once every millisecond - or once a year.</div><div><br></div><div>Just knowing that your message is 1MB tells me nothing about the other parts</div>
<div>of your system - when GB memories came I stopped calling MBytes "large"</div><div>and TB disks means that GB files are not really "large" - these are relative terms.</div><div>Today Peta bytes is "large" - but we get a 1000x scale change every ten years</div>
<div><br></div><div>There no intrinsic reason why it should not work - Just make sure the packet length</div><div>will fit into 4 bytes and use {packet,4}.</div><div><br></div><div>I have a file replication system in Erlang - I send all small files in single messages</div>
<div>(I think the cut-off is 10 MB) and larger files in chunks (mainly so I can restart them</div><div>if things go wrong)</div><div><br></div><div>Oh and a port-driver should be fine for this.</div><div><br></div><div>Cheers</div>
<div><br></div><div>[aside] performance always surprises me - I think reading a few hundred small files</div><div>will take a long time - but it takes a blink of a second - I guess this is because</div><div>it would take me a heck of a long time to do this - GHz clock speeds are so</div>
<div>fast that just about everything I think would take a long time doesn't.</div><div><br></div><div><br></div><div><br></div><div>/Joe</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div>
<div><br></div><div><br></div><div><br><br><div class="gmail_quote">On Fri, Nov 9, 2012 at 3:31 PM, Richard Evans <span dir="ltr"><<a>richardpri...@gmail.<u></u>com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I am using a port to communicate between c and erlang, as described in Joe Armstrong's excellent book, and in this tutorial: <a href="http://www.erlang.org/doc/tutorial/c_port.html#id63121" style="white-space:pre-wrap" target="_blank">http://www.erlang.<u></u>org/doc/tutorial/c_port.html#<u></u>id63121</a><div>
<br></div><div>So far, the traffic between the two has been pretty small: typically a hundred bytes or so at a time. </div><div><br></div><div>But now I need to (occasionally) send a large block of binary data from c to erlang: up to 1 meg. <br>
<div><div><br></div><div>How should I handle such large blocks of binary data? Is there anything wrong with increasing the size of the buffer (called buf in the tutorial mentioned above) to 1 meg? </div><div><br></div><div>
Is there a better way of doing this? (NB I am using a port, rather than a port-driver (in which the c-code is linked directly to the erlang code)).</div><div><br></div><div>thanks,</div><div>Richard Evans</div>
</div></div>
<br>______________________________<u></u>_________________<br>
erlang-questions mailing list<br>
<a>erlang-q...@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/<u></u>listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div>