[erlang-questions] Sending 2.5 MB message with com.ericsson.otp.erlang

[CGS]

Just to give an example, consider a large file encoded in base 64. Each 
data packet should contain:
1. header (fixed size): ID + total number of packets + position + packet 
length;
3. data: packet length x 64 chars.
When arriving, cut the header, align the data in a string based on 
packet position and decode the string. The advantage is that you know 
what characters you can expect from the packet and you can quantify the 
file in pieces of well known size. Also, you can send/receive the data 
through parallel threads because you will know when it's completed.

The drawbacks of such a method are:
1. using more computational resources;
2. adding more data to the transfer.
So, you will need to find the optimum size for your packets.

I hope this method will give you some ideas at least.

Cheers,
CGS




On 10/27/2011 04:33 PM, CGS wrote:
> Sorry, it's base 64, not 64 bits. my mind was still "compiling" 
> something else when I started to write that post. My apologies.
>
> CGS
>
>
>
> On 10/27/2011 04:07 PM, CGS wrote:
>> Encode your document in 64 bits (e.g., 
>> http://www.erlang.org/doc/man/base64.html) and slice it where you 
>> think it's optimal. For JAVA, I don't remember if there is or not 
>> such encoding. Nevertheless, the idea is used in messaging (and not 
>> only), where large quantity of data (e.g., file transfer in between 
>> users) is transfered.
>>
>> CGS
>>
>>
>> On 10/27/2011 03:41 PM, Martin Dimitrov wrote:
>>> Thanks for the suggestion but how do I  send chucked Erlang tuple? The
>>> OtpMbox.send() method doesn't send bytes.
>>>
>>> Martin
>>>
>>> On 10/27/2011 12:03 PM, CGS wrote:
>>>> Hi Martin,
>>>>
>>>> I suppose you can quantify your data in smaller chunks optimized for
>>>> transfer speed. That's one idea, but I am sure there are other ideas
>>>> which may be even better.
>>>>
>>>> Cheers,
>>>> CGS
>>>>
>>>>
>>>>
>>>> On 10/27/2011 10:46 AM, Martin Dimitrov wrote:
>>>>> Hello,
>>>>>
>>>>> In out app, the Java node sends messages to Erlang through
>>>>> com.ericsson.otp.erlang package. In Java we create an OtpNode as:
>>>>>
>>>>> OtpNode oNode = new OtpNode(NODE);
>>>>> oNode.setCookie(COOKIE);
>>>>> OtpMbox mbox = oNode.createMbox(MBOX);
>>>>>
>>>>> and then send using
>>>>>
>>>>> mbox.send(OtpErlangPid, OtpErlangTuple);
>>>>>
>>>>> All is well when sending small in size messages (several KB). But 
>>>>> when
>>>>> sending a relatively large message (2.5MB) it takes 8-10 seconds. The
>>>>> interesting thing is that the time it takes does not linearly 
>>>>> progress
>>>>> with the size of the message: 250KB message took around 100
>>>>> milliseconds.
>>>>>
>>>>> Does anyone know ways to optimize the transfer?
>>>>>
>>>>> Thanks
>>>>>
>>>>> Martin
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED
>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>