[erlang-questions] (ArgumentError) argument error :erlang.binary_to_term

Sun Jan 29 11:50:45 CET 2017

No idea what's wrong - but you could start checking that

1) binary_to_term works locally (it should)
2) the data is transmitted correctly
    print the md5 checksum of the binary before sending and after reception
    erlang has a BIF for this erlang:md5

/Joe


On Sat, Jan 28, 2017 at 1:55 AM, Kevin Johnson
<johnson786.kevin@REDACTED> wrote:
> Hi,
>
> I am using Elixir to send data using :gen_tcp. The syntax presented here may
> be elementary Elixir syntax, however the issue is specifically related to
> the usage of term_to_binary and binary_to_term conversions over gen_tcp.
>
> This is the command I used on the client side:
>>
>> :gen_tcp.send(client_socket, :erlang.term_to_binary(data))
>
>
> This is the command I use on the server side:
>>
>>   defp handle_data(data) do
>>     data
>>     |> :erlang.binary_to_term
>>   end
>
>
> When my data gets received on the server side, it seems that at a certain
> size threshold that it errors out like this:
>
>> pry(6)> [error] GenServer #PID<0.502.0> terminating
>>
>> ** (ArgumentError) argument error
>>
>>  :erlang.binary_to_term(<<131, 116, 0, 0, 0, 28, 109, 0, 0, 0, 5, 66, 49,
>> 95, 81, 49, 108, 0, 0, 0, 1, 116, 0, 0, 0, 1, 109, 0, 0, 0, 4, 116, 101,
>> 120, 116, 109, 0, 0, 0, 11, 84, 69, 88, 84, 32, 65, 78, 83, 87, 69, ...>>)
>>
>> (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
>>
>> (stdlib) gen_server.erl:667: :gen_server.handle_msg/5
>>
>> (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
>>
>> Last message: {:tcp, #Port<0.10115>, <<131, 116, 0, 0, 0, 28, 109, 0, 0,
>> 0, 5, 66, 49, 95, 81, 49, 108, 0, 0, 0, 1, 116, 0, 0, 0, 1, 109, 0, 0, 0, 4,
>> 116, 101, 120, 116, 109, 0, 0, 0, 11, 84, 69, 88, 84, 32, 65, 78, 83, ...>>}
>
>
> Here is some example data that will just pass fine:
> %{"B2_Q18" => [%{"choice_id" => "B2_Q18_C1"}], "B3_B9_Q1" => [%{"choice_id"
> => "B3_B9_Q1_C1"}], "B2_Q7" => [%{"choice_id" => "B2_Q7_C1"}], "B5_B1_Q8" =>
> [%{"choice_id" => "B5_B1_Q8_C1"}], "B3_B6_Q1" => [%{"choice_id" =>
> "B3_B6_Q1_C1"}], "B5_B2_Q5" => [%{"choice_id" => "B5_B2_Q5_C1"},
> %{"choice_id" => "B5_B2_Q5_C2"}], "B3_B7_Q1" => [%{"choice_id" =>
> "B3_B7_Q1_C1"}], "B3_B4_Q1" => [%{"choice_id" => "B3_B4_Q1_C1"}], "B2_Q16"
> => [%{"choice_id" => "B2_Q16_C1"}], "B2_Q20" => [%{"choice_id" =>
> "B2_Q20_C1"}], "B2_Q5" => [%{"choice_id" => "B2_Q5_C1"}], "B2_Q9" =>
> [%{"choice_id" => "B2_Q9_C1"}], "B2_Q17" => [%{"choice_id" => "B2_Q17_C1"}],
> "B3_B2_Q1" => [%{"choice_id" => "B3_B2_Q1_C1"}], "B5_B2_Q3" =>
> [%{"choice_id" => "B5_B2_Q3_C1"}], "B3_B3_Q1" => [%{"choice_id" =>
> "B3_B3_Q1_C1"}], "B5_B1_Q1" => [%{"text" => "TEXT ANSWER"}], "B2_Q14" =>
> [%{"choice_id" => "B2_Q14_C1"}], "B5_B1_Q3" => [%{"choice_id" =>
> "B5_B1_Q3_C22"}], "B5_B2_Q2" => [%{"choice_id" => "B5_B2_Q2_C1"}], "B2_Q4"
> => [%{"choice_id" => "B2_Q4_C1"}], "B2_Q10" => [%{"choice_id" =>
> "B2_Q10_C1"}], "B4_Q1" => [%{"choice_id" => "B4_Q1_C1"}, %{"choice_id" =>
> "B4_Q1_C10"}], "B5_B1_Q6" => [%{"choice_id" => "B5_B1_Q6_C1"}], "B2_Q6" =>
> [%{"choice_id" => "B2_Q6_C1"}], "B2_Q2" => [%{"choice_id" => "B2_Q2_C1"}],
> "B2_Q1" => [%{"choice_id" => "B2_Q1_C1"}]}
>
> The moment I make it slightly longer, by like appending an extra key "a" =>
> "b" it will give me the above error.
>
> It seems to be that the size of the keys plays an issue here, because if I
> convert all the above "choice_id" keys to a mere "c", then even with the
> extra entry "a" => "b" included like following below will work just fine:
> %{"B2_Q18" => [%{"c" => "B2_Q18_C1"}], "B3_B9_Q1" => [%{"c" =>
> "B3_B9_Q1_C1"}], "B2_Q7" => [%{"c" => "B2_Q7_C1"}], "B5_B1_Q8" => [%{"c" =>
> "B5_B1_Q8_C1"}], "B3_B6_Q1" => [%{"c" => "B3_B6_Q1_C1"}], "B5_B2_Q5" =>
> [%{"c" => "B5_B2_Q5_C1"}, %{"c" => "B5_B2_Q5_C2"}], "B3_B7_Q1" => [%{"c" =>
> "B3_B7_Q1_C1"}], "B3_B4_Q1" => [%{"c" => "B3_B4_Q1_C1"}], "B2_Q16" => [%{"c"
> => "B2_Q16_C1"}], "B2_Q20" => [%{"c" => "B2_Q20_C1"}], "B2_Q5" => [%{"c" =>
> "B2_Q5_C1"}], "B2_Q9" => [%{"c" => "B2_Q9_C1"}], "B2_Q17" => [%{"c" =>
> "B2_Q17_C1"}], "B3_B2_Q1" => [%{"c" => "B3_B2_Q1_C1"}], "B5_B2_Q3" => [%{"c"
> => "B5_B2_Q3_C1"}], "B3_B3_Q1" => [%{"c" => "B3_B3_Q1_C1"}], "B5_B1_Q1" =>
> [%{"text" => "TEXT ANSWER"}], "B2_Q14" => [%{"c" => "B2_Q14_C1"}],
> "B5_B1_Q3" => [%{"c" => "B5_B1_Q3_C22"}], "B5_B2_Q2" => [%{"c" =>
> "B5_B2_Q2_C1"}], "B2_Q4" => [%{"c" => "B2_Q4_C1"}], "B2_Q10" => [%{"c" =>
> "B2_Q10_C1"}], "B4_Q1" => [%{"c" => "B4_Q1_C1"}, %{"c" => "B4_Q1_C10"}],
> "B5_B1_Q6" => [%{"c" => "B5_B1_Q6_C1"}], "B2_Q6" => [%{"c" => "B2_Q6_C1"}],
> "B2_Q2" => [%{"c" => "B2_Q2_C1"}], "B2_Q1" => [%{"c" => "B2_Q1_C1"}], "a" =>
> "b"}
>
> The following on the other hand will just fail:
> data =
> %{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
> => "q",
> "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
> => "q",
> "ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
> => "q"}
> :gen_tcp.send(client_socket, :erlang.term_to_binary(data))
>
> The only issue that I was able to track down here is a conjunction of two
> things:
> 1) A limit to the size of total key length inside a map/hash
> 2) sending that hash as a binary over tcp and converting back to term
>
> In all of the above cases, a direct conversion of :erlang.term_to_binary
> followed by :erlang.binary_to_term works just fine without any issues. The
> issue only comes about after that binary was send over tcp and then
> :erlang.binary_to_term is attempted.
>
> I would greatly appreciate if anyone can provide me guidance in this matter,
> point out to me any relevant documentation that officially states any of
> these constraints if they are in fact constraints, and if they are not
> supposed to be constraints then kindly instruct me what additional
> information may be needed for me to get to the bottom of this.
>
> Thank you.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>