From xpdoka-dated-1147117921.33483b@REDACTED Wed May 3 21:52:01 2006 From: xpdoka-dated-1147117921.33483b@REDACTED (Dominic Williams) Date: Wed, 03 May 2006 21:52:01 +0200 Subject: Warning term [return]ed by compile:file/2 changed in R10B Message-ID: <445909E1.8000408@dominicwilliams.net> Hello, The format of the warnings term returned by: compile:file(foo,[return]) has changed in R10B, and is never empty even when there are no warnings. E.g. on a file that compiles without warnings, previous versions returned: {ok, foo, []} while R10B returns: {ok,foo,[{"./foo.erl",[]},{"./foo.erl",[]},{"./foo.erl",[]}]} E.g. on a file that compiles with several warnings: -module(warn). -export([foo/1]). foo(Arg) -> ok. bar(Arg) -> Arg; bar(5) -> hello. baz(Arg) -> fun(Arg) -> Arg end. Previous versions returned: {ok,warn, [{"./warn.erl", [{9,erl_lint,{shadowed_var,'Arg','fun'}}, {11,erl_lint,{unused_function,{bar,1}}}, {11,erl_lint,{unused_function,{baz,1}}}]}]} while R10B returns the more complicated: {ok,warn, [{"./warn.erl", [{4,erl_lint,{unused_var,'Arg'}}, {12,erl_lint,{unused_var,'Arg'}}, {13,erl_lint,{shadowed_var,'Arg','fun'}}, {17,erl_lint,{unused_function,{bar,1}}}, {17,erl_lint,{unused_function,{baz,1}}}]}, {"./warn.erl",[]}, {"./warn.erl",[{9,sys_core_fold,{nomatch_shadow,7}}]}, {"./warn.erl",[]}]} (It's not that R10B warns about more things that is the problem, it's that the warnings are spread across several lists, some of which are returned even if they are empty.) Regards, Dominic Williams http://www.dominicwilliams.net ---- From xpdoka-dated-1147118103.c55bea@REDACTED Wed May 3 21:54:22 2006 From: xpdoka-dated-1147118103.c55bea@REDACTED (Dominic Williams) Date: Wed, 03 May 2006 21:54:22 +0200 Subject: erlang.el (emacs) problems Message-ID: <44590A6E.2090400@dominicwilliams.net> Hello, I ran into a couple of problems with erlang.el (emacs mode). Erlang Mode Version 2.5.2 Emacs version 21.4.1 1) Calling erlang-find-tag (M-.) with a module:tag form of argument results in an error: Symbol's value as variable is void: tags-table-format-hooks This is solved for my emacs by replacing the calls to tags-table-format-hooks by calls to tags-table-format-functions. 2) With font-lock-maximum-decoration set to 3, hexadecimal literals are incorrectly highlighted as variables (e.g. in 16#C5, C5 is highlighted as though it were a variable). Regards, Dominic Williams http://www.dominicwilliams.net ---- From gunilla@REDACTED Thu May 4 12:08:08 2006 From: gunilla@REDACTED (Gunilla Arendt) Date: Thu, 04 May 2006 12:08:08 +0200 Subject: erlang.el (emacs) problems References: <44590A6E.2090400@dominicwilliams.net> Message-ID: Thanks for pointing this out. / Gunilla Erlang/OTP team Dominic Williams wrote: > Hello, > > I ran into a couple of problems with erlang.el (emacs mode). > > Erlang Mode Version 2.5.2 > Emacs version 21.4.1 > > 1) Calling erlang-find-tag (M-.) with a module:tag form of argument > results in an error: > Symbol's value as variable is void: tags-table-format-hooks > > This is solved for my emacs by replacing the calls to > tags-table-format-hooks by calls to tags-table-format-functions. > > 2) With font-lock-maximum-decoration set to 3, hexadecimal > literals are incorrectly highlighted as variables (e.g. in 16#C5, > C5 is highlighted as though it were a variable). > > Regards, > > Dominic Williams > http://www.dominicwilliams.net > > ---- From Hans.Bolinder@REDACTED Thu May 4 14:00:54 2006 From: Hans.Bolinder@REDACTED (Hans.Bolinder@REDACTED) Date: Thu, 4 May 2006 14:00:54 +0200 Subject: Warning term [return]ed by compile:file/2 changed in R10B In-Reply-To: <445909E1.8000408@dominicwilliams.net> References: <445909E1.8000408@dominicwilliams.net> Message-ID: <17497.60662.86778.603767@gargle.gargle.HOWL> [Dominic Williams:] > The format of the warnings term returned by: > > compile:file(foo,[return]) > > has changed in R10B, and is never empty even when there are no warnings. > E.g. on a file that compiles without warnings, previous versions returned: > > {ok, foo, []} > > while R10B returns: > > {ok,foo,[{"./foo.erl",[]},{"./foo.erl",[]},{"./foo.erl",[]}]} The old behaviour will be back in the upcoming R11B release. Best regards, Hans Bolinder, Erlang/OTP From rlenglet@REDACTED Fri May 12 05:46:25 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Fri, 12 May 2006 12:46:25 +0900 Subject: erl_interface: potential buffer overflow in every call to a ei_decode_* function Message-ID: <200605121246.25629.rlenglet@users.forge.objectweb.org> Hello, No ei_decode_* function takes a buffer size as an argument. Therefore, when decoding wrong data, those functions may read after the end of the buffer. This may be detected afterwards, by comparing the returned index with the buffer size, but an access out of the buffer bounds may already have provoked a segmentation fault before the decoding function call returns. The solution is to add a buffer size argument to every ei_decode_* function, and to compare it with the index for every buffer access. Regards, -- Romain LENGLET From rlenglet@REDACTED Fri May 12 07:59:40 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Fri, 12 May 2006 14:59:40 +0900 Subject: [patch] add a 'dont_pack_binaries' option to open_port/2 to disable packing small binaries in port_command/2 Message-ID: <200605121459.41128.rlenglet@users.forge.objectweb.org> Hello, Here is a patch that adds an option to open_port/2 that disables binaries packing in the implementation of port_command/2. Packing is normally triggered when more than 16 binaries are passed in the io-list. In that case, contiguous "small enough" binaries are copied together into single buffers. With this patch, when this option is set, packing is never triggered, ensuring that all binaries are passed by reference and are never copied. Preventing packing is necessary when the boundaries between binaries must be preserved. This patch is against the otp_src_R10B_2006-02-27 snapshot. I have no patch for the documentation, but I suggest adding this comment into the documentation about open_port/2, in the 'erlang' module documentation: dont_pack_binaries Prevents packing (copying/grouping of small binaries in io-lists into larger buffers) in calls to port_command/2. Regards, -- Romain LENGLET -------------- next part -------------- A non-text attachment was scrubbed... Name: otp-dontpackbins_2006-05-12.patch Type: text/x-diff Size: 3473 bytes Desc: not available URL: From raimo@REDACTED Fri May 12 09:11:38 2006 From: raimo@REDACTED (Raimo Niskanen) Date: 12 May 2006 09:11:38 +0200 Subject: [patch] add a 'dont_pack_binaries' option to open_port/2 to disable packing small binaries in port_command/2 References: <200605121459.41128.rlenglet@users.forge.objectweb.org> Message-ID: I think your suggestion is an excellent idea. Unfortunately it will not make it into the R11B first release, but probably the next. Thank you for the patch! > Packing is normally triggered when more than 16 binaries are > passed in the io-list. In that case, contiguous "small enough" > binaries are copied together into single buffers. > With this patch, when this option is set, packing is never > triggered, ensuring that all binaries are passed by reference > and are never copied. > > Preventing packing is necessary when the boundaries between > binaries must be preserved. > > This patch is against the otp_src_R10B_2006-02-27 snapshot. > > I have no patch for the documentation, but I suggest adding this > comment into the documentation about open_port/2, in > the 'erlang' module documentation: > > dont_pack_binaries > Prevents packing (copying/grouping of small binaries in > io-lists into larger buffers) in calls to port_command/2. > > > Regards, > > -- > Romain LENGLET > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rlenglet@REDACTED Mon May 15 04:46:50 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Mon, 15 May 2006 11:46:50 +0900 Subject: [patches] erl_interface: bug fixes, ei_print_term and show_term merge, and new ei_x_decode_* functions In-Reply-To: <200605121246.25629.rlenglet@users.forge.objectweb.org> References: <200605121246.25629.rlenglet@users.forge.objectweb.org> Message-ID: <200605151146.50694.rlenglet@users.forge.objectweb.org> Hello, Please review the three attached patches, to apply in the following order, against the otp_src_R10B_2006-02-27 snapshot / erl_interface version 3.5.4: - 01_ei_bugfixes.patch: Corrects a number of bugs: - in send_exit.c: malloc(len) instead of malloc(index) (index is always 0 at that point!), which made the function always return -1 when len>EISMALLBUF; - when decoding references, only the first 3 bytes are read, but the index must be increment of the total size, not only 3; - in ei_skip_term, the decoding of ERL_LARGE_BIG_EXT was boggus (the length returned by ei_get_type_internal in that case is *not* the length of the encoded term!); - in ei_decode_trace, the index may have been updated even if the case decoding fails in the middle of decoding; - same problem in print_term, for the decoding of ERL_LIST_EXT, ERL_SMALL_TUPLE_EXT and ERL_LARGE_TUPLE_EXT; - in show_term, replace calls to ei_decode_skip_bignum by calls to ei_skip_term, which does the same job; remove ei_decode_skip_bignum. Also adds some safety tests: - when decoding a double (ERL_FLOAT_EXT), check that the last of the 31 bytes is a \0, to make sure that the sscanf does not scan after those 31 bytes; - when decoding a fun, check that the buffer allocated for free variables is not NULL; - when decoding a fun, check consistency of free variables data length. - 02_ei_printtermshowtermmerged.patch: Merges show_term into ei_print_term, and keeps the best of both functions. Modifies the output format for binaries, atoms, pids, ports, refs, strings, and bignums to better fit erlang's *_to_list bifs and io_lib's functions. Replaces all uses of show_term by uses of ei_print_term. - 03_ei_xdecode.patch: Adds a ei_x_decode_* variant for every ei_decode_* function (resp. ei_x_get_type* / ei_get_type*), which takes an ei_x_buff *x argument instead of a const char *buf, int *index pair. This allows to check the index against the buffer size (x->buffsz) to prevent reading after the buffer end. The ei_x_decode_* variants simply perform tests before every read (every get8(s), get16be(s), etc.), using a new check_get macro defined in putget.h. The ei_decode_* unsafe variants are now implemented as stubs that call their corresponding ei_x_decode_* variant, in src/legacy/ei_decode_legacy.c. The internal functions ei_show_recmsg and ei_show_sendmsg have been modified to call ei_x_decode_* instead of ei_decode_*, and this change is reflected in their new signature (they take ei_x_buff * arguments instead of const char *). Uses of ei_show_recmsg and ei_show_sendmsg have been accordingly modified in src/connect/*.c and src/not_used/*.c. All code in src/decode/* and src/misc/* now uses the new ei_x_decode_* variants. Some "slave work" remains to be done, to replace all uses of ei_decode_* by uses of ei_x_decode_* in src/connect/*, src/prog/*, and src/registry/*. I have checked that all patches compile correctly, without introducing new warnings. Regards, -- Romain LENGLET -------------- next part -------------- A non-text attachment was scrubbed... Name: 03_ei_xdecode.patch.gz Type: application/x-gzip Size: 13416 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 01_ei_bugfixes.patch.gz Type: application/x-gzip Size: 3296 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 02_ei_printtermshowtermmerged.patch.gz Type: application/x-gzip Size: 4487 bytes Desc: not available URL: From rlenglet@REDACTED Mon May 15 06:01:59 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Mon, 15 May 2006 13:01:59 +0900 Subject: [patches] erl_interface: bug fixes, ei_print_term and show_term merge, and new ei_x_decode_* functions In-Reply-To: <200605151146.50694.rlenglet@users.forge.objectweb.org> References: <200605121246.25629.rlenglet@users.forge.objectweb.org> <200605151146.50694.rlenglet@users.forge.objectweb.org> Message-ID: <200605151301.59258.rlenglet@users.forge.objectweb.org> > Please review the three attached patches, to apply in the > following order, against the otp_src_R10B_2006-02-27 snapshot > / erl_interface version 3.5.4: I just checked that those patches also apply cleanly against the otp_src_R10B_2006-05-15 snapshot / erl_interface version 3.5.5. -- Romain LENGLET Pr. Chiba Shigeru Group Dept. of Mathematical and Computing Sciences Tokyo Institute of Technology From rlenglet@REDACTED Sat May 27 07:04:50 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Sat, 27 May 2006 14:04:50 +0900 Subject: [PATCH] erl_interface: wrong decoding of strings Message-ID: <200605271404.51139.rlenglet@users.forge.objectweb.org> Hi, When the length of a string is > 65535, it is encoded as a flat list of small integers. A NIL tail is (very logically!) encoded as part of the encoded string in that case, as demonstrated by the attached Erlang module. In erl_interface, the implementation of ei_decode_string handles both cases. However, the integers list case is wrongly handled: the NIL tail is not decoded as it should. Attached are two versions of my patch for that bug: - ei_decode_string.patch is to be applied against erl_interface version 3.5.5. - ei_x_decode_string.patch must be applied after the three patches that I sent on 2006-05-15, to patch the ei_x_decode_string function instead. http://www.erlang.org/ml-archive/erlang-bugs/200605/msg00007.html By the way, none of those patches has been applied in the latest snapshots. Any prediction about when they will be applied (if ever)? Regards, -- Romain LENGLET -------------- next part -------------- -module(teststring). -export([start/0]). start() -> test_string_encoding(70000), test_string_encoding(10). test_string_encoding(Size) -> String = string:copies("b", Size), Bin = term_to_binary(String), if Size > 65535 -> % 65535==MAX_STRING_LEN in externals.c analyze_list(Bin); true -> analyze_string(Bin) end. analyze_string(Bin) -> <<131, 107, Size:16, Bin2/binary>> = Bin, io:format("string size: ~w~n", [Size]), Bin3 = read_chars(Size, Bin2), 0 = size(Bin3). % ***** No NIL tail. ***** analyze_list(Bin) -> <<131, 108, Size:32, Bin2/binary>> = Bin, io:format("list size: ~w~n", [Size]), Bin3 = read_small_ints(Size, Bin2), <<106>> = Bin3. % ***** There is a NIL tail at the end! ***** read_chars(0, Bin) -> Bin; read_chars(Count, Bin) -> <<$b, Bin2/binary>> = Bin, read_chars(Count-1, Bin2). read_small_ints(0, Bin) -> Bin; read_small_ints(Count, Bin) -> <<97, $b, Bin2/binary>> = Bin, read_small_ints(Count-1, Bin2). -------------- next part -------------- A non-text attachment was scrubbed... Name: ei_decode_string.patch Type: text/x-diff Size: 402 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ei_x_decode_string.patch Type: text/x-diff Size: 642 bytes Desc: not available URL: From rlenglet@REDACTED Mon May 29 08:55:29 2006 From: rlenglet@REDACTED (Romain Lenglet) Date: Mon, 29 May 2006 15:55:29 +0900 Subject: [PATCH] erl_interface: added tests for index>=0 Message-ID: <200605291555.29365.rlenglet@users.forge.objectweb.org> Hi, Please apply this patch that adds safety tests that the input index is >= 0 in the ei_x_decode_* functions. This patch (05_ei_x_positiveindex.patch) must be applied after my previous patches: 01_ei_bugfixes.patch 02_ei_printtermshowtermmerged.patch 03_ei_xdecode.patch ei_x_decode_string.patch For the convenience, I have also sent a patch that applies all of those patches at once: ei_x_decode__fullpatch_2006-05-29.patch Regards, -- Romain LENGLET -------------- next part -------------- A non-text attachment was scrubbed... Name: 05_ei_x_positiveindex.patch.gz Type: application/x-gzip Size: 1739 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ei_x_decode__fullpatch_2006-05-29.patch.gz Type: application/x-gzip Size: 16137 bytes Desc: not available URL: