From raimo+erlang-patches@REDACTED Fri Jul 2 13:38:41 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Fri Jul 2 13:38:41 2010 Subject: What's cooking in erlang/otp (2010-07-02) Message-ID: <20100702134722.C5DFB1265@fingolfin.du.uab.ericsson.se> This issue of "What's cooking" is a weekend late due to midsummer holiday. Sorry about that. R14A was released as planned. We are now developing towards the R14B release and primarily will accept bugfixes for that. New features are not desired in that release... ------------------------------------------------------------ [New topics] * sa/callback-attr (compiler, inets, kernel, stdlib) (2010-06-08) 6 commits - Add callback specs into 'application' module in kernel (b7d428e) - Add callback specs to tftp module following internet documentation (2e8516f) - Add callback specs to inets_service module following possibly deprecated comments (45aaeb8) - Add '-callback' attributes in stdlib's behaviours (3fc32b3) - Automatical generation of 'behaviour_info' function from '-callback' attributes (7b89152) - Add '-callback' attribute to language syntax (3ccb12d) ------------------------------------------------------------ [Stalled] * bg/safer-local_to_univ (erts) (2010-02-11) 1 commit - erl_time_sup.c: test for error return from mktime() (0f62f0b) We will not include this branch in R13B04 for two reasons: o We are not 100% sure that this change is harmless on all platforms in all timezones. o We have a better solution for the underlying problem in the pipeline soon to surface in 'dev'. Action expected from: OTP Team * db/tv_nthtail_fix (tv) (2009-12-01) 1 commit - Fix for tv which restarts while trying to open a table (28f1e36) Since our long-term goal is to migrate from gs towards wx, maintaining the old gs applications has a low priority for the Erlang/OTP team. Therefore, we will accept patches that seem sound, but we are unlikely to spend much of our own time fixing bugs. The author of this topic branch has found the real reason for the problem, but uncovered additional issues. Action expected from: The topic author - to decide whether to include the topic branch (which is a workaround, but does eliminate the crash) or to wait for a better fix from himself or someone else with an interest to advance the topic. * er/ei-decode_msg (erl_interface) (2010-03-11) 1 commit - Add function to decode distribution header (864091a) Needs documentation and test cases. Action expected from: Topic author * jp/dependencies_makefile (compiler, erts) (2010-02-14) 3 commits - squash! eliminate dialyzer warning (434e2ca) - squash! fix snprintf for Windows (bf2708d) - Add dependencies Makefile generation to erlc(1) and compile(3) (7cbc437) Approved in principle by the OTP Technical Board; now awaits a closer review of the implementation. Action expected from: Bj??rn Gustavsson (the reviewer) * kr/gs-browser (gs) (2010-02-10) 1 commit - Change lib/gs/src/tool_utils.erl to select multiple browsers other than Netscape (802d159) This topic branch introduces a new call to the deprecated regexp module. Action expected from: Topic author or anyone else with an interest to advance the topic ------------------------------------------------------------ [Cooking] * bg/temp-hipe-fixes (compiler, hipe, kernel) (2010-03-15) 6 commits - Avoid crash by skipping compilation_SUITE:on_load/1 (234e13e) - Inline less aggressively for native-code compilation (9fcb48b) - Crudely fix return type for the lists:key{search,find,member}/3 (ef71597) - seq_trace_SUITE: Don't native-compile (2a81b1a) - Disable native code if on_load is used (3befa76) - andor_SUITE: Don't native-compile (97aa2da) This branch contains temporary workarounds to avoid failing test cases and is never intended to graduate. * cf/timer_tc (stdlib) (2010-04-06) 1 commit - Add timer:tc/2 to measure the elapsed time of anonymous functions (2cca650) * gl/jinterface-pom (jinterface, otp) (2010-02-08) 1 commit - Generate pom.xml during jinterface build (165fbca) * jf/run_erl-disable-flow-control (erts) (2010-03-05) 1 commit - Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control (d0775cd) * jn/gen_stream (stdlib) (2010-05-17) 1 commit - Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) . Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) This topic branch has failing test cases. shell_SUITE:start_restricted_from_shell; We have now tested with and without this topic branch and when it is present that test case fails. We consider it proved to be guilty. The test case does not fail when run standalone so it seems to be tricky... Also, some of its own test cases fail sporadicly apparently since the assumption there shoul be no more processes after the test case does not hold; probably a timing issue. Action expected from: Topic author. * jr/cover-src-path (tools) (2010-04-23) 1 commit - Fix searching for source files in analyse_to_file/* (b9d5068) * ks/auth-specs (kernel) (2010-06-09) 1 commit - auth: Fix types and specs (1c51135) * ks/cleanups (compiler, debugger, docbuilder, stdlib) (2010-04-10) 4 commits - compiler: Fix incorrect types and specs (8e79baa) - escript: Add more types to records (f2a2139) - debugger: Clean up as suggested by tidier (9e962d7) - docbuilder: Clean up as suggested by tidier (3647def) Was formerly called ks/cleanups-after-r13b04. * mk/net-dragonfly-bsd-patches (asn1, common_test, erl_interface, erts, et, gs, mnesia, odbc, reltool, runtime_tools, test_server, wx) (2010-06-23) 7 commits - Remove unused variables (dab5af5) - Do not hardcode browser name (82dcc82) - Make sure the odbcserver binary finds the ODBC library at run-time (c86dff5) - Use xxx(void) instead of xxx() (7142d52) - Use proper install method (5314ae6) - Add support for DragonFly BSD (03baa22) - Add support for NetBSD (6b72959) * pg/beam_lib_cmp_2_return_type_specification (stdlib) (2010-06-17) 1 commit - Fix beam_lib:cmp/2 return type specification (8dad90e) * pg/hipe_crash_with_on_load (erts) (2010-06-27) 1 commit - Fix crash when calling functions in a module with an on_load attribute from a native module (495804b) * pg/honor-start-type-in-rel-files (sasl) (2010-06-06) 1 commit - Honor start type in .rel files when building relup files (7599900) . Honor start type in .rel files when building relup files (7f0192d) * rani/sctp-udp-recv-error (erts, kernel) (2010-07-02) 2 commits - Add test for gen_udp:connect (a7d8a32) - Fix inet_drv to detect passive mode UDP errors for SCTP builds (c78bc8b) * sb/make-files-like-erlc (tools) (2010-05-18) 1 commit - Change make:files to behave more like erlc (5e9d051) * sd/epmd-stop-cli (erts) (2010-06-16) 2 commits - fix typo (101b731) - allow epmd -stop name to unregister a client from epmd (b070575) * se/http_response_empty_phrase (erts) (2010-06-10) 1 commit - {packet,http} allow empty response phrase (288f54c) * sf/erts_de_busy_limit (erts) (2010-06-08) 1 commit - Add flag-based setting for 'erts_de_busy_limit' constant (57c0eaf) . Add flag-based setting for 'erts_de_busy_limit' constant (b584553) * ta/asn1-reporting (asn1) (2010-04-23) 1 commit - asn1ct: Make formatting of errors and warnings consistent (93face2) * ta/filetypo (kernel) (2010-06-11) 1 commit - Correct trivial typos in file manual (6973f77) * uw/epmd-fullnames (kernel) (2010-05-07) 1 commit - Allow the erlang nodename host part to differ from the hostname (e0774be) ------------------------------------------------------------ [Dropped] * hb/edoc (edoc) (2010-04-01) 1 commit . EDoc and Dialyzer specs and types (516e16c) Temporarily dropped. Will be fixed to use updated dialyzer. * pg/xmerl_scan_hex_entities (xmerl) (2010-06-07) 1 commit . Fix decoding of hex entities (&#xXXXX;) in xmerl_scan (36448fc) The bug it fixes is recognized, but the fix in the branch is not complete... The test case is in our internal 'dev' branch but the test suites are not released yet due to remaining copyright issues. We will fix the bug in a future release. From pguyot@REDACTED Sun Jul 4 15:19:22 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 4 Jul 2010 15:19:22 +0200 Subject: Optimization of beams string table generation Message-ID: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> Hello, beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2. git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From raimo+erlang-patches@REDACTED Mon Jul 5 11:41:32 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Mon, 5 Jul 2010 11:41:32 +0200 Subject: [erlang-patches] Optimization of beams string table generation In-Reply-To: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> References: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> Message-ID: <20100705094132.GA8878@erix.ericsson.se> On Sun, Jul 04, 2010 at 03:19:22PM +0200, Paul Guyot wrote: > Hello, > > beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2. > > git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table Thank you. It will be included in 'pu', after line breaking the commit comment. Note that we prefer hyphen ('-') as delimiter in branch names, except for when it ought to be underscore. E.g pg/optimize-beam_dict-string-table. I kept your name this time to minimize my hazzle... > > http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a > > Regards, > > Paul > -- > Semiocast http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From pguyot@REDACTED Wed Jul 7 00:13:29 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 7 Jul 2010 00:13:29 +0200 Subject: Segfault on crash dump with hipe Message-ID: Hello, There is a bug that causes a segmentation fault when dumping the crash log if hipe is enabled and there are native modules loaded. A fix is available here: git fetch git://github.com/pguyot/otp.git pg/fix-segfault-on-crash_dump-with-hipe http://github.com/pguyot/otp/commit/089e48ca0c0486393dd1e886aa18fc4b870b1e32 Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From raimo+erlang-patches@REDACTED Wed Jul 7 11:57:40 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Wed, 7 Jul 2010 11:57:40 +0200 Subject: [erlang-patches] Segfault on crash dump with hipe In-Reply-To: References: Message-ID: <20100707095740.GA12040@erix.ericsson.se> Accidentally replied to erlang-bugs only due to cross-posting.. On Wed, Jul 07, 2010 at 12:13:29AM +0200, Paul Guyot wrote: > Hello, > > There is a bug that causes a segmentation fault when dumping the crash log if hipe is enabled and there are native modules loaded. > > A fix is available here: > git fetch git://github.com/pguyot/otp.git pg/fix-segfault-on-crash_dump-with-hipe Thank you! It will be included in 'pu'. Can you also line break your commit comments to say 76 columns in the future...? I will do it on this one. > > http://github.com/pguyot/otp/commit/089e48ca0c0486393dd1e886aa18fc4b870b1e32 > > Regards, > > Paul > -- > Semiocast http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Wed Jul 7 16:46:28 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Wed, 7 Jul 2010 16:46:28 +0200 Subject: [erlang-bugs] Bug: anonymous gen_event process + format_status. In-Reply-To: References: Message-ID: <20100707144628.GA20745@erix.ericsson.se> On Sun, Jun 13, 2010 at 02:02:12PM -0700, Geoff Cant wrote: > > Hi all, it seems during the last round of format_status improvements a > bug was introduced for anonymous gen_event processes. > > Repro: > {ok, Pid} = gen_event:start(), sys:get_status(Pid). > > Expected: Status report > Actual: Crash due to calling lists:concat on a list containing a pid() > > I've done some work on format_status here: > git fetch git://github.com/archaelus/otp.git format_status_improvements Thank you! It will be included in 'pu' as gc/gen-format-status-improvements. I have squashed the commits and reformatted/reworded the commit comment according to our standards. Sorry I missed this one when it was posted. It was on the wrong list (erlang-bugs) so my radar was on low effect... or something. Sorry about that. Instead I did some more work on the commit formatting. > > I've fixed the bug by porting the gen_fsm StatusHdr code into gen_event > at 00b032608c3ad3573d2b. Then I refactored the StatusHdr code out of > gen_server, gen_fsm and gen_event into 'gen' at 9103fa5136d360839aad, > and extended the test suite to test for the anon gen_event format status > bug at 480f1282d9bcdd7d40c3. > > Cheers, > -- > Geoff Cant > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Fri Jul 9 09:43:58 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Fri Jul 9 09:43:58 2010 Subject: What's cooking in erlang/otp (2010-07-09) Message-ID: <20100709101728.8179E14BD@fingolfin.du.uab.ericsson.se> R14A was released as planned. We are now developing towards the R14B release and primarily will accept bugfixes for that. New features are not desired in that release... ------------------------------------------------------------ [Graduated] * cf/timer_tc (stdlib) (2010-04-06) 1 commit + Add timer:tc/2 to measure the elapsed time of anonymous functions (2cca650) * gl/jinterface-pom (jinterface, otp) (2010-02-08) 1 commit + Generate pom.xml during jinterface build (165fbca) * ks/auth-specs (kernel) (2010-06-09) 1 commit + auth: Fix types and specs (1c51135) * ks/cleanups (compiler, debugger, docbuilder, stdlib) (2010-04-10) 4 commits + compiler: Fix incorrect types and specs (8e79baa) + escript: Add more types to records (f2a2139) + debugger: Clean up as suggested by tidier (9e962d7) + docbuilder: Clean up as suggested by tidier (3647def) * pg/beam_lib_cmp_2_return_type_specification (stdlib) (2010-06-17) 1 commit + Fix beam_lib:cmp/2 return type specification (8dad90e) * rani/sctp-udp-recv-error (erts, kernel) (2010-06-21) 1 commit + Fix inet_drv to detect passive mode UDP errors for SCTP builds (2128fce) * sd/epmd-stop-cli (erts) (2010-06-16) 1 commit + allow epmd -stop name to unregister a client from epmd (ebcab38) * se/http_response_empty_phrase (erts) (2010-06-10) 1 commit + {packet,http} allow empty response phrase (288f54c) * ta/filetypo (kernel) (2010-06-11) 1 commit + Correct trivial typos in file manual (6973f77) ------------------------------------------------------------ [New topics] * gc/gen-format-status-improvements (stdlib) (2010-06-12) 1 commit - Fix format_status bug for unregistered gen_event processes (95ed86f) * pg/fix-segfault-on-crash_dump-with-hipe (erts) (2010-07-06) 1 commit - Fix a bug that could produce a segmentation fault when dumping the crash log with hipe enabled and natively compiled modules (089e48c) * pg/optimize_beam_dict_string_table (compiler) (2010-07-04) 1 commit - Optimize string table generation for beams (cc1e0b4) ------------------------------------------------------------ [Stalled] * bg/safer-local_to_univ (erts) (2010-02-11) 1 commit - erl_time_sup.c: test for error return from mktime() (0f62f0b) We will not include this branch in R13B04 for two reasons: o We are not 100% sure that this change is harmless on all platforms in all timezones. o We have a better solution for the underlying problem in the pipeline soon to surface in 'dev'. Action expected from: OTP Team * db/tv_nthtail_fix (tv) (2009-12-01) 1 commit - Fix for tv which restarts while trying to open a table (28f1e36) Since our long-term goal is to migrate from gs towards wx, maintaining the old gs applications has a low priority for the Erlang/OTP team. Therefore, we will accept patches that seem sound, but we are unlikely to spend much of our own time fixing bugs. The author of this topic branch has found the real reason for the problem, but uncovered additional issues. Action expected from: The topic author - to decide whether to include the topic branch (which is a workaround, but does eliminate the crash) or to wait for a better fix from himself or someone else with an interest to advance the topic. * er/ei-decode_msg (erl_interface) (2010-03-11) 1 commit - Add function to decode distribution header (864091a) Needs documentation and test cases. Action expected from: Topic author * jp/dependencies_makefile (compiler, erts) (2010-02-14) 3 commits - squash! eliminate dialyzer warning (434e2ca) - squash! fix snprintf for Windows (bf2708d) - Add dependencies Makefile generation to erlc(1) and compile(3) (7cbc437) Approved in principle by the OTP Technical Board; now awaits a closer review of the implementation. Action expected from: Bj??rn Gustavsson (the reviewer) * kr/gs-browser (gs) (2010-02-10) 1 commit - Change lib/gs/src/tool_utils.erl to select multiple browsers other than Netscape (802d159) This topic branch introduces a new call to the deprecated regexp module. Action expected from: Topic author or anyone else with an interest to advance the topic * jn/gen_stream (stdlib) (2010-05-17) 1 commit - Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) . Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) This topic branch has failing test cases. shell_SUITE:start_restricted_from_shell; We have now tested with and without this topic branch and when it is present that test case fails. We consider it proved to be guilty. The test case does not fail when run standalone so it seems to be tricky... Also, some of its own test cases fail sporadicly apparently since the assumption there shoul be no more processes after the test case does not hold; probably a timing issue. Action expected from: Topic author. ------------------------------------------------------------ [Cooking] * bg/temp-hipe-fixes (compiler, hipe, kernel) (2010-03-15) 6 commits - Avoid crash by skipping compilation_SUITE:on_load/1 (234e13e) - Inline less aggressively for native-code compilation (9fcb48b) - Crudely fix return type for the lists:key{search,find,member}/3 (ef71597) - seq_trace_SUITE: Don't native-compile (2a81b1a) - Disable native code if on_load is used (3befa76) - andor_SUITE: Don't native-compile (97aa2da) This branch contains temporary workarounds to avoid failing test cases and is never intended to graduate. * jf/run_erl-disable-flow-control (erts) (2010-03-05) 1 commit - Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control (d0775cd) * jr/cover-src-path (tools) (2010-04-23) 1 commit - Fix searching for source files in analyse_to_file/* (b9d5068) * mk/net-dragonfly-bsd-patches (asn1, common_test, erl_interface, erts, et, gs, mnesia, odbc, reltool, runtime_tools, test_server, wx) (2010-06-23) 7 commits - Remove unused variables (dab5af5) - Do not hardcode browser name (82dcc82) - Make sure the odbcserver binary finds the ODBC library at run-time (c86dff5) - Use xxx(void) instead of xxx() (7142d52) - Use proper install method (5314ae6) - Add support for DragonFly BSD (03baa22) - Add support for NetBSD (6b72959) We are investigating if finding ODBC run-time library (c86dff5) is done the right(tm) way, otherwise it seems fine... * pg/hipe_crash_with_on_load (erts) (2010-06-27) 1 commit - Fix crash when calling functions in a module with an on_load attribute from a native module (495804b) * pg/honor-start-type-in-rel-files (sasl) (2010-06-06) 1 commit - Honor start type in .rel files when building relup files (7599900) . Honor start type in .rel files when building relup files (7f0192d) This change is too big to make it into the R14A -> R14B release. It will have to be included later. * sa/callback-attr (compiler, inets, kernel, stdlib) (2010-06-08) 6 commits - Add callback specs into 'application' module in kernel (b7d428e) - Add callback specs to tftp module following internet documentation (2e8516f) - Add callback specs to inets_service module following possibly deprecated comments (45aaeb8) - Add '-callback' attributes in stdlib's behaviours (3fc32b3) - Automatical generation of 'behaviour_info' function from '-callback' attributes (7b89152) - Add '-callback' attribute to language syntax (3ccb12d) We have having difficulties fiding the time to review this change properly for the R14B release, which is a pity since it should come in a major release... It will probably have to be included later. * sb/make-files-like-erlc (tools) (2010-05-18) 1 commit - Change make:files to behave more like erlc (5e9d051) * sf/erts_de_busy_limit (erts) (2010-06-08) 1 commit - Add flag-based setting for 'erts_de_busy_limit' constant (57c0eaf) . Add flag-based setting for 'erts_de_busy_limit' constant (b584553) * ta/asn1-reporting (asn1) (2010-04-23) 1 commit - asn1ct: Make formatting of errors and warnings consistent (93face2) * uw/epmd-fullnames (kernel) (2010-05-07) 1 commit - Allow the erlang nodename host part to differ from the hostname (e0774be) ------------------------------------------------------------ [Dropped] * hb/edoc (edoc) (2010-04-01) 1 commit . EDoc and Dialyzer specs and types (516e16c) Temporarily dropped. Will be fixed to use updated dialyzer. * pg/xmerl_scan_hex_entities (xmerl) (2010-06-07) 1 commit . Fix decoding of hex entities (&#xXXXX;) in xmerl_scan (36448fc) The bug it fixes is recognized, but the fix in the branch is not complete... The test case is in our internal 'dev' branch but the test suites are not released yet due to remaining copyright issues. We will fix the bug in a future release. From jay@REDACTED Fri Jul 9 21:31:32 2010 From: jay@REDACTED (jay@REDACTED) Date: Fri, 9 Jul 2010 12:31:32 -0700 (PDT) Subject: Trying to update jn/gen_stream Message-ID: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> I did a fresh synch with dev and tried to merge in the pu topic jn/gen_stream, but I can't seem to find it. I did the following. git pull upstream dev make make release_docs $ERL_TOP/bin/erl => R14B git push origin origin:refs/heads/gen_stream_track_test_procs git checkout --track -b gen_track_test_procs git log --oneline origin/pu | grep jn aaeb58c Merge branch 'jn/ic-docs-without-java' into pu 333d2b2 Merge branch 'jn/sasl-format_report' into ccase/r13b04_dev 61489dd Merge branch 'jn/supervisor_child_count_only' into ccase/r13b04_dev git branch -f jn/gen_stream 65f3db8^2 fatal: Not a valid object name: '65f3db8^2' git branch -f jn/gen_stream 65f3db8 fatal: Not a valid object name: '65f3db8' I guess starting from 'dev' was not a good thing, although I did want to verify that I can build and run tests in dev properly. How do I find my project in 'pu' and properly create a tracking branch that I can use to update it? jay From tuncer.ayaz@REDACTED Sat Jul 10 12:37:58 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 10 Jul 2010 12:37:58 +0200 Subject: [erlang-patches] Trying to update jn/gen_stream In-Reply-To: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> References: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> Message-ID: On Fri, Jul 9, 2010 at 9:31 PM, wrote: > How do I find my project in 'pu' and properly create a tracking branch > that I can use to update it? Hi Jay, have you tried using Bjorn's create-topic-branches Perl script? It's documented here: http://wiki.github.com/erlang/otp/branches. $ create-topic-branches upstream/dev..upstream/pu|grep 'jn/gen_stream' git branch -f jn/gen_stream 2009b7a^2 $ create-topic-branches upstream/dev..upstream/pu|grep 'jn/gen_stream'|sh $ git checkout jn/gen_stream From rzezeski@REDACTED Sat Jul 10 23:11:04 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Sat, 10 Jul 2010 17:11:04 -0400 Subject: Fixes to httpc docs Message-ID: git fetch git@REDACTED:progski/otp.git fix_httpc_docs I made some fixes to the inets documentation in relation to the http client. There were some references to the deprecated http module which I changed to httpc. -Ryan From pguyot@REDACTED Sun Jul 11 19:48:47 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 11 Jul 2010 19:48:47 +0200 Subject: fixed hipe:load/1 Message-ID: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> Hello, hipe:load/1 just doesn't work in R14A (or dev). A fix with non-regression tests can be found here: git fetch git://github.com/pguyot/otp.git pg/fix-hipe-load http://github.com/pguyot/otp/commit/66e2d5d13a0f55019e83a77a6a09fa255c72da9a Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From pguyot@REDACTED Sun Jul 11 20:00:39 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 11 Jul 2010 20:00:39 +0200 Subject: load native code for early modules Message-ID: Hello, The native chunks of modules loaded before the code server are never loaded, which makes it pretty pointless to natively compile them in the first place (with --enable-native-libs). A patch that loads them just after the code server is started can be found here: git fetch git://github.com/pguyot/otp.git pg/load-native-code-for-early-modules http://github.com/pguyot/otp/commit/2637b8a24e2d9ba3653e580f21a70976915dc201 As a side effect, this patch will decrease the load time of dialyzer on installations configured with --enable-native-libs. Indeed, dialyzer natively recompiles modules including some that were loaded before the code server (such as lists, gb_trees, etc.). Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From rzezeski@REDACTED Sun Jul 11 20:30:58 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Sun, 11 Jul 2010 14:30:58 -0400 Subject: Fixes to httpc docs In-Reply-To: References: Message-ID: I suppose I should have given my read-only URL. git fetch git://github.com/progski/otp.git fix_httpc_docs On Sat, Jul 10, 2010 at 5:11 PM, Ryan Zezeski wrote: > git fetch git@REDACTED:progski/otp.git fix_httpc_docs > > I made some fixes to the inets documentation in relation to the http > client. There were some references to the deprecated http module which I > changed to httpc. > > -Ryan > From mikpe@REDACTED Mon Jul 12 11:41:51 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 12 Jul 2010 11:41:51 +0200 Subject: [erlang-patches] fixed hipe:load/1 In-Reply-To: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> References: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> Message-ID: <19514.58207.210584.367098@pilspetsen.it.uu.se> Paul Guyot writes: > Hello, > > hipe:load/1 just doesn't work in R14A (or dev). A fix with non-regression tests can be found here: > > git fetch git://github.com/pguyot/otp.git pg/fix-hipe-load > > http://github.com/pguyot/otp/commit/66e2d5d13a0f55019e83a77a6a09fa255c72da9a Confirmed and for the fix in hipe.erl: ACK. Who uses hipe:load/1 anyway? I'm just wondering if it's a relic from ancient history, or if it actually offers functionality not available elsewhere. The test suite thing is orthogonal. I don't mind it, but it won't be used by HiPE developers since our test suite is completely separate from Erlang/OTP's. From pguyot@REDACTED Mon Jul 12 12:37:04 2010 From: pguyot@REDACTED (Paul Guyot) Date: Mon, 12 Jul 2010 12:37:04 +0200 Subject: [erlang-patches] fixed hipe:load/1 In-Reply-To: <19514.58207.210584.367098@pilspetsen.it.uu.se> References: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> <19514.58207.210584.367098@pilspetsen.it.uu.se> Message-ID: <520F90F1-A55A-48CE-B69C-A8399052AB6B@kallisys.net> Mikael, Thank you for your response. Le 12 juil. 2010 ? 11:41, Mikael Pettersson a ?crit : > Who uses hipe:load/1 anyway? I'm just wondering if it's a relic from > ancient history, or if it actually offers functionality not available > elsewhere. I guess no one uses hipe:load/1 since it is broken. It is documented in hipe:help_hiper(), though. We wanted to use it as a workaround to the other bug I submitted a patch for (native code of early modules is not loaded), and we ended up duplicating the code in our internal tool that invokes dialyzer. Considering that there is a compilation option to compile a code natively but to not load this code ([native, {hipe, [no_load]}]), I guess hipe:load/1 makes sense even if the other bug is fixed. > The test suite thing is orthogonal. I don't mind it, but it won't be > used by HiPE developers since our test suite is completely separate from > Erlang/OTP's. Is the test suite used by HiPE developers open source ? Where is it located ? Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From mikpe@REDACTED Mon Jul 12 15:16:36 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 12 Jul 2010 15:16:36 +0200 Subject: [erlang-patches] fixed hipe:load/1 In-Reply-To: <520F90F1-A55A-48CE-B69C-A8399052AB6B@kallisys.net> References: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> <19514.58207.210584.367098@pilspetsen.it.uu.se> <520F90F1-A55A-48CE-B69C-A8399052AB6B@kallisys.net> Message-ID: <19515.5556.482148.369657@pilspetsen.it.uu.se> Paul Guyot writes: > Mikael, > > Thank you for your response. > > Le 12 juil. 2010 ? 11:41, Mikael Pettersson a ?crit : > > > Who uses hipe:load/1 anyway? I'm just wondering if it's a relic from > > ancient history, or if it actually offers functionality not available > > elsewhere. > > I guess no one uses hipe:load/1 since it is broken. It is documented in hipe:help_hiper(), though. We wanted to use it as a workaround to the other bug I submitted a patch for (native code of early modules is not loaded), and we ended up duplicating the code in our internal tool that invokes dialyzer. Considering that there is a compilation option to compile a code natively but to not load this code ([native, {hipe, [no_load]}]), I guess hipe:load/1 makes sense even if the other bug is fixed. But [native,{hipe,[no_load]}] stores the native code in the .beam file if I'm not mistaken. So you'd just need to load or upgrade that module to bring the native code into the VM. I still don't see why hipe:load/1 is needed. > > The test suite thing is orthogonal. I don't mind it, but it won't be > > used by HiPE developers since our test suite is completely separate from > > Erlang/OTP's. > > Is the test suite used by HiPE developers open source ? Where is it located ? It's intended to be open source, but for historical reasons it's still located on a departmental CVS server with limited public access. I intend to convert it to git "any day now". From tuncer.ayaz@REDACTED Mon Jul 12 17:11:00 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 12 Jul 2010 17:11:00 +0200 Subject: asn1ct: change unusual whitespace char from 0xA0 to 0x20 Message-ID: As pointed out by Joe there is an unusual whitespace character on line 1677. Change it from 0xA0 to 0x20. git fetch git://github.com/tuncer/otp.git asn1ct-fix-whitespace From raimo+erlang-patches@REDACTED Tue Jul 13 13:42:25 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Tue, 13 Jul 2010 13:42:25 +0200 Subject: [erlang-patches] Trying to update jn/gen_stream In-Reply-To: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> References: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> Message-ID: <20100713114225.GA8503@erix.ericsson.se> On Fri, Jul 09, 2010 at 12:31:32PM -0700, jay@REDACTED wrote: > I did a fresh synch with dev and tried to merge in the pu topic > jn/gen_stream, but I can't seem to find it. I did the following. > > git pull upstream dev > make > make release_docs > $ERL_TOP/bin/erl => R14B > git push origin origin:refs/heads/gen_stream_track_test_procs > git checkout --track -b gen_track_test_procs > > git log --oneline origin/pu | grep jn > > aaeb58c Merge branch 'jn/ic-docs-without-java' into pu > 333d2b2 Merge branch 'jn/sasl-format_report' into ccase/r13b04_dev > 61489dd Merge branch 'jn/supervisor_child_count_only' into ccase/r13b04_dev > > > git branch -f jn/gen_stream 65f3db8^2 > fatal: Not a valid object name: '65f3db8^2' > > > git branch -f jn/gen_stream 65f3db8 > fatal: Not a valid object name: '65f3db8' > > I guess starting from 'dev' was not a good thing, although I did want to > verify that I can build and run tests in dev properly. No, this was a good thing. > > > How do I find my project in 'pu' and properly create a tracking branch > that I can use to update it? You should own the master branch. I should be tracking you. Tuncer has hinted you about the 'create-topic-branches' script. Here is a more low-level hint, "your" branch is as you figured out parent number two (or 1, or the right, or ?) of the merge commit you found and here is another way to dig it out. The way you did it ought to work, but perhaps this fetch was missing: $ git fetch upstream pu $ git log --oneline --parents upstream/pu | grep jn 2009b7a 7ba2cc7 65f3db8 Merge branch 'jn/gen_stream' into pu 6d1a56d c0895d1 2016a08 Merge branch 'jn/ic-docs-without-java' into dev 333d2b2 ad8ff21 109a257 Merge branch 'jn/sasl-format_report' into ccase/r13b04_dev 61489dd aad8754 1686757 Merge branch 'jn/supervisor_child_count_only' into ccase/r13b04_dev $ git show --name-only 65f3db8 commit 65f3db8de97410d3d683dbbc9419ba975232ad46 Author: Jay Nelson Date: Mon May 17 02:06:59 2010 -0700 Add new gen_stream behaviour for efficiently consuming serial streams Introduce a new behaviour for serially processing text files, generating lazy or infinite streams or providing a serial interface for a binary. Implement the behaviour functionality using a gen_server while internally managing multiple processes to load the stream. Stripe stream data across the worker processes to enable overlapped I/O when reading from the source. Impose a simple API for consuming the stream serially, a block at a time. The initial arguments for the gen_stream specify the number of processes used, the number of data chunks per process, the size of each chunk a blocking factor for reading multiple chunks at a time and an optional function parameter to transform the data while buffering it. The serial stream delivers the (potentially transformed) data from the internal process buffers as it is requested by the calling application. lib/stdlib/doc/src/Makefile lib/stdlib/doc/src/gen_stream.xml lib/stdlib/doc/src/ref_man.xml lib/stdlib/src/Makefile lib/stdlib/src/gen_stream.erl lib/stdlib/src/stdlib.app.src lib/stdlib/test/Makefile lib/stdlib/test/gen_stream_SUITE.erl lib/stdlib/test/gen_stream_SUITE_data/alpha.txt lib/stdlib/test/gen_stream_odd_nums.erl $ git branch jn/gen_stream 65f3db8de97410d3d683dbbc9419ba975232ad46 $ git checkout jn/gen_stream $ git rebase dev > > jay > > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Tue Jul 13 13:45:45 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Tue, 13 Jul 2010 13:45:45 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: <20100713114545.GB8503@erix.ericsson.se> On Sun, Jul 11, 2010 at 08:00:39PM +0200, Paul Guyot wrote: > Hello, > > The native chunks of modules loaded before the code server are never loaded, which makes it pretty pointless to natively compile them in the first place (with --enable-native-libs). A patch that loads them just after the code server is started can be found here: > > git fetch git://github.com/pguyot/otp.git pg/load-native-code-for-early-modules Thank you, looks fine! It will be included in 'pu'. > > http://github.com/pguyot/otp/commit/2637b8a24e2d9ba3653e580f21a70976915dc201 > > As a side effect, this patch will decrease the load time of dialyzer on installations configured with --enable-native-libs. Indeed, dialyzer natively recompiles modules including some that were loaded before the code server (such as lists, gb_trees, etc.). > > Regards, > > Paul > -- > Semiocast http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Tue Jul 13 13:52:54 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Tue, 13 Jul 2010 13:52:54 +0200 Subject: [erlang-patches] fixed hipe:load/1 In-Reply-To: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> References: <9EF53416-B42C-4844-B4E1-97308453C76A@kallisys.net> Message-ID: <20100713115254.GC8503@erix.ericsson.se> On Sun, Jul 11, 2010 at 07:48:47PM +0200, Paul Guyot wrote: > Hello, > > hipe:load/1 just doesn't work in R14A (or dev). A fix with non-regression tests can be found here: > > git fetch git://github.com/pguyot/otp.git pg/fix-hipe-load Thank you! It will be included in 'pu'. > > http://github.com/pguyot/otp/commit/66e2d5d13a0f55019e83a77a6a09fa255c72da9a > > Regards, > > Paul > -- > Semiocast http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Tue Jul 13 13:55:27 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Tue, 13 Jul 2010 13:55:27 +0200 Subject: [erlang-patches] asn1ct: change unusual whitespace char from 0xA0 to 0x20 In-Reply-To: References: Message-ID: <20100713115527.GD8503@erix.ericsson.se> On Mon, Jul 12, 2010 at 05:11:00PM +0200, Tuncer Ayaz wrote: > As pointed out by Joe there is an unusual whitespace character on > line 1677. Change it from 0xA0 to 0x20. > > git fetch git://github.com/tuncer/otp.git asn1ct-fix-whitespace a non-breakable space in erlang code, interesting... Thank you, it will be included in 'pu'! > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tomas@REDACTED Tue Jul 13 17:39:16 2010 From: tomas@REDACTED (Tomas Johansson) Date: Tue, 13 Jul 2010 17:39:16 +0200 Subject: Increased timestamp granularity in Common Test Message-ID: Hello. The Common Test log functions ct:log/1/2/3 and ct:pal/1/2/3 uses a timestamp with a granularity down to seconds. This is not fine-grained enough for us, so I've changed this to include also milliseconds: git fetch git://github.com/jotsen/otp.git ct-timestamp-granularity Regards, Tomas From raimo+erlang-patches@REDACTED Tue Jul 13 19:52:13 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Tue, 13 Jul 2010 19:52:13 +0200 Subject: [erlang-patches] Trying to update jn/gen_stream In-Reply-To: <41255.76.217.22.137.1279040254.squirrel@www.5x5.net> References: <62087.64.81.32.110.1278703892.squirrel@www.5x5.net> <20100713114225.GA8503@erix.ericsson.se> <41255.76.217.22.137.1279040254.squirrel@www.5x5.net> Message-ID: <20100713175213.GA12557@erix.ericsson.se> On Tue, Jul 13, 2010 at 09:57:34AM -0700, jay@REDACTED wrote: > > The way you did > > it ought to work, but perhaps this fetch was missing: > > > > $ git fetch upstream pu > > Thanks, yes that was the problem. I could see the branch after I did that > this weekend. Tuncer helped me with some other commands for grabbing > patches, etc. > > My availability is spotty, but I will try to get time to work on issues > with the test cases. It's a bummer that some oversights on my part in the > test code is likely what's holding it up. > > > $ git branch jn/gen_stream 65f3db8de97410d3d683dbbc9419ba975232ad46 > > $ git checkout jn/gen_stream > > $ git rebase dev > > > > You mentioned that you should be tracking my master branch and that makes > sense to me now. Unfortunately, when you merged in to 'pu' I assumed that > my branch wasn't necessary, so I deleted it. Not really an issue, but if > I do 'git checkout jn/gen_stream' I assume I will essentially be working > off 'pu' but it will be forward mergeable and you can pick it up properly. Not exactly, better... If you do 'git checkout jn/gen_stream' where you have digged out that branch point from the merge parent in 'pu', you will find that it is actually a branch originating from a recent 'dev'; I have rebased, squashed and polished it a bit, but otherwise it is your original changeset. I merge from (my version) of your branch into 'pu', therefore your branch (my version) and where it originates is possible to recover by dismantling 'pu'. I heard Linus Torvalds does not keep backups of his work since he has enough reliable people over the globe that has got some version of his branches, so they are always possible to resque... > > jay > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vinoski@REDACTED Wed Jul 14 20:01:52 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 14 Jul 2010 14:01:52 -0400 Subject: check return value of epmd listen Message-ID: While looking into a strange repetitive accept() failure in epmd I noticed that the return value of the listen() isn't checked for errors. This patch fixes that. git fetch git://github.com/vinoski/otp.git epmd-check-listen --steve From raimo+erlang-patches@REDACTED Thu Jul 15 12:25:45 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Thu, 15 Jul 2010 12:25:45 +0200 Subject: [erlang-patches] Increased timestamp granularity in Common Test In-Reply-To: References: Message-ID: <20100715102545.GA10333@erix.ericsson.se> On Tue, Jul 13, 2010 at 05:39:16PM +0200, Tomas Johansson wrote: > Hello. > > The Common Test log functions ct:log/1/2/3 and ct:pal/1/2/3 uses a > timestamp with a granularity down to seconds. > This is not fine-grained enough for us, so I've changed this to > include also milliseconds: > > git fetch git://github.com/jotsen/otp.git ct-timestamp-granularity Thank you! It will be included in 'pu'. > > Regards, > Tomas > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Thu Jul 15 12:26:05 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Thu, 15 Jul 2010 12:26:05 +0200 Subject: [erlang-patches] check return value of epmd listen In-Reply-To: References: Message-ID: <20100715102605.GB10333@erix.ericsson.se> On Wed, Jul 14, 2010 at 02:01:52PM -0400, Steve Vinoski wrote: > While looking into a strange repetitive accept() failure in epmd I > noticed that the return value of the listen() isn't checked for > errors. This patch fixes that. > > git fetch git://github.com/vinoski/otp.git epmd-check-listen Thank you! It will be included in 'pu'. > > --steve > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jay@REDACTED Thu Jul 15 19:17:06 2010 From: jay@REDACTED (jay@REDACTED) Date: Thu, 15 Jul 2010 10:17:06 -0700 (PDT) Subject: gen_stream test suite fix for spurious process count failure Message-ID: <57198.64.81.32.114.1279214226.squirrel@5x5.net> I changed the test suite for gen_stream to not rely on erlang:system_info(process_count). Instead I exposed the processes using a new interface and check the actual number processes returned by the gen_stream server. git fetch git://github.com/jaynel/otp.git gen_stream-test-track-procs I hope this works. I pulled a recent dev, then got jn/gen_stream and applied it as a patch on top of my dev. My new changes plus the patch are all bundled into this single commit. I only changed two files: lib/stdlib/src/gen_stream.erl lib/stdlib/test/gen_stream.erl If you have any trouble, just cherry pick these two files and apply them on top of jn/gen_stream in pu. This fixes the spurious random failures. It does not attempt to address the failure caused in the shell test suite. I think I know how to track that one down and hope to get to it in the next few days. jay From jay@REDACTED Fri Jul 16 00:25:06 2010 From: jay@REDACTED (jay@REDACTED) Date: Thu, 15 Jul 2010 15:25:06 -0700 (PDT) Subject: gen_stream to be resubmitted later today Message-ID: <39786.76.217.22.137.1279232706.squirrel@www.5x5.net> Based on an email discussion with Tuncer, I will recreate the gen_stream branch and resubmit today's fix on that branch. Please hold off fetching until I do so. jay From jay@REDACTED Fri Jul 16 08:26:10 2010 From: jay@REDACTED (jay@REDACTED) Date: Thu, 15 Jul 2010 23:26:10 -0700 (PDT) Subject: gen_stream test suite fix available Message-ID: <57923.64.81.32.114.1279261570.squirrel@5x5.net> I changed the test suite for gen_stream to not rely on erlang:system_info(process_count). Instead I exposed the processes using a new interface and check the actual number processes returned by the gen_stream server. git fetch git://github.com/jaynel/otp.git gen_stream I started with dev, then got jn/gen_stream and applied it as a patch on top of my dev and committed. The new changes are a second commit on this new branch. I only changed two files: lib/stdlib/src/gen_stream.erl lib/stdlib/test/gen_stream_SUITE.erl If you have any trouble, just cherry pick these two files and apply them on top of your jn/gen_stream. I will continue working on the gen_stream branch. This fixes the spurious random failures. It does not attempt to address the failure caused in the shell test suite. I think I know how to track that one down and hope to get to it in the next few days. jay From raimo+erlang-patches@REDACTED Fri Jul 16 11:29:06 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Fri, 16 Jul 2010 11:29:06 +0200 Subject: [erlang-patches] gen_stream test suite fix available In-Reply-To: <57923.64.81.32.114.1279261570.squirrel@5x5.net> References: <57923.64.81.32.114.1279261570.squirrel@5x5.net> Message-ID: <20100716092906.GA29628@erix.ericsson.se> On Thu, Jul 15, 2010 at 11:26:10PM -0700, jay@REDACTED wrote: > > I changed the test suite for gen_stream to not rely on > erlang:system_info(process_count). Instead I exposed the processes using > a new interface and check the actual number processes returned by the > gen_stream server. > > > git fetch git://github.com/jaynel/otp.git gen_stream Thank you! It will be included in 'pu'. > > > I started with dev, then got jn/gen_stream and > applied it as a patch on top of my dev and committed. > The new changes are a second commit on this new branch. > > I only changed two files: > lib/stdlib/src/gen_stream.erl > lib/stdlib/test/gen_stream_SUITE.erl > > If you have any trouble, just cherry pick these two files and apply them > on top of your jn/gen_stream. I will continue working on the gen_stream > branch. > > This fixes the spurious random failures. It does not attempt to address > the failure caused in the shell test suite. I think I know how to track > that one down and hope to get to it in the next few days. > > jay > > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-patches@REDACTED Fri Jul 16 09:52:55 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Fri Jul 16 09:52:55 2010 Subject: What's cooking in erlang/otp (2010-07-16) Message-ID: <20100716100833.2FE9014D3@fingolfin.du.uab.ericsson.se> The next week (Jul 18..) will be a slow week regarding 'pu' updates. Most are on vacation, there will most probably not be anyone watching erlang-patches for contributions... We are now developing towards the R14B release and primarily will accept bugfixes for that. New features are not desired in that release... ------------------------------------------------------------ [New topics] * pg/fix-hipe-load (hipe) (2010-07-11) 1 commit - Fix hipe:load/1 (c1f9c3f) * pg/load-native-code-for-early-modules (kernel) (2010-07-11) 1 commit - Load native code for modules loaded before the code server (2637b8a) * rz/fix-httpc-docs (inets) (2010-07-10) 2 commits - Fix http client docs (294dfc0) - Fix broken links in inets docs (7e417c3) * sv/epmd-check-listen (erts) (2010-07-14) 1 commit - check return value of epmd server listen() call (c5c8baa) * ta/asn1ct-fix-whitespace (asn1) (2010-07-12) 1 commit - asn1ct: change unusual whitespace char from 0xA0 to 0x20 (c43d20b) * tj/ct-timestamp-granularity (common_test) (2010-07-13) 1 commit - Include milliseconds in timestamps in Common Test log entries (4d18f18) ------------------------------------------------------------ [Stalled] * bg/safer-local_to_univ (erts) (2010-02-11) 1 commit - erl_time_sup.c: test for error return from mktime() (0f62f0b) We will not include this branch in R13B04 for two reasons: o We are not 100% sure that this change is harmless on all platforms in all timezones. o We have a better solution for the underlying problem in the pipeline soon to surface in 'dev'. Action expected from: OTP Team * db/tv_nthtail_fix (tv) (2009-12-01) 1 commit - Fix for tv which restarts while trying to open a table (28f1e36) Since our long-term goal is to migrate from gs towards wx, maintaining the old gs applications has a low priority for the Erlang/OTP team. Therefore, we will accept patches that seem sound, but we are unlikely to spend much of our own time fixing bugs. The author of this topic branch has found the real reason for the problem, but uncovered additional issues. Action expected from: The topic author - to decide whether to include the topic branch (which is a workaround, but does eliminate the crash) or to wait for a better fix from himself or someone else with an interest to advance the topic. * er/ei-decode_msg (erl_interface) (2010-03-11) 1 commit - Add function to decode distribution header (864091a) Needs documentation and test cases. Action expected from: Topic author * jp/dependencies_makefile (compiler, erts) (2010-02-14) 3 commits - squash! eliminate dialyzer warning (434e2ca) - squash! fix snprintf for Windows (bf2708d) - Add dependencies Makefile generation to erlc(1) and compile(3) (7cbc437) Approved in principle by the OTP Technical Board; now awaits a closer review of the implementation. Action expected from: Bj??rn Gustavsson (the reviewer) * kr/gs-browser (gs) (2010-02-10) 1 commit - Change lib/gs/src/tool_utils.erl to select multiple browsers other than Netscape (802d159) This topic branch introduces a new call to the deprecated regexp module. Action expected from: Topic author or anyone else with an interest to advance the topic ------------------------------------------------------------ [Cooking] * jn/gen_stream (stdlib) (2010-07-15) 2 commits - Track gen_stream processes directly for accurate test results (38a4083) - Patch dev to reproduce jn/gen_stream (7694a74) This topic branch has now got more reliable testcases. We will see about the notorious shell_SUITE:start_restricted_from_shell, but the other spurious failures should be gone. * bg/temp-hipe-fixes (compiler, hipe, kernel) (2010-03-15) 6 commits - Avoid crash by skipping compilation_SUITE:on_load/1 (234e13e) - Inline less aggressively for native-code compilation (9fcb48b) - Crudely fix return type for the lists:key{search,find,member}/3 (ef71597) - seq_trace_SUITE: Don't native-compile (2a81b1a) - Disable native code if on_load is used (3befa76) - andor_SUITE: Don't native-compile (97aa2da) This branch contains temporary workarounds to avoid failing test cases and is never intended to graduate. * gc/gen-format-status-improvements (stdlib) (2010-06-12) 1 commit - Fix format_status bug for unregistered gen_event processes (95ed86f) * jf/run_erl-disable-flow-control (erts) (2010-03-05) 1 commit - Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control (d0775cd) * jr/cover-src-path (tools) (2010-04-23) 1 commit - Fix searching for source files in analyse_to_file/* (b9d5068) * mk/net-dragonfly-bsd-patches (asn1, common_test, erl_interface, erts, et, gs, mnesia, odbc, reltool, runtime_tools, test_server, wx) (2010-06-23) 7 commits - Remove unused variables (dab5af5) - Do not hardcode browser name (82dcc82) - Make sure the odbcserver binary finds the ODBC library at run-time (c86dff5) - Use xxx(void) instead of xxx() (7142d52) - Use proper install method (5314ae6) - Add support for DragonFly BSD (03baa22) - Add support for NetBSD (6b72959) We are investigating if finding ODBC run-time library (c86dff5) is done the right(tm) way, otherwise it seems fine... * pg/fix-segfault-on-crash_dump-with-hipe (erts) (2010-07-06) 1 commit - Fix a bug that could produce a segmentation fault when dumping the crash log with hipe enabled and natively compiled modules (089e48c) * pg/hipe_crash_with_on_load (erts) (2010-06-27) 1 commit - Fix crash when calling functions in a module with an on_load attribute from a native module (495804b) * pg/honor-start-type-in-rel-files (sasl) (2010-06-06) 1 commit - Honor start type in .rel files when building relup files (7599900) . Honor start type in .rel files when building relup files (7f0192d) This change is too big to make it into the R14A -> R14B release. It will have to be included later. * pg/optimize_beam_dict_string_table (compiler) (2010-07-04) 1 commit - Optimize string table generation for beams (cc1e0b4) * sa/callback-attr (compiler, inets, kernel, stdlib) (2010-06-08) 6 commits - Add callback specs into 'application' module in kernel (b7d428e) - Add callback specs to tftp module following internet documentation (2e8516f) - Add callback specs to inets_service module following possibly deprecated comments (45aaeb8) - Add '-callback' attributes in stdlib's behaviours (3fc32b3) - Automatical generation of 'behaviour_info' function from '-callback' attributes (7b89152) - Add '-callback' attribute to language syntax (3ccb12d) We have having difficulties fiding the time to review this change properly for the R14B release, which is a pity since it should come in a major release... It will probably have to be included later. * sb/make-files-like-erlc (tools) (2010-05-18) 1 commit - Change make:files to behave more like erlc (5e9d051) * sf/erts_de_busy_limit (erts) (2010-06-08) 1 commit - Add flag-based setting for 'erts_de_busy_limit' constant (57c0eaf) . Add flag-based setting for 'erts_de_busy_limit' constant (b584553) * ta/asn1-reporting (asn1) (2010-04-23) 1 commit - asn1ct: Make formatting of errors and warnings consistent (93face2) * uw/epmd-fullnames (kernel) (2010-05-07) 1 commit - Allow the erlang nodename host part to differ from the hostname (e0774be) ------------------------------------------------------------ [Dropped] * hb/edoc (edoc) (2010-04-01) 1 commit . EDoc and Dialyzer specs and types (516e16c) Temporarily dropped. Will be fixed to use updated dialyzer. * pg/xmerl_scan_hex_entities (xmerl) (2010-06-07) 1 commit . Fix decoding of hex entities (&#xXXXX;) in xmerl_scan (36448fc) The bug it fixes is recognized, but the fix in the branch is not complete... The test case is in our internal 'dev' branch but the test suites are not released yet due to remaining copyright issues. We will fix the bug in a future release. From cristian@REDACTED Fri Jul 16 13:21:49 2010 From: cristian@REDACTED (Cristian Greco) Date: Fri, 16 Jul 2010 13:21:49 +0200 Subject: Fixes for small typos in documentation Message-ID: <20100716132149.573583da@regolo> Hi, I fixed some small fixes in documentation files, the patch is available here: git fetch git://github.com/cristiangreco/otp.git small_docs_typos Hope this helps. Thanks, -- Cristian Greco GPG key ID: 0xCF4D32E4 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From raimo+erlang-patches@REDACTED Fri Jul 16 15:37:42 2010 From: raimo+erlang-patches@REDACTED (Raimo Niskanen) Date: Fri, 16 Jul 2010 15:37:42 +0200 Subject: [erlang-patches] Fixes for small typos in documentation In-Reply-To: <20100716132149.573583da@regolo> References: <20100716132149.573583da@regolo> Message-ID: <20100716133742.GA27120@erix.ericsson.se> On Fri, Jul 16, 2010 at 01:21:49PM +0200, Cristian Greco wrote: > Hi, > > I fixed some small fixes in documentation files, the patch is available > here: > > git fetch git://github.com/cristiangreco/otp.git small_docs_typos Thank you. It will be included in 'pu'. The last before my vacation... > > Hope this helps. > > Thanks, > -- > Cristian Greco > GPG key ID: 0xCF4D32E4 -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jay@REDACTED Fri Jul 16 18:03:07 2010 From: jay@REDACTED (jay@REDACTED) Date: Fri, 16 Jul 2010 09:03:07 -0700 (PDT) Subject: gen_stream tests success proof Message-ID: <60867.64.81.32.114.1279296187.squirrel@5x5.net> I committed a new proof version of lib/stdlib/test/gen_stream_SUITE.erl git fetch git://github.com/jaynel/otp.git gen_stream I'm not sure if my environment is wrong or what, but every time I run shell_SUITE.erl it fails on line 384 and line 584 (even if I start from scratch and call no other tests): 384: ?line PreReply = scan(<<"rr(prim_file).">>), % preloaded... 584: ?line PA = filename:dirname(code:which(?MODULE)), The latter seemed to be the simplest way to test if the code loader is still working. So I sprinkled it into every test function in gen_stream_SUITE.erl and at the end of the last test function. My result: no failures. Try it on your environment and see if you get different results. This last commit is a test proof, it shouldn't be included if there is a graduated release. From pguyot@REDACTED Sat Jul 17 15:07:22 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 17 Jul 2010 15:07:22 +0200 Subject: fix of hipe crashes in gc Message-ID: <928AF5E2-AB0D-4102-9628-B3E4DBE3353D@kallisys.net> Hello, I have been experiencing crashes in the garbage collector when running erlang with natively compiled modules, and especially with OTP configured with --enable-native-libs. This has been discussed before, including by others: http://www.erlang.org/cgi-bin/ezmlm-cgi/4/50033 http://www.erlang.org/cgi-bin/ezmlm-cgi/4/39462 http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1583 I finally got a reproduceable case and nailed the bug down. The bug was introduced in R12B-0, when the function erts_gc_after_bif_call was updated to take 4 parameters but the assembly glue code was not updated to pass proper values for the third and fourth parameters. A fix is available here: git fetch git://github.com/pguyot/otp.git pg/fix-hipe-crash-in-gc_after_bif http://github.com/pguyot/otp/commit/c42ba9962d7e83accec797c56e5480a2d0b6281a Please note that I only tested it on x86 and amd64 platforms. Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From mikpe@REDACTED Sat Jul 17 15:56:29 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 17 Jul 2010 15:56:29 +0200 Subject: [erlang-patches] fix of hipe crashes in gc In-Reply-To: <928AF5E2-AB0D-4102-9628-B3E4DBE3353D@kallisys.net> References: <928AF5E2-AB0D-4102-9628-B3E4DBE3353D@kallisys.net> Message-ID: <19521.46733.210179.226719@pilspetsen.it.uu.se> Paul Guyot writes: > Hello, > > I have been experiencing crashes in the garbage collector when running erlang with natively compiled modules, and especially with OTP configured with --enable-native-libs. > > This has been discussed before, including by others: > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/50033 No test case was ever provided, though in hindsight the "arity=" should have caught my eye. > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/39462 Missed that one. Nothing in $Subject or the first two messages indicated native code, so the third message was likely ignored. Again, "arity=" in the third message is the crucial hint. > http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1583 That one was apparently resolved as being a different problem. Nothing there indicates that ertc_gc_after_bif_call was involved. > > I finally got a reproduceable case and nailed the bug down. The bug was introduced in R12B-0, when the function erts_gc_after_bif_call was updated to take 4 parameters but the assembly glue code was not updated to pass proper values for the third and fourth parameters. > > A fix is available here: > > git fetch git://github.com/pguyot/otp.git pg/fix-hipe-crash-in-gc_after_bif > http://github.com/pguyot/otp/commit/c42ba9962d7e83accec797c56e5480a2d0b6281a Thanks for identifying the cause. The fix looks reasonable, but can you please post it with git-send-email or somesuch, there are coding style details I want you to fix but I cannot accurately describe them without the text of the patch itself. /Mikael (Not directed at you personally, but this is a good example IMO of how the Erlang adaptation of git has failed. Not providing patches inline (together with git links) makes it difficult to review them and discuss specific details. In contrast, the Linux kernel community gets this right.) From pguyot@REDACTED Sat Jul 17 18:32:15 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 17 Jul 2010 18:32:15 +0200 Subject: [erlang-patches] fix of hipe crashes in gc In-Reply-To: <19521.46733.210179.226719@pilspetsen.it.uu.se> References: <928AF5E2-AB0D-4102-9628-B3E4DBE3353D@kallisys.net> <19521.46733.210179.226719@pilspetsen.it.uu.se> Message-ID: Mikael, Thank you for your quick reply. Le 17 juil. 2010 ? 15:56, Mikael Pettersson a ?crit : >> http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1583 > > That one was apparently resolved as being a different problem. > Nothing there indicates that ertc_gc_after_bif_call was involved. It's in the stack trace (element 4). Unlike the other crash logs, the parameters are not displayed, but I suspect it's the same bug. > Thanks for identifying the cause. The fix looks reasonable, but can you > please post it with git-send-email or somesuch, there are coding style > details I want you to fix but I cannot accurately describe them without > the text of the patch itself. I sent the patch in a separate e-mail (I wanted to change the subject line with --compose, but failed). Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From khirin@REDACTED Sun Jul 18 08:13:40 2010 From: khirin@REDACTED (Evgeny Khirin) Date: Sun, 18 Jul 2010 09:13:40 +0300 Subject: filename:join breaks Windows UNC paths Message-ID: <201007180913.40475.khirin@zahav.net.il> Hello, "Normalize" feature of filename:join breaks Windows UNC paths: filename:join("//host/share", "file") returns "/host/share/file" on Windows instead "//host/share/file". Patch below fixes the problem. Regards, Evgeny. ------------------- patch -------------------------------- diff --git a/lib/stdlib/src/filename.erl b/lib/stdlib/src/filename.erl index 01c06e4..8da54ef 100644 --- a/lib/stdlib/src/filename.erl +++ b/lib/stdlib/src/filename.erl @@ -316,6 +316,10 @@ join(Name1, Name2) when is_atom(Name2) -> %% It is the responsibility of the caller to ensure that RelativeName %% is relative. +join1([$/, $/ | Rest], RelativeName, [], win32) -> [$/, $/ | join1(Rest, RelativeName, [], win32)]; +join1([$/, $\\ | Rest], RelativeName, [], win32) -> [$/, $/ | join1(Rest, RelativeName, [], win32)]; +join1([$\\, $/ | Rest], RelativeName, [], win32) -> [$/, $/ | join1(Rest, RelativeName, [], win32)]; +join1([$\\, $\\ | Rest], RelativeName, [], win32) -> [$/, $/ | join1(Rest, RelativeName, [], win32)]; join1([UcLetter, $:|Rest], RelativeName, [], win32) when is_integer(UcLetter), UcLetter >= $A, UcLetter =< $Z -> join1(Rest, RelativeName, [$:, UcLetter+$a-$A], win32); ------------------- patch -------------------------------- From tuncer.ayaz@REDACTED Tue Jul 20 15:32:15 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 20 Jul 2010 15:32:15 +0200 Subject: Correct a couple reltool documentation errors Message-ID: Fix typos and other errors. git fetch git://github.com/tuncer/otp.git reltool-doc From michael.santos@REDACTED Wed Jul 21 17:14:17 2010 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 21 Jul 2010 11:14:17 -0400 Subject: [PATCH] inet: fix ifr_name buffer overflow Message-ID: <20100721151417.GA27040@ecn.lan> The byte holding the length of the interface name for the ifget/2 functions is used in a signed context and can become negative, causing the ifreq.ifr_name buffer to be overrun. Test case: inet:ifget(lists:duplicate(128, "x"), [addr]). --- erts/emulator/drivers/common/inet_drv.c | 6 +++--- lib/kernel/test/inet_SUITE.erl | 12 ++++++++++-- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 0ea5493..e5024d3 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -3905,7 +3905,7 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, INTERFACE_INFO* ifp; long namaddr; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = (unsigned char)buf[0]) > len)) goto error; if (parse_addr(buf+1, namlen, &namaddr) < 0) goto error; @@ -4099,7 +4099,7 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, struct ifreq ifreq; int namlen; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = (unsigned char)buf[0]) > len)) goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, @@ -4252,7 +4252,7 @@ static int inet_ctl_ifset(inet_descriptor* desc, char* buf, int len, int namlen; char* b_end = buf + len; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = (unsigned char)buf[0]) > len)) goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index eb8f918..d475ec7 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -26,7 +26,8 @@ t_gethostbyaddr_v6/1, t_getaddr_v6/1, t_gethostbyname_v6/1, ipv4_to_ipv6/1, host_and_addr/1, parse/1, t_gethostnative/1, gethostnative_parallell/1, cname_loop/1, - gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1]). + gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1, + getif_ifr_name_overflow/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -39,7 +40,7 @@ all(suite) -> ipv4_to_ipv6, host_and_addr, parse,t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level,gethostnative_soft_restart, - getif]. + getif,getif_ifr_name_overflow]. init_per_testcase(_Func, Config) -> Dog = test_server:timetrap(test_server:seconds(60)), @@ -891,6 +892,13 @@ getif(Config) when is_list(Config) -> ?line true = ip_member(Loopback, Addresses), ?line ok. +getif_ifr_name_overflow(doc) -> + "Test long interface names do not overrun buffer"; +getif_ifr_name_overflow(Config) when is_list(Config) -> + %% emulator should not crash + ?line {ok,[]} = inet:ifget(lists:duplicate(128, "x"), [addr]), + ok. + %% Works just like lists:member/2, except that any {127,_,_,_} tuple %% matches any other {127,_,_,_}. We do this to handle Linux systems %% that use (for instance) 127.0.1.1 as the IP address for the hostname. -- 1.5.6.4 From michael.santos@REDACTED Wed Jul 21 17:15:21 2010 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 21 Jul 2010 11:15:21 -0400 Subject: [PATCH] inet: null terminate ifr_name buffer Message-ID: <20100721151521.GA27069@ecn.lan> The buffer holding the interface name should be null terminated. See man 7 socket on Linux or the definiton of IFNAMSIZ in net/if.h on FreeBSD: Length of interface external name, including terminating '\0'. --- erts/emulator/drivers/common/inet_drv.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index e5024d3..72deedf 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -4103,7 +4103,7 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, - (namlen > IFNAMSIZ) ? IFNAMSIZ : namlen); + (namlen >= IFNAMSIZ) ? IFNAMSIZ-1 : namlen); buf += (namlen+1); len -= (namlen+1); sptr = sbuf; @@ -4256,7 +4256,7 @@ static int inet_ctl_ifset(inet_descriptor* desc, char* buf, int len, goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, - (namlen > IFNAMSIZ) ? IFNAMSIZ : namlen); + (namlen >= IFNAMSIZ) ? IFNAMSIZ-1 : namlen); buf += (namlen+1); len -= (namlen+1); -- 1.5.6.4 From michael.santos@REDACTED Wed Jul 21 17:15:52 2010 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 21 Jul 2010 11:15:52 -0400 Subject: [PATCH] inet: fix getservbyname buffer overflow Message-ID: <20100721151552.GA27098@ecn.lan> The byte holding the length of the interface name for the getservbyname/2 function is used in a signed context and can become negative, causing the buffer to be overrun. Make the same change for getservbyport/2. Test case: inet:getservbyname(list_to_atom(lists:flatten(lists:duplicate(128, "x"))), tcp). --- erts/emulator/drivers/common/inet_drv.c | 6 +++--- lib/kernel/test/inet_SUITE.erl | 11 +++++++++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 72deedf..1a6a9dd 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -6847,13 +6847,13 @@ static int inet_ctl(inet_descriptor* desc, int cmd, char* buf, int len, if (len < 2) return ctl_error(EINVAL, rbuf, rsize); - n = buf[0]; buf++; len--; + n = (unsigned char)buf[0]; buf++; len--; if (n >= len) /* the = sign makes the test inklude next length byte */ return ctl_error(EINVAL, rbuf, rsize); memcpy(namebuf, buf, n); namebuf[n] = '\0'; len -= n; buf += n; - n = buf[0]; buf++; len--; + n = (unsigned char)buf[0]; buf++; len--; if (n > len) return ctl_error(EINVAL, rbuf, rsize); memcpy(protobuf, buf, n); @@ -6876,7 +6876,7 @@ static int inet_ctl(inet_descriptor* desc, int cmd, char* buf, int len, port = get_int16(buf); port = sock_htons(port); buf += 2; - n = buf[0]; buf++; len -= 3; + n = (unsigned char)buf[0]; buf++; len -= 3; if (n > len) return ctl_error(EINVAL, rbuf, rsize); memcpy(protobuf, buf, n); diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index d475ec7..86c2a0c 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -27,7 +27,7 @@ ipv4_to_ipv6/1, host_and_addr/1, parse/1, t_gethostnative/1, gethostnative_parallell/1, cname_loop/1, gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1, - getif_ifr_name_overflow/1]). + getif_ifr_name_overflow/1,getservbyname_overflow/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -40,7 +40,7 @@ all(suite) -> ipv4_to_ipv6, host_and_addr, parse,t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level,gethostnative_soft_restart, - getif,getif_ifr_name_overflow]. + getif,getif_ifr_name_overflow,getservbyname_overflow]. init_per_testcase(_Func, Config) -> Dog = test_server:timetrap(test_server:seconds(60)), @@ -899,6 +899,13 @@ getif_ifr_name_overflow(Config) when is_list(Config) -> ?line {ok,[]} = inet:ifget(lists:duplicate(128, "x"), [addr]), ok. +getservbyname_overflow(doc) -> + "Test long service names do not overrun buffer"; +getservbyname_overflow(Config) when is_list(Config) -> + %% emulator should not crash + ?line {error,einval} = inet:getservbyname(list_to_atom(lists:flatten(lists:duplicate(128, "x"))), tcp), + ok. + %% Works just like lists:member/2, except that any {127,_,_,_} tuple %% matches any other {127,_,_,_}. We do this to handle Linux systems %% that use (for instance) 127.0.1.1 as the IP address for the hostname. -- 1.5.6.4 From michael.santos@REDACTED Wed Jul 21 20:50:58 2010 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 21 Jul 2010 14:50:58 -0400 Subject: [PATCH] inet: support retrieving MAC address on BSD Message-ID: <20100721185058.GA26218@ecn.lan> On systems supporting getaddrinfo(), support looking up the MAC address from inet:ifget/2. The results have the same quirks as with Linux: if the MAC address is longer than 6 bytes (e.g., fw0 under Mac OS X), the address is truncated; if the interface does not have a MAC address (e.g., lo0), an address consisting of 0's is returned. --- erts/configure.in | 3 ++ erts/emulator/drivers/common/inet_drv.c | 40 +++++++++++++++++++++++++++--- 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/erts/configure.in b/erts/configure.in index 63bf548..74cb953 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -1764,6 +1764,9 @@ AC_CHECK_FUNCS([fdatasync]) dnl Find which C libraries are required to use fdatasync AC_SEARCH_LIBS(fdatasync, [rt]) +AC_CHECK_HEADERS(net/if_dl.h ifaddrs.h) +AC_CHECK_FUNCS([getifaddrs]) + dnl ---------------------------------------------------------------------- dnl Checks for features/quirks in the system that affects Erlang. dnl ---------------------------------------------------------------------- diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 0ea5493..e55208f 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -48,6 +48,12 @@ #include #endif +#ifdef HAVE_NET_IF_DL_H +#include +#endif +#ifdef HAVE_IFADDRS_H +#include +#endif /* All platforms fail on malloc errors. */ #define FATAL_MALLOC @@ -4089,6 +4095,10 @@ static int inet_ctl_getiflist(inet_descriptor* desc, char** rbuf, int rsize) } +/* FIXME: temporary hack */ +#ifndef IFHWADDRLEN +#define IFHWADDRLEN 6 +#endif static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, char** rbuf, int rsize) @@ -4128,6 +4138,32 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, /* raw memcpy (fix include autoconf later) */ sys_memcpy(sptr, (char*)(&ifreq.ifr_hwaddr.sa_data), IFHWADDRLEN); sptr += IFHWADDRLEN; +#elif defined(HAVE_GETIFADDRS) + struct ifaddrs *ifa, *ifp; + int found = 0; + + if (getifaddrs(&ifa) == -1) + goto error; + + for (ifp = ifa; ifp; ifp = ifp->ifa_next) { + if ((ifp->ifa_addr->sa_family == AF_LINK) && + (sys_strcmp(ifp->ifa_name, ifreq.ifr_name) == 0)) { + found = 1; + break; + } + } + + if (found == 0) { + freeifaddrs(ifa); + break; + } + + buf_check(sptr, s_end, 1+IFHWADDRLEN); + *sptr++ = INET_IFOPT_HWADDR; + sys_memcpy(sptr, ((struct sockaddr_dl *)ifp->ifa_addr)->sdl_data + + ((struct sockaddr_dl *)ifp->ifa_addr)->sdl_nlen, IFHWADDRLEN); + freeifaddrs(ifa); + sptr += IFHWADDRLEN; #endif break; } @@ -4240,10 +4276,6 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, return ctl_error(EINVAL, rbuf, rsize); } -/* FIXME: temporary hack */ -#ifndef IFHWADDRLEN -#define IFHWADDRLEN 6 -#endif static int inet_ctl_ifset(inet_descriptor* desc, char* buf, int len, char** rbuf, int rsize) -- 1.5.6.4 From michael.santos@REDACTED Thu Jul 22 02:44:33 2010 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 21 Jul 2010 20:44:33 -0400 Subject: [PATCH] Fix typos in the distribution handshake doc Message-ID: <20100722004433.GA26873@ecn.lan> --- lib/kernel/internal_doc/distribution_handshake.txt | 38 ++++++++++---------- 1 files changed, 19 insertions(+), 19 deletions(-) diff --git a/lib/kernel/internal_doc/distribution_handshake.txt b/lib/kernel/internal_doc/distribution_handshake.txt index f64ebe0..aea290f 100644 --- a/lib/kernel/internal_doc/distribution_handshake.txt +++ b/lib/kernel/internal_doc/distribution_handshake.txt @@ -11,7 +11,7 @@ The TCP/IP distribution uses a handshake which expects a connection based protocol, i.e. the protocol does not include any authentication after the handshake procedure. -This is not entirelly safe, as it is vulnerable against takeover +This is not entirely safe, as it is vulnerable against takeover attacks, but it is a tradeoff between fair safety and performance. The cookies are never sent in cleartext and the handshake procedure @@ -23,7 +23,7 @@ random numbers. DEFINITIONS ----------- -A challenge is a 32 bit integer number in big endian. Below the function +A challenge is a 32 bit integer number in big endian order. Below the function gen_challenge() returns a random 32 bit integer used as a challenge. A digest is a (16 bytes) MD5 hash of [the Challenge (as text) concatenated @@ -46,19 +46,19 @@ The cookies are text strings that can be viewed as passwords. Every message in the handshake starts with a 16 bit big endian integer which contains the length of the message (not counting the two initial bytes). In erlang this corresponds to the gen_tcp option {packet, 2}. Note that after -the handshake, the distribution switches to 4 byte backet headers. +the handshake, the distribution switches to 4 byte packet headers. THE HANDSHAKE IN DETAIL ----------------------- -Imagine two nodes, node A, which initiates the handshake and node B, whitch +Imagine two nodes, node A, which initiates the handshake and node B, which accepts the connection. 1) connect/accept: A connects to B via TCP/IP and B accepts the connection. 2) send_name/receive_name: A sends an initial identification to B. B receives the message. The message looks -like this (every "square" beeing one byte and the packet header removed): +like this (every "square" being one byte and the packet header removed): +---+--------+--------+-----+-----+-----+-----+-----+-----+-...-+-----+ |'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Name0|Name1| ... |NameN| @@ -67,7 +67,7 @@ like this (every "square" beeing one byte and the packet header removed): The 'n' is just a message tag, Version0 & Version1 is the distribution version selected by node A, based on information from EPMD. (16 bit big endian) -Flag0 ... Flag3 is capability flags, the capabilities defined in dist.hrl. +Flag0 ... Flag3 are capability flags, the capabilities defined in dist.hrl. (32 bit big endian) Name0 ... NameN is the full nodename of A, as a string of bytes (the packet length denotes how long it is). @@ -91,9 +91,9 @@ alive: A connection to the node is already active, which either means This is the format of the status message: -+---+-------+-------+ ... +-------+ ++---+-------+-------+-...-+-------+ |'s'|Status0|Status1| ... |StatusN| -+---+-------+-------+ ... +-------+ ++---+-------+-------+-...-+-------+ 's' is the message tag Status0 ... StatusN is the status as a string (not terminated) @@ -111,35 +111,35 @@ initially sent from A to B, with the addition of a 32 bit challenge: +---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+--- |'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Chal0|Chal1|Chal2|Chal3| -+---+--------+--------+-----+-----+-----+-----+-----+-----+---- +-----+--- ++---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+--- ------+-----+-...-+-----+ Name0|Name1| ... |NameN| ------+-----+-... +-----+ -Where Chal0 ... Chal3 is the challenge as a 32 bit biog endian integer +Where Chal0 ... Chal3 is the challenge as a 32 bit big endian integer and the other fields are B's version, flags and full nodename. 5) send_challenge_reply/recv_challenge_reply: Now A has generated a digest and it's own challenge. Those are sent together in a package to B: -+---+-----+-----+-----+-----+-----+-----+-----+-----+ -|'r'|Chal0|Chal1|Chal2|Chal3|Dige0|Dige1|Dige2|Dige3| -+---+-----+-----+-----+-----+-----+-----+---- +-----+ ++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+ +|'r'|Chal0|Chal1|Chal2|Chal3|Dige0|Dige1|Dige2|Dige3| ... |Dige15| ++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+ Where 'r' is the tag, Chal0 ... Chal3 is A's challenge for B to handle and -Dige0 ... Dige3 is the digest that A constructed from the challenge B sent +Dige0 ... Dige15 is the digest that A constructed from the challenge B sent in the previous step. 6) recv_challenge_ack/send_challenge_ack: B checks that the digest received from A is correct and generates a digest from the challenge received from A. The digest is then sent to A. The message looks like this: -+---+-----+-----+-----+-----+ -|'a'|Dige0|Dige1|Dige2|Dige3| -+---+-----+-----+---- +-----+ ++---+-----+-----+-----+-----+-...-+------+ +|'a'|Dige0|Dige1|Dige2|Dige3| ... |Dige15| ++---+-----+-----+-----+-----+-...-+------+ -Where 'a' is the tag and Dige0 ... Dige3 is the digest calculated by B +Where 'a' is the tag and Dige0 ... Dige15 is the digest calculated by B for A's challenge. 7) A checks the digest from B and the connection is up. @@ -206,7 +206,7 @@ Currently the following capability flags are defined: %% The node implements distributed process monitoring. -define(DFLAG_DIST_MONITOR,8). -%% The node uses separate tag for fun's (labmdas) in the distribution protocol. +%% The node uses separate tag for fun's (lambdas) in the distribution protocol. -define(DFLAG_FUN_TAGS,16). An R6 erlang node implements all of the above, while a C or Java node only -- 1.7.0.4 From michael.santos@REDACTED Fri Jul 23 17:26:44 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 23 Jul 2010 11:26:44 -0400 Subject: [PATCH] inet: fix ifr_name buffer overflow In-Reply-To: <20100721151417.GA27040@ecn.lan> References: <20100721151417.GA27040@ecn.lan> Message-ID: <20100723152644.GA30765@ecn.lan> On Wed, Jul 21, 2010 at 11:14:17AM -0400, Michael Santos wrote: > - if ((len == 0) || ((namlen = buf[0]) > len)) > + if ((len == 0) || ((namlen = (unsigned char)buf[0]) > len)) I just discovered the get_int8() macro. I'll update the patch and re-send. Sorry for the noise. From michael.santos@REDACTED Fri Jul 23 17:28:46 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 23 Jul 2010 11:28:46 -0400 Subject: [PATCH] inet: fix ifr_name buffer overflow Message-ID: <20100723152846.GE30790@ecn.lan> The byte holding the length of the interface name for the ifget/2 functions is used in a signed context and can become negative, causing the ifreq.ifr_name buffer to be overrun. Test case: inet:ifget(lists:duplicate(128, "x"), [addr]). --- erts/emulator/drivers/common/inet_drv.c | 6 +++--- lib/kernel/test/inet_SUITE.erl | 12 ++++++++++-- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 0ea5493..e5024d3 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -3905,7 +3905,7 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, INTERFACE_INFO* ifp; long namaddr; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = get_int8(buf)) > len)) goto error; if (parse_addr(buf+1, namlen, &namaddr) < 0) goto error; @@ -4099,7 +4099,7 @@ static int inet_ctl_ifget(inet_descriptor* desc, char* buf, int len, struct ifreq ifreq; int namlen; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = get_int8(buf)) > len)) goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, @@ -4252,7 +4252,7 @@ static int inet_ctl_ifset(inet_descriptor* desc, char* buf, int len, int namlen; char* b_end = buf + len; - if ((len == 0) || ((namlen = buf[0]) > len)) + if ((len == 0) || ((namlen = get_int8(buf)) > len)) goto error; sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); sys_memcpy(ifreq.ifr_name, buf+1, diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index eb8f918..d475ec7 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -26,7 +26,8 @@ t_gethostbyaddr_v6/1, t_getaddr_v6/1, t_gethostbyname_v6/1, ipv4_to_ipv6/1, host_and_addr/1, parse/1, t_gethostnative/1, gethostnative_parallell/1, cname_loop/1, - gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1]). + gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1, + getif_ifr_name_overflow/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -39,7 +40,7 @@ all(suite) -> ipv4_to_ipv6, host_and_addr, parse,t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level,gethostnative_soft_restart, - getif]. + getif,getif_ifr_name_overflow]. init_per_testcase(_Func, Config) -> Dog = test_server:timetrap(test_server:seconds(60)), @@ -891,6 +892,13 @@ getif(Config) when is_list(Config) -> ?line true = ip_member(Loopback, Addresses), ?line ok. +getif_ifr_name_overflow(doc) -> + "Test long interface names do not overrun buffer"; +getif_ifr_name_overflow(Config) when is_list(Config) -> + %% emulator should not crash + ?line {ok,[]} = inet:ifget(lists:duplicate(128, "x"), [addr]), + ok. + %% Works just like lists:member/2, except that any {127,_,_,_} tuple %% matches any other {127,_,_,_}. We do this to handle Linux systems %% that use (for instance) 127.0.1.1 as the IP address for the hostname. -- 1.5.6.4 From michael.santos@REDACTED Fri Jul 23 17:29:20 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 23 Jul 2010 11:29:20 -0400 Subject: [PATCH] inet: fix getservbyname buffer overflow Message-ID: <20100723152920.GF30790@ecn.lan> The byte holding the length of the interface name for the getservbyname/2 function is used in a signed context and can become negative, causing the buffer to be overrun. Make the same change for getservbyport/2. Test case: inet:getservbyname(list_to_atom(lists:flatten(lists:duplicate(128, "x"))), tcp). --- erts/emulator/drivers/common/inet_drv.c | 6 +++--- lib/kernel/test/inet_SUITE.erl | 11 +++++++++-- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 72deedf..1a6a9dd 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -6847,13 +6847,13 @@ static int inet_ctl(inet_descriptor* desc, int cmd, char* buf, int len, if (len < 2) return ctl_error(EINVAL, rbuf, rsize); - n = buf[0]; buf++; len--; + n = get_int8(buf); buf++; len--; if (n >= len) /* the = sign makes the test inklude next length byte */ return ctl_error(EINVAL, rbuf, rsize); memcpy(namebuf, buf, n); namebuf[n] = '\0'; len -= n; buf += n; - n = buf[0]; buf++; len--; + n = get_int8(buf); buf++; len--; if (n > len) return ctl_error(EINVAL, rbuf, rsize); memcpy(protobuf, buf, n); @@ -6876,7 +6876,7 @@ static int inet_ctl(inet_descriptor* desc, int cmd, char* buf, int len, port = get_int16(buf); port = sock_htons(port); buf += 2; - n = buf[0]; buf++; len -= 3; + n = get_int8(buf); buf++; len -= 3; if (n > len) return ctl_error(EINVAL, rbuf, rsize); memcpy(protobuf, buf, n); diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index d475ec7..86c2a0c 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -27,7 +27,7 @@ ipv4_to_ipv6/1, host_and_addr/1, parse/1, t_gethostnative/1, gethostnative_parallell/1, cname_loop/1, gethostnative_soft_restart/1,gethostnative_debug_level/1,getif/1, - getif_ifr_name_overflow/1]). + getif_ifr_name_overflow/1,getservbyname_overflow/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -40,7 +40,7 @@ all(suite) -> ipv4_to_ipv6, host_and_addr, parse,t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level,gethostnative_soft_restart, - getif,getif_ifr_name_overflow]. + getif,getif_ifr_name_overflow,getservbyname_overflow]. init_per_testcase(_Func, Config) -> Dog = test_server:timetrap(test_server:seconds(60)), @@ -899,6 +899,13 @@ getif_ifr_name_overflow(Config) when is_list(Config) -> ?line {ok,[]} = inet:ifget(lists:duplicate(128, "x"), [addr]), ok. +getservbyname_overflow(doc) -> + "Test long service names do not overrun buffer"; +getservbyname_overflow(Config) when is_list(Config) -> + %% emulator should not crash + ?line {error,einval} = inet:getservbyname(list_to_atom(lists:flatten(lists:duplicate(128, "x"))), tcp), + ok. + %% Works just like lists:member/2, except that any {127,_,_,_} tuple %% matches any other {127,_,_,_}. We do this to handle Linux systems %% that use (for instance) 127.0.1.1 as the IP address for the hostname. -- 1.5.6.4 From bgustavsson@REDACTED Sun Jul 25 15:23:14 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 25 Jul 2010 15:23:14 +0200 Subject: [erlang-patches] Possible BEAM compiler patch In-Reply-To: <1277376592.3640.12.camel@tri10.nomadiclab.com> References: <1277376592.3640.12.camel@tri10.nomadiclab.com> Message-ID: On Thu, Jun 24, 2010 at 12:49 PM, Martti Kuparinen wrote: > Hi, > > The NetBSD pkgsrc has this patch (pkgsrc/lang/erlang/patches/patch-ah) > > > --- lib/compiler/src/v3_codegen.erl.orig ? ? ? ?2008-02-05 > 08:37:35.000000000 -0 > 500 > +++ lib/compiler/src/v3_codegen.erl > @@ -1481,6 +1481,8 @@ cg_binary_size_1([], Bits, Acc) -> > ? ? Sizes = filter(fun({_,{integer,0}}) -> false; > ? ? ? ? ? ? ? ? ? ? ?(_) -> true end, Sizes0), > ? ? case Sizes of > + ? ? ? [] -> > + ? ? ? ? ? {bs_init2,[{integer,0}]}; > ? ? ? ?[{1,_}|_] -> > ? ? ? ? ? ?{bs_init_bits,cg_binary_bytes_to_bits(Sizes, [])}; > ? ? ? ?[{8,_}|_] -> > > > but I don't know it that makes any sense or not. Could someone from the > compiler team look at this and say if I should nuke this or submit a > proper patch request so it can be included in the pu branch... > Thanks! It makes sense. Please include a test case in the patch to ensure that the new clause will be tested. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Sun Jul 25 16:24:08 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 25 Jul 2010 16:24:08 +0200 Subject: doc-fixes Message-ID: When I see errors in documentation while reading I have to correct them. It's a habit :). Therefore I've created the doc-fixes branch which also contains the previous reltool fixes from the reltool-doc branch. You can ignore the previous reltool-doc branch as it's deleted by now. If you prefer to have one branch per application modified I can split the patches but don't think that would make sense. git fetch git://github.com/tuncer/otp.git doc-fixes From tuncer.ayaz@REDACTED Sun Jul 25 16:25:51 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 25 Jul 2010 16:25:51 +0200 Subject: Correct a couple reltool documentation errors In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 3:32 PM, Tuncer Ayaz wrote: > Fix typos and other errors. > > git fetch git://github.com/tuncer/otp.git reltool-doc Please ignore and use doc-fixes branch instead. From bgustavsson@REDACTED Mon Jul 26 12:13:50 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 26 Jul 2010 12:13:50 +0200 Subject: Regarding mk/net-dragonfly-bsd-patches Message-ID: We have found some serious issues with the changes to the Makefile for the odbc applications and therefore I have now removed that commit from the mk/net-dragonfly-bsd-patches branch included in pu. (I will push out the updated pu branch later today.) Specifically: 1. We have have a policy against using the -R flag because it can cause problems when Erlang/OTP is built on one computer and run on another. 2. ld on Mac OS X does not support the -R flag, so the build will fail. 3. The build will fail if PREFIX is empty. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Jul 26 12:22:41 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 26 Jul 2010 12:22:41 +0200 Subject: [erlang-patches] doc-fixes In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 4:24 PM, Tuncer Ayaz wrote: > > git fetch git://github.com/tuncer/otp.git doc-fixes > Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Jul 26 14:45:32 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 26 Jul 2010 14:45:32 +0200 Subject: [erlang-patches] [PATCH] Fix typos in the distribution handshake doc In-Reply-To: <20100722004433.GA26873@ecn.lan> References: <20100722004433.GA26873@ecn.lan> Message-ID: On Thu, Jul 22, 2010 at 2:44 AM, Michael Santos wrote: > --- > lib/kernel/internal_doc/distribution_handshake.txt | 38 ++++++++++---------- > 1 files changed, 19 insertions(+), 19 deletions(-) > Thanks! Will apply directly to dev, with this additional correction squashed in: diff --git a/lib/kernel/internal_doc/distribution_handshake.txt b/lib/kernel/internal_doc/distribution_handshake.txt index aea290f..6a3ee22 100644 --- a/lib/kernel/internal_doc/distribution_handshake.txt +++ b/lib/kernel/internal_doc/distribution_handshake.txt @@ -120,7 +120,7 @@ Where Chal0 ... Chal3 is the challenge as a 32 bit big endian integer and the other fields are B's version, flags and full nodename. 5) send_challenge_reply/recv_challenge_reply: Now A has generated -a digest and it's own challenge. Those are sent together in a package +a digest and its own challenge. Those are sent together in a package to B: +---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+ -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Jul 26 14:47:38 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 26 Jul 2010 14:47:38 +0200 Subject: [erlang-patches] [PATCH] inet: fix getservbyname buffer overflow In-Reply-To: <20100723152920.GF30790@ecn.lan> References: <20100723152920.GF30790@ecn.lan> Message-ID: On Fri, Jul 23, 2010 at 5:29 PM, Michael Santos wrote: > The byte holding the length of the interface name for the getservbyname/2 > function is used in a signed context and can become negative, causing > the buffer to be overrun. Make the same change for getservbyport/2. > Thanks! I have put this patch and the other two inet patches in a branch called ms/inet-bug-fixes and it will be included in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Jul 26 15:43:07 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 26 Jul 2010 15:43:07 +0200 Subject: pu has been updated Message-ID: pu has now been updated. All patches sent the last week should now be included. Please send me an email if I have forgotten to include any patch. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Jul 28 09:03:08 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 28 Jul 2010 09:03:08 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: On Sun, Jul 11, 2010 at 8:00 PM, Paul Guyot wrote: > The native chunks of modules loaded before the code server are never loaded, which makes it pretty pointless to natively compile them in the first place (with --enable-native-libs). A patch that loads them just after the code server is started can be found here: > > git fetch git://github.com/pguyot/otp.git pg/load-native-code-for-early-modules > > http://github.com/pguyot/otp/commit/2637b8a24e2d9ba3653e580f21a70976915dc201 > > As a side effect, this patch will decrease the load time of dialyzer on installations configured with --enable-native-libs. Indeed, dialyzer natively recompiles modules including some that were loaded before the code server (such as lists, gb_trees, etc.). > Your added code is located in the code_server module, but it is not executed in the code_server process. This is confusing for both human readers and a test case called code_SUITE:ext_mod_dep/1 which attempts to find unsafe calls in the code_server process. Therefore, I suggest that you move the code to the 'code' module (putting the call to load_native_code_for_all_loaded/0 in the do_start/1 function). Minor style nit: You don't need parenthesis around a catch unless you want to assign the result to a variable. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From pguyot@REDACTED Wed Jul 28 10:41:36 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 28 Jul 2010 10:41:36 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: Le 28 juil. 2010 ? 09:03, Bj?rn Gustavsson a ?crit : > Your added code is located in the code_server module, > but it is not executed in the code_server process. > > This is confusing for both human readers and a test > case called code_SUITE:ext_mod_dep/1 which attempts > to find unsafe calls in the code_server process. > > Therefore, I suggest that you move the code to the 'code' > module (putting the call to load_native_code_for_all_loaded/0 > in the do_start/1 function). > > Minor style nit: You don't need parenthesis around a catch > unless you want to assign the result to a variable. Thank you for the review! I have just committed suggested changes here: http://github.com/pguyot/otp/commit/a30b889c01cb8cc9e0ad021f41b7ed27cf97fed7 I also added a non-regression test: http://github.com/pguyot/otp/commit/5f23082a8d842ca0a414748e011c4b10240c8fcc Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From bgustavsson@REDACTED Wed Jul 28 15:14:21 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 28 Jul 2010 15:14:21 +0200 Subject: [erlang-patches] Optimization of beams string table generation In-Reply-To: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> References: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> Message-ID: On Sun, Jul 4, 2010 at 3:19 PM, Paul Guyot wrote: > Hello, > > beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2. > > git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table > > http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a > Can you give some more information about the claims in the commit message? I did the following in $ERL_TOP/release/tests/stdlib_test (using the R14A compiler): erlc +time -I ../test_server re_testoutput1_split_test.erl The interesting part of the output are the following lines: core_module : 10.26 s 60987.5 kB v3_codegen : 5.83 s 17908.4 kB beam_asm : 0.46 s 2.5 kB The module contains several thousand strings that are used in bit syntax matching. The core_module and v3_codegen passes are very slow. The beam_asm pass is slower than usual, but fast compared to the slow passes. Do you have a module that will make the beam_asm pass very slow? If not, the change is still interesting because the new code is shorter and easier to understand, but the commit message must be revised. Regarding the implementation, it will probably be faster to append new strings to the string table like this: NewDict = Dict#asm{strings = <>, string_offset=NextOffset+byte_size(StrBin)}, Regarding the test case, a better place for it should be the compilation_SUITE module. (The name of the test modules may not be the best, but compile_SUITE (mainly) tests different compiler options, while compilation_SUITE tests miscellanous language features.) -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Jul 28 15:37:24 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 28 Jul 2010 15:37:24 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: 2010/7/28 Paul Guyot : > > Thank you for the review! > > I have just committed suggested changes here: > http://github.com/pguyot/otp/commit/a30b889c01cb8cc9e0ad021f41b7ed27cf97fed7 > > I also added a non-regression test: > http://github.com/pguyot/otp/commit/5f23082a8d842ca0a414748e011c4b10240c8fcc I used "git rebase -i" and combined all three commits to one. (We are not interested in keeping the development history of a patch.) I have also updated the test case so that it will be skipped if the Erlang/OTP does not have hipe support enabled, and broken some long lines: http://github.com/erlang/otp/commit/a8b8ec5e858da86531933b545f752f436e411b58 If that is OK to you and if there will be no problems in our daily builds, I might merge this branch to dev tomorrow. Can you fix the test cases in pg/fix-hipe-load and pg/hipe_crash_with_on_load so that they are skipped if the Erlang/OTP system does not have hipe support enabled? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Jul 28 15:54:59 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 28 Jul 2010 15:54:59 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: 2010/7/28 Bj?rn Gustavsson : > Can you fix the test cases in pg/fix-hipe-load and pg/hipe_crash_with_on_load > so that they are skipped if the Erlang/OTP system does not have hipe > support enabled? Thinking a little bit more, I realize that if every test case in a module should be skipped if a certain condition, there are better (but non-obvious) ways than to change every single test case. So I suggest that you'll wait and I'll have look at how to best do the skipping tomorrow. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From pguyot@REDACTED Wed Jul 28 17:26:14 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 28 Jul 2010 17:26:14 +0200 Subject: [erlang-patches] Optimization of beams string table generation In-Reply-To: References: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> Message-ID: Le 28 juil. 2010 ? 15:14, Bj?rn Gustavsson a ?crit : > Can you give some more information about the claims in the commit > message? > > I did the following in $ERL_TOP/release/tests/stdlib_test > (using the R14A compiler): > > erlc +time -I ../test_server re_testoutput1_split_test.erl > > The interesting part of the output are the following lines: > > core_module : 10.26 s 60987.5 kB > v3_codegen : 5.83 s 17908.4 kB > beam_asm : 0.46 s 2.5 kB > > The module contains several thousand strings > that are used in bit syntax matching. The core_module > and v3_codegen passes are very slow. The beam_asm > pass is slower than usual, but fast compared to the > slow passes. I was not aware of +time, and found out about the slowness of beam_asm using fcover and compiling a module with several thousands of clauses like: f(<<"binary string">>) -> atom; With 1000 of such clauses: lists:foreach(fun(_) -> <> = crypto:rand_bytes(10), S = erlang:integer_to_list(I, 16), io:format("f(<<\"~s\">>) -> atom~s;\n", [S, S]) end, lists:seq(0, 1000)). I get: core_module : 0.05 s 25922.5 kB v3_codegen : 0.09 s 6660.6 kB beam_asm : 0.28 s 2.6 kB vs core_module : 0.06 s 20558.1 kB v3_codegen : 0.08 s 6617.9 kB beam_asm : 0.03 s 1.7 kB with the patch (both installations are configured with --enable-native). > Regarding the implementation, it will probably be > faster to append new strings to the string table > like this: > > NewDict = Dict#asm{strings = <>, > string_offset=NextOffset+byte_size(StrBin)}, > > Regarding the test case, a better place for it should be the > compilation_SUITE module. > > (The name of the test modules may not be the best, but > compile_SUITE (mainly) tests different compiler options, > while compilation_SUITE tests miscellanous language features.) Do you want me to update the branch ? Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From pguyot@REDACTED Wed Jul 28 17:27:27 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 28 Jul 2010 17:27:27 +0200 Subject: [erlang-patches] load native code for early modules In-Reply-To: References: Message-ID: Le 28 juil. 2010 ? 15:54, Bj?rn Gustavsson a ?crit : > 2010/7/28 Bj?rn Gustavsson : > >> Can you fix the test cases in pg/fix-hipe-load and pg/hipe_crash_with_on_load >> so that they are skipped if the Erlang/OTP system does not have hipe >> support enabled? > > Thinking a little bit more, I realize that if every test case > in a module should be skipped if a certain condition, > there are better (but non-obvious) ways than to change > every single test case. > > So I suggest that you'll wait and I'll have look at how > to best do the skipping tomorrow. OK! Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From tom@REDACTED Wed Jul 28 21:00:02 2010 From: tom@REDACTED (Tom Moertel) Date: Wed, 28 Jul 2010 15:00:02 -0400 Subject: Fix cover's HTML-generation logic to escape ampersands in source code Message-ID: [I'm resubmitting this patch since it never made it to the list the first time around. --Tom] When cover generates HTML files and embeds source code within them, it does not escape ampersands in the source code. These bare ampersands get misinterpreted by web browsers as HTML character-entity references; as a result, source code containing ampersands renders improperly. This small patch fixes the problem by causing bare ampersands to be escaped into as "&" character-entity references. git fetch git://github.com/tmoertel/otp.git fix_cover_html_escaping It's a tiny patch, so I'm enclosing it, too. Cheers, Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-cover-s-HTML-generation-logic-to-escape-ampersan.patch Type: text/x-patch Size: 1239 bytes Desc: not available URL: From bgustavsson@REDACTED Thu Jul 29 09:36:08 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 29 Jul 2010 09:36:08 +0200 Subject: [erlang-patches] Optimization of beams string table generation In-Reply-To: References: <054C718E-0EAD-4501-B0AF-92FD929B1F60@kallisys.net> Message-ID: 2010/7/28 Paul Guyot : > With 1000 of such clauses: > lists:foreach(fun(_) -> <> = crypto:rand_bytes(10), S = erlang:integer_to_list(I, 16), io:format("f(<<\"~s\">>) -> atom~s;\n", [S, S]) end, lists:seq(0, 1000)). > > I get: > ?core_module ? ? ? ? ? ? ? ? ? : ? ? ? 0.05 s ? 25922.5 kB > ?v3_codegen ? ? ? ? ? ? ? ? ? ?: ? ? ? 0.09 s ? ?6660.6 kB > ?beam_asm ? ? ? ? ? ? ? ? ? ? ?: ? ? ? 0.28 s ? ? ? 2.6 kB > > vs > ?core_module ? ? ? ? ? ? ? ? ? : ? ? ? 0.06 s ? 20558.1 kB > ?v3_codegen ? ? ? ? ? ? ? ? ? ?: ? ? ? 0.08 s ? ?6617.9 kB > ?beam_asm ? ? ? ? ? ? ? ? ? ? ?: ? ? ? 0.03 s ? ? ? 1.7 kB > I get similar figures (without any native code). >> Regarding the implementation, it will probably be >> faster to append new strings to the string table >> like this: >> >> ? ? ? ? ? NewDict = Dict#asm{strings = <>, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?string_offset=NextOffset+byte_size(StrBin)}, Compiling a module with 5000 clauses I can confirm that this version is faster. >> >> Regarding the test case, a better place for it should be the >> compilation_SUITE module. >> >> (The name of the test modules may not be the best, but >> compile_SUITE (mainly) tests different compiler options, >> while compilation_SUITE tests miscellanous language features.) > > Do you want me to update the branch ? Normally I would ask you to update the branch yourself. But since I have already done measurements using your branch, it was easy enough for me to do the changes and revise the commit message to emphasize the simplification of the code rather than the optimization. Here is the updated version (so far only in my own github repository): http://github.com/bjorng/otp/commit/173d1fd1c3fef385f73accc4b2bbb1b6f92ac3f5 If you approve this version, I will include it in 'pu' later today. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Jul 29 12:22:11 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 29 Jul 2010 12:22:11 +0200 Subject: [erlang-patches] Fix cover's HTML-generation logic to escape ampersands in source code In-Reply-To: References: Message-ID: On Wed, Jul 28, 2010 at 9:00 PM, Tom Moertel wrote: > git fetch git://github.com/tmoertel/otp.git fix_cover_html_escaping Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Jul 29 16:08:30 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 29 Jul 2010 16:08:30 +0200 Subject: Updated branches Message-ID: I have updated the following branches included in pu: pg/fix-hipe-load: http://github.com/erlang/otp/commit/919619576f587055b7c7da070cb4903042c66601 pg/hipe_crash_with_on_load: http://github.com/erlang/otp/commit/8708fa492b19ad247a4939e266637d97c9fe1fbb Added copyright notices to new files and made sure that the hipe-specific test cases are skipped if run-time system is not hipe-enabled. pg/optimize_beam_dict_string_table: http://github.com/erlang/otp/commit/173d1fd1c3fef385f73accc4b2bbb1b6f92ac3f5 The changes are described in a previous email. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jay@REDACTED Fri Jul 30 17:58:39 2010 From: jay@REDACTED (jay@REDACTED) Date: Fri, 30 Jul 2010 08:58:39 -0700 (PDT) Subject: Any help with gen_stream would be appreciated Message-ID: <61245.64.81.32.110.1280505519.squirrel@www.5x5.net> I would like to see gen_stream get into R14B if possible. I can't diagnose because my shell_SUITE fails every time I run it, even standalone. Anyone out there who can reproduce the following? 1) shell_SUITE standalone passes 2) gen_stream followed by shell_SUITE causes failure - or - 3) Using jn/gen_stream, gen_stream_SUITE fails The last check in I sprinkled in the failing shell_SUITE command to find a test case which triggers it, but I can't get it to fail in my environment. tia, jay