From juhani@REDACTED Fri Oct 1 06:24:25 2010 From: juhani@REDACTED (=?ISO-8859-1?Q?Juhani_R=E4nkimies?=) Date: Fri, 1 Oct 2010 07:24:25 +0300 Subject: Fix appending to large files (>4GB) on Windows Message-ID: Append mode doesn't work for files larger that 4GB on Windows. The problem occurs when the file is larger than 4GB at opening time. If it's smaller at opening time, it can grow beyond 4GB. I suspect that passing NULL lpDistanceToMoveHigh to SetFilePointer in the file driver causes moves larger that 2^32 to fail. http://msdn.microsoft.com/EN-US/library/aa365541.aspx git fetch git://github.com/juranki/otp.git windows_file_append_2 http://github.com/juranki/otp/tree/windows_file_append_2 Cliff Williams identified the problem when debugging couchdb: http://mail-archives.apache.org/mod_mbox/couchdb-user/201009.mbox/%3C4CA3520C.7050606@REDACTED%3E -juhani -- Juhani R?nkimies +358 40 762 4323 From bgustavsson@REDACTED Fri Oct 1 15:07:47 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 1 Oct 2010 15:07:47 +0200 Subject: [erlang-patches] Fix appending to large files (>4GB) on Windows In-Reply-To: References: Message-ID: On Fri, Oct 1, 2010 at 6:24 AM, Juhani R?nkimies wrote: > Append mode doesn't work for files larger that 4GB on Windows. The > problem occurs when the file is larger than 4GB at opening time. If > it's smaller at opening time, it can grow beyond 4GB. > > I suspect that passing NULL lpDistanceToMoveHigh to SetFilePointer in > the file driver causes moves larger that 2^32 to fail. > http://msdn.microsoft.com/EN-US/library/aa365541.aspx > > git fetch git://github.com/juranki/otp.git windows_file_append_2 > Thanks! Included in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Fri Oct 1 15:09:02 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 1 Oct 2010 15:09:02 +0200 Subject: [erlang-bugs] hibernate/3 and HiPE In-Reply-To: <11B31815-6B3D-44C4-819B-7E9764406984@kallisys.net> References: <509D06B0-008C-4AFE-B356-0FBBBDA71086@kallisys.net> <94679BA9-92D1-4A8E-938D-E401C918C570@kallisys.net> <19617.60761.599137.961032@pilspetsen.it.uu.se> <6C940E94-C9AD-4441-84DE-B34201F6A791@kallisys.net> <19619.9805.31482.502922@pilspetsen.it.uu.se> <11B31815-6B3D-44C4-819B-7E9764406984@kallisys.net> Message-ID: On Thu, Sep 30, 2010 at 10:28 AM, Paul Guyot wrote: > Hello, > > The following branch contains fixes for hibernate/3 and HiPE: > http://github.com/pguyot/otp/commits/fix-hibernate-with-hipe > git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe > Thanks! pu updated. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kostis@REDACTED Fri Oct 1 15:56:45 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 01 Oct 2010 16:56:45 +0300 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: <4CA5E89D.1040807@cs.ntua.gr> Bj?rn Gustavsson wrote: > On Thu, Sep 30, 2010 at 1:22 PM, Tuncer Ayaz wrote: >> dialyzer_plt:included_files/1 calls get_record_from_file/1 and may >> therefore also return {error, not_valid}. > > Thanks! Will include in pu. Thanks to Tuncer for noticing and fixing this issue! However, I would prefer that the following patch gets pushed to 'dev' instead of his (ta/fix-dialyzer_plt-spec): git fetch git://github.com/kostis/otp.git dialyzer_plt-type Kostis From fdmanana@REDACTED Fri Oct 1 16:12:45 2010 From: fdmanana@REDACTED (Filipe David Manana) Date: Fri, 1 Oct 2010 15:12:45 +0100 Subject: [erlang-patches] Fix appending to large files (>4GB) on Windows In-Reply-To: References: Message-ID: Awesome. Thanks a lot for the patch :) 2010/10/1 Bj?rn Gustavsson : > On Fri, Oct 1, 2010 at 6:24 AM, Juhani R?nkimies wrote: >> Append mode doesn't work for files larger that 4GB on Windows. The >> problem occurs when the file is larger than 4GB at opening time. If >> it's smaller at opening time, it can grow beyond 4GB. >> >> I suspect that passing NULL lpDistanceToMoveHigh to SetFilePointer in >> the file driver causes moves larger that 2^32 to fail. >> http://msdn.microsoft.com/EN-US/library/aa365541.aspx >> >> git fetch git://github.com/juranki/otp.git windows_file_append_2 >> > > Thanks! Included in pu. > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > > -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From tuncer.ayaz@REDACTED Fri Oct 1 16:13:33 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 1 Oct 2010 16:13:33 +0200 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: <4CA5E89D.1040807@cs.ntua.gr> References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> <4CA5E89D.1040807@cs.ntua.gr> Message-ID: 2010/10/1 Kostis Sagonas : > Bj?rn Gustavsson wrote: >> >> On Thu, Sep 30, 2010 at 1:22 PM, Tuncer Ayaz >> wrote: >>> >>> dialyzer_plt:included_files/1 calls get_record_from_file/1 and may >>> therefore also return {error, not_valid}. >> >> Thanks! Will include in pu. > > Thanks to Tuncer for noticing and fixing this issue! > > However, I would prefer that the following patch gets pushed to 'dev' > instead of his (ta/fix-dialyzer_plt-spec): > > ?git fetch git://github.com/kostis/otp.git ?dialyzer_plt-type Ack. From tuncer.ayaz@REDACTED Fri Oct 1 16:13:33 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 1 Oct 2010 16:13:33 +0200 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: <4CA5E89D.1040807@cs.ntua.gr> References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> <4CA5E89D.1040807@cs.ntua.gr> Message-ID: 2010/10/1 Kostis Sagonas : > Bj?rn Gustavsson wrote: >> >> On Thu, Sep 30, 2010 at 1:22 PM, Tuncer Ayaz >> wrote: >>> >>> dialyzer_plt:included_files/1 calls get_record_from_file/1 and may >>> therefore also return {error, not_valid}. >> >> Thanks! Will include in pu. > > Thanks to Tuncer for noticing and fixing this issue! > > However, I would prefer that the following patch gets pushed to 'dev' > instead of his (ta/fix-dialyzer_plt-spec): > > ?git fetch git://github.com/kostis/otp.git ?dialyzer_plt-type Ack. From tuncer.ayaz@REDACTED Fri Oct 1 16:13:33 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 1 Oct 2010 16:13:33 +0200 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: <4CA5E89D.1040807@cs.ntua.gr> References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> <4CA5E89D.1040807@cs.ntua.gr> Message-ID: 2010/10/1 Kostis Sagonas : > Bj?rn Gustavsson wrote: >> >> On Thu, Sep 30, 2010 at 1:22 PM, Tuncer Ayaz >> wrote: >>> >>> dialyzer_plt:included_files/1 calls get_record_from_file/1 and may >>> therefore also return {error, not_valid}. >> >> Thanks! Will include in pu. > > Thanks to Tuncer for noticing and fixing this issue! > > However, I would prefer that the following patch gets pushed to 'dev' > instead of his (ta/fix-dialyzer_plt-spec): > > ?git fetch git://github.com/kostis/otp.git ?dialyzer_plt-type Ack. From tuncer.ayaz@REDACTED Fri Oct 1 16:13:33 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 1 Oct 2010 16:13:33 +0200 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: <4CA5E89D.1040807@cs.ntua.gr> References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> <4CA5E89D.1040807@cs.ntua.gr> Message-ID: 2010/10/1 Kostis Sagonas : > Bj?rn Gustavsson wrote: >> >> On Thu, Sep 30, 2010 at 1:22 PM, Tuncer Ayaz >> wrote: >>> >>> dialyzer_plt:included_files/1 calls get_record_from_file/1 and may >>> therefore also return {error, not_valid}. >> >> Thanks! Will include in pu. > > Thanks to Tuncer for noticing and fixing this issue! > > However, I would prefer that the following patch gets pushed to 'dev' > instead of his (ta/fix-dialyzer_plt-spec): > > ?git fetch git://github.com/kostis/otp.git ?dialyzer_plt-type Ack. From michael.santos@REDACTED Sat Oct 2 01:54:51 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:54:51 -0400 Subject: [PATCH 1/8] dialyzer: prevent buffer overflows Message-ID: <20101001235451.GA31031@ecn.lan> Check length of buffers used with environment variables and debug messages. --- erts/etc/common/Makefile.in | 2 +- erts/etc/common/dialyzer.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index 9665566..333390b 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -333,7 +333,7 @@ $(OBJDIR)/erlc.o: erlc.c $(CC) $(CFLAGS) -o $@ -c erlc.c $(BINDIR)/dialyzer@REDACTED@: $(OBJDIR)/dialyzer.o - $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/dialyzer.o -L$(OBJDIR) $(LIBS) + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/dialyzer.o -L$(OBJDIR) $(LIBS) $(ERTS_INTERNAL_LIBS) $(OBJDIR)/dialyzer.o: dialyzer.c $(CC) $(CFLAGS) -o $@ -c dialyzer.c diff --git a/erts/etc/common/dialyzer.c b/erts/etc/common/dialyzer.c index 4b4c112..4453e63 100644 --- a/erts/etc/common/dialyzer.c +++ b/erts/etc/common/dialyzer.c @@ -147,6 +147,9 @@ main(int argc, char** argv) env = get_env("DIALYZER_EMULATOR"); emulator = env ? env : get_default_emulator(argv[0]); + if (strlen(emulator) >= MAXPATHLEN) + error("Value of environment variable DIALYZER_EMULATOR is too large"); + /* * Allocate the argv vector to be used for arguments to Erlang. * Arrange for starting to pushing information in the middle of @@ -228,7 +231,7 @@ main(int argc, char** argv) static void push_words(char* src) { - char sbuf[1024]; + char sbuf[MAXPATHLEN]; char* dst; dst = sbuf; @@ -360,7 +363,7 @@ error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "dialyzer: %s\n", sbuf); exit(1); @@ -389,6 +392,9 @@ get_default_emulator(char* progname) char sbuf[MAXPATHLEN]; char* s; + if (strlen(progname) >= sizeof(sbuf)) + return ERL_NAME; + strcpy(sbuf, progname); for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { if (IS_DIRSEP(*s)) { -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:55:17 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:55:17 -0400 Subject: [PATCH 2/8] erlc: prevent buffer overflows Message-ID: <20101001235517.GA31047@ecn.lan> Check buffer operations and increase the size of the buffer used for holding command line arguments, since the "-D" switch will be expanded into 3 arguments when passed to erl. --- erts/etc/common/Makefile.in | 2 +- erts/etc/common/erlc.c | 12 +++++++++--- erts/test/erlc_SUITE.erl | 16 ++++++++++++++-- 3 files changed, 24 insertions(+), 6 deletions(-) diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index 333390b..fe7d7b4 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -327,7 +327,7 @@ $(OBJDIR)/$(ERLEXEC).o: $(ERLEXECDIR)/$(ERLEXEC).c $(CC) -I$(EMUDIR) $(CFLAGS) -o $@ -c $(ERLEXECDIR)/$(ERLEXEC).c endif $(BINDIR)/erlc@REDACTED@: $(OBJDIR)/erlc.o - $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/erlc.o -L$(OBJDIR) $(LIBS) + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/erlc.o -L$(OBJDIR) $(LIBS) $(ERTS_INTERNAL_LIBS) $(OBJDIR)/erlc.o: erlc.c $(CC) $(CFLAGS) -o $@ -c erlc.c diff --git a/erts/etc/common/erlc.c b/erts/etc/common/erlc.c index 09aca19..b59453a 100644 --- a/erts/etc/common/erlc.c +++ b/erts/etc/common/erlc.c @@ -160,6 +160,9 @@ main(int argc, char** argv) env = get_env("ERLC_EMULATOR"); emulator = env ? env : get_default_emulator(argv[0]); + if (strlen(emulator) >= MAXPATHLEN) + error("Value of environment variable ERLC_EMULATOR is too large"); + /* * Allocate the argv vector to be used for arguments to Erlang. * Arrange for starting to pushing information in the middle of @@ -170,7 +173,7 @@ main(int argc, char** argv) * base of the eargv vector, and move it up later. */ - eargv_size = argc*4+100; + eargv_size = argc*6+100; eargv_base = (char **) emalloc(eargv_size*sizeof(char*)); eargv = eargv_base; eargc = 0; @@ -419,7 +422,7 @@ process_opt(int* pArgc, char*** pArgv, int offset) static void push_words(char* src) { - char sbuf[1024]; + char sbuf[MAXPATHLEN]; char* dst; dst = sbuf; @@ -595,7 +598,7 @@ error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "erlc: %s\n", sbuf); exit(1); @@ -624,6 +627,9 @@ get_default_emulator(char* progname) char sbuf[MAXPATHLEN]; char* s; + if (strlen(progname) >= sizeof(sbuf)) + return ERL_NAME; + strcpy(sbuf, progname); for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { if (IS_DIRSEP(*s)) { diff --git a/erts/test/erlc_SUITE.erl b/erts/test/erlc_SUITE.erl index 437f020..4797f78 100644 --- a/erts/test/erlc_SUITE.erl +++ b/erts/test/erlc_SUITE.erl @@ -21,13 +21,13 @@ %% Tests the erlc command by compiling various types of files. -export([all/1, compile_erl/1, compile_yecc/1, compile_script/1, - compile_mib/1, good_citizen/1, deep_cwd/1]). + compile_mib/1, good_citizen/1, deep_cwd/1, arg_overflow/1]). -include_lib("test_server/include/test_server.hrl"). all(suite) -> [compile_erl, compile_yecc, compile_script, compile_mib, - good_citizen, deep_cwd]. + good_citizen, deep_cwd, arg_overflow]. %% Copy from erlc_SUITE_data/include/erl_test.hrl. @@ -189,6 +189,18 @@ deep_cwd_1(PrivDir) -> ?line true = filelib:is_file("test.beam"), ok. +%% Test that a large number of command line switches does not +%% overflow the argument buffer +arg_overflow(Config) when is_list(Config) -> + ?line {SrcDir, _OutDir, Cmd} = get_cmd(Config), + ?line FileName = filename:join(SrcDir, "erl_test_ok.erl"), + ?line Args = lists:flatten([ ["-D", integer_to_list(N), "=1 "] || + N <- lists:seq(1,10000) ]), + ?line run(Config, Cmd, FileName, Args, + ["Warning: function foo/0 is unused\$", + "_OK_"]), + ok. + erlc() -> case os:find_executable("erlc") of false -> -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:55:38 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:55:38 -0400 Subject: [PATCH 3/8] erlexec: prevent buffer overflows Message-ID: <20101001235538.GA31061@ecn.lan> Truncate buffer operations on environment variables. --- erts/etc/common/erlexec.c | 42 ++++++++++++++++++++++-------------------- 1 files changed, 22 insertions(+), 20 deletions(-) diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index c1fc2ae..6f66b73 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -309,7 +309,7 @@ free_env_val(char *value) } /* - * Add the arcitecture suffix to the program name if needed, + * Add the architecture suffix to the program name if needed, * except on Windows, where we insert it just before ".DLL". */ static char* @@ -560,7 +560,7 @@ int main(int argc, char **argv) usage("+MYm"); } emu = add_extra_suffixes(emu, emu_type); - sprintf(tmpStr, "%s" DIRSEP "%s" BINARY_EXT, bindir, emu); + erts_snprintf(tmpStr, sizeof(tmpStr), "%s" DIRSEP "%s" BINARY_EXT, bindir, emu); emu = strsave(tmpStr); add_Eargs(emu); /* Will be argv[0] -- necessary! */ @@ -571,12 +571,12 @@ int main(int argc, char **argv) s = get_env("PATH"); if (!s) { - sprintf(tmpStr, "%s" PATHSEP "%s" DIRSEP "bin", bindir, rootdir); + erts_snprintf(tmpStr, sizeof(tmpStr), "%s" PATHSEP "%s" DIRSEP "bin", bindir, rootdir); } else if (strstr(s, bindir) == NULL) { - sprintf(tmpStr, "%s" PATHSEP "%s" DIRSEP "bin" PATHSEP "%s", bindir, + erts_snprintf(tmpStr, sizeof(tmpStr), "%s" PATHSEP "%s" DIRSEP "bin" PATHSEP "%s", bindir, rootdir, s); } else { - sprintf(tmpStr, "%s", s); + erts_snprintf(tmpStr, sizeof(tmpStr), "%s", s); } free_env_val(s); set_env("PATH", tmpStr); @@ -714,7 +714,7 @@ int main(int argc, char **argv) error("-man not supported on Windows"); #else argv[i] = "man"; - sprintf(tmpStr, "%s/man", rootdir); + erts_snprintf(tmpStr, sizeof(tmpStr), "%s/man", rootdir); set_env("MANPATH", tmpStr); execvp("man", argv+i); error("Could not execute the 'man' command."); @@ -1145,10 +1145,10 @@ start_epmd(char *epmd) if (!epmd) { epmd = epmd_cmd; #ifdef __WIN32__ - sprintf(epmd_cmd, "%s" DIRSEP "epmd", bindir); + erts_snprintf(epmd_cmd, sizeof(epmd_cmd), "%s" DIRSEP "epmd", bindir); arg1 = "-daemon"; #else - sprintf(epmd_cmd, "%s" DIRSEP "epmd -daemon", bindir); + erts_snprintf(epmd_cmd, sizeof(epmd_cmd), "%s" DIRSEP "epmd -daemon", bindir); #endif } #ifdef __WIN32__ @@ -1224,7 +1224,7 @@ void error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "erlexec: %s\n", sbuf); exit(1); @@ -1304,14 +1304,14 @@ static void get_start_erl_data(char *file) if (env) reldir = strsave(env); else { - sprintf(tmpbuffer, "%s/releases", rootdir); + erts_snprintf(tmpbuffer, sizeof(tmpbuffer), "%s/releases", rootdir); reldir = strsave(tmpbuffer); } free_env_val(env); if (file == NULL) - sprintf(start_erl_data, "%s/start_erl.data", reldir); + erts_snprintf(start_erl_data, sizeof(start_erl_data), "%s/start_erl.data", reldir); else - sprintf(start_erl_data, "%s", file); + erts_snprintf(start_erl_data, sizeof(start_erl_data), "%s", file); fp = _open(start_erl_data, _O_RDONLY ); if( fp == -1 ) error( "open failed on %s",start_erl_data ); @@ -1341,16 +1341,16 @@ static void get_start_erl_data(char *file) } bindir = emalloc(512); - sprintf(bindir,"%s/erts-%s/bin",rootdir,tmpbuffer); + erts_snprintf(bindir,512,"%s/erts-%s/bin",rootdir,tmpbuffer); /* BINDIR=$ROOTDIR/erts-$ERTS_VSN/bin */ tprogname = progname; progname = emalloc(strlen(tprogname) + 20); - sprintf(progname,"%s -start_erl",tprogname); + erts_snprintf(progname,strlen(tprogname) + 20,"%s -start_erl",tprogname); boot_script = emalloc(512); config_script = emalloc(512); - sprintf(boot_script, "%s/%s/start", reldir, otpstring); - sprintf(config_script, "%s/%s/sys", reldir, otpstring); + erts_snprintf(boot_script, 512, "%s/%s/start", reldir, otpstring); + erts_snprintf(config_script, 512, "%s/%s/sys", reldir, otpstring); } @@ -1358,7 +1358,7 @@ static void get_start_erl_data(char *file) static char *replace_filename(char *path, char *new_base) { int plen = strlen(path); - char *res = malloc((plen+strlen(new_base)+1)*sizeof(char)); + char *res = emalloc((plen+strlen(new_base)+1)*sizeof(char)); char *p; strcpy(res,path); @@ -1373,7 +1373,7 @@ static char *path_massage(char *long_path) { char *p; - p = malloc(MAX_PATH+1); + p = emalloc(MAX_PATH+1); strcpy(p, long_path); GetShortPathName(p, p, MAX_PATH); return p; @@ -1509,7 +1509,8 @@ get_parameters(int argc, char** argv) /* Determine bindir from absolute path to executable */ char *p; char buffer[PATH_MAX]; - strcpy(buffer, argv[0]); + strncpy(buffer, argv[0], sizeof(buffer)); + buffer[sizeof(buffer)-1] = '\0'; for (p = buffer+strlen(buffer)-1 ; p >= buffer && *p != '/'; --p) ; @@ -1522,7 +1523,8 @@ get_parameters(int argc, char** argv) /* Determine rootdir from absolute path to bindir */ char *p; char buffer[PATH_MAX]; - strcpy(buffer, bindir); + strncpy(buffer, bindir, sizeof(buffer)); + buffer[sizeof(buffer)-1] = '\0'; for (p = buffer+strlen(buffer)-1; p >= buffer && *p != '/'; --p) ; -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:55:54 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:55:54 -0400 Subject: [PATCH 4/8] escript: prevent buffer overflows Message-ID: <20101001235554.GA31076@ecn.lan> Check buffer operations on input from escripts, the command line and environment variables. --- erts/etc/common/Makefile.in | 2 +- erts/etc/common/escript.c | 52 ++++++++++++-------- lib/stdlib/test/escript_SUITE.erl | 15 +++++- lib/stdlib/test/escript_SUITE_data/arg_overflow | 5 ++ .../test/escript_SUITE_data/linebuf_overflow | 5 ++ 5 files changed, 57 insertions(+), 22 deletions(-) create mode 100755 lib/stdlib/test/escript_SUITE_data/arg_overflow create mode 100755 lib/stdlib/test/escript_SUITE_data/linebuf_overflow diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index fe7d7b4..7f502b2 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -345,7 +345,7 @@ $(OBJDIR)/typer.o: typer.c $(CC) $(CFLAGS) -o $@ -c typer.c $(BINDIR)/escript@REDACTED@: $(OBJDIR)/escript.o - $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/escript.o -L$(OBJDIR) $(LIBS) + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/escript.o -L$(OBJDIR) $(LIBS) $(ERTS_INTERNAL_LIBS) $(OBJDIR)/escript.o: escript.c $(CC) $(CFLAGS) -o $@ -c escript.c diff --git a/erts/etc/common/escript.c b/erts/etc/common/escript.c index 1bc5eb7..6ed79c9 100644 --- a/erts/etc/common/escript.c +++ b/erts/etc/common/escript.c @@ -151,6 +151,9 @@ find_prog(char *origpath) char relpath[PMAX]; char abspath[PMAX]; + if (strlen(origpath) >= sizeof(relpath)) + error("Path too long"); + strcpy(relpath, origpath); if (strstr(relpath, DIRSEPSTR) == NULL) { @@ -180,19 +183,21 @@ find_prog(char *origpath) end = strstr(beg, PATHSEPSTR); if (end != NULL) { sz = end - beg; - strncpy(dir, beg, sz); - dir[sz] = '\0'; } else { sz = strlen(beg); - strcpy(dir, beg); look_for_sep = FALSE; } + if (sz >= sizeof(dir)) { + beg = end + 1; + continue; + } + strncpy(dir, beg, sz); + dir[sz] = '\0'; beg = end + 1; #ifdef __WIN32__ - strcpy(wildcard, dir); - strcat(wildcard, DIRSEPSTR); - strcat(wildcard, relpath); /* basename */ + erts_snprintf(wildcard, sizeof(wildcard), "%s" DIRSEPSTR "%s", + dir, relpath /* basename */); dir_handle = FindFirstFile(wildcard, &find_data); if (dir_handle == INVALID_HANDLE_VALUE) { /* Try next directory in path */ @@ -217,9 +222,8 @@ find_prog(char *origpath) if (strcmp(origpath, dirp->d_name) == 0) { /* Wow we found the executable. */ - strcpy(relpath, dir); - strcat(relpath, DIRSEPSTR); - strcat(relpath, dirp->d_name); + erts_snprintf(relpath, sizeof(relpath), "%s" DIRSEPSTR "%s", + dir, dirp->d_name); closedir(dp); look_for_sep = FALSE; break; @@ -291,7 +295,7 @@ append_shebang_args(char* scriptname) /* Find end of arg */ end = beg; - while (end && end[0] != ' ') { + while (end && end < (linebuf+LINEBUFSZ-1) && end[0] != ' ') { if (end[0] == '\n') { newline = TRUE; end[0]= '\0'; @@ -335,13 +339,16 @@ main(int argc, char** argv) emulator = get_default_emulator(argv[0]); } + if (strlen(emulator) >= PMAX) + error("Value of environment variable ESCRIPT_EMULATOR is too large"); + /* * Allocate the argv vector to be used for arguments to Erlang. * Arrange for starting to pushing information in the middle of * the array, to allow easy addition of commands in the beginning. */ - eargv_size = argc*4+1000; + eargv_size = argc*4+1000+LINEBUFSZ/2; eargv_base = (char **) emalloc(eargv_size*sizeof(char*)); eargv = eargv_base; eargc = 0; @@ -387,7 +394,8 @@ main(int argc, char** argv) if (argc <= 1) { error("Missing filename\n"); } - strcpy(scriptname, argv[1]); + strncpy(scriptname, argv[1], sizeof(scriptname)); + scriptname[sizeof(scriptname)-1] = '\0'; argc--; argv++; } else { @@ -395,16 +403,17 @@ main(int argc, char** argv) int len; #endif absname = find_prog(argv[0]); - strcpy(scriptname, absname); - efree(absname); #ifdef __WIN32__ - len = strlen(scriptname); - if (len >= 4 && _stricmp(scriptname+len-4, ".exe") == 0) { - scriptname[len-4] = '\0'; + len = strlen(absname); + if (len >= 4 && _stricmp(absname+len-4, ".exe") == 0) { + absname[len-4] = '\0'; } #endif - strcat(scriptname, ".escript"); + erts_snprintf(scriptname, sizeof(scriptname), "%s.escript", + absname); + efree(absname); + } /* @@ -455,7 +464,7 @@ main(int argc, char** argv) static void push_words(char* src) { - char sbuf[1024]; + char sbuf[PMAX]; char* dst; dst = sbuf; @@ -584,7 +593,7 @@ error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "escript: %s\n", sbuf); exit(1); @@ -619,6 +628,9 @@ get_default_emulator(char* progname) char sbuf[MAXPATHLEN]; char* s; + if (strlen(progname) >= sizeof(sbuf)) + return ERL_NAME; + strcpy(sbuf, progname); for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { if (IS_DIRSEP(*s)) { diff --git a/lib/stdlib/test/escript_SUITE.erl b/lib/stdlib/test/escript_SUITE.erl index 77fd190..162ca60 100644 --- a/lib/stdlib/test/escript_SUITE.erl +++ b/lib/stdlib/test/escript_SUITE.erl @@ -31,6 +31,7 @@ epp/1, create_and_extract/1, foldl/1, + overflow/1, verify_sections/3 ]). @@ -48,7 +49,8 @@ all(suite) -> archive_script, epp, create_and_extract, - foldl + foldl, + overflow ]. init_per_testcase(_Case, Config) -> @@ -736,6 +738,17 @@ emulate_escript_foldl(Fun, Acc, File) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +overflow(Config) when is_list(Config) -> + Data = ?config(data_dir, Config), + Dir = filename:absname(Data), %Get rid of trailing slash. + ?line run(Dir, "arg_overflow", + [<<"ExitCode:0">>]), + ?line run(Dir, "linebuf_overflow", + [<<"ExitCode:0">>]), + ok. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + run(Dir, Cmd0, Expected0) -> Expected = iolist_to_binary(expected_output(Expected0, Dir)), Cmd = case os:type() of diff --git a/lib/stdlib/test/escript_SUITE_data/arg_overflow b/lib/stdlib/test/escript_SUITE_data/arg_overflow new file mode 100755 index 0000000..dd5accc --- /dev/null +++ b/lib/stdlib/test/escript_SUITE_data/arg_overflow @@ -0,0 +1,5 @@ +#! /usr/bin/env escript +%% -*- erlang -*- +%%!x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x +main(_) -> + halt(0). diff --git a/lib/stdlib/test/escript_SUITE_data/linebuf_overflow b/lib/stdlib/test/escript_SUITE_data/linebuf_overflow new file mode 100755 index 0000000..33133c1 --- /dev/null +++ b/lib/stdlib/test/escript_SUITE_data/linebuf_overflow @@ -0,0 +1,5 @@ +#! /usr/bin/env escript +%% -*- erlang -*- +%%!xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +main(_) -> + halt(0). -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:56:10 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:56:10 -0400 Subject: [PATCH 5/8] heart: prevent buffer overflow Message-ID: <20101001235610.GA31090@ecn.lan> --- erts/etc/common/heart.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a/erts/etc/common/heart.c b/erts/etc/common/heart.c index 4f73894..3e19e5f 100644 --- a/erts/etc/common/heart.c +++ b/erts/etc/common/heart.c @@ -375,7 +375,8 @@ main(int argc, char **argv) _setmode(erlin_fd,_O_BINARY); _setmode(erlout_fd,_O_BINARY); #endif - strcpy(program_name, argv[0]); + strncpy(program_name, argv[0], sizeof(program_name)); + program_name[sizeof(program_name)-1] = '\0'; notify_ack(erlout_fd); cmd[0] = '\0'; do_terminate(message_loop(erlin_fd,erlout_fd)); @@ -728,7 +729,11 @@ heart_cmd_reply(int fd, char *s) struct msg m; int len = strlen(s) + 1; /* Include \0 */ - /* FIXME if s >= MSG_BODY_SIZE error */ + /* if s >= MSG_BODY_SIZE, return a write + * failure immediately. + */ + if (len > sizeof(m.fill)) + return -1; m.op = HEART_CMD; m.len = htons(len + 2); /* Include Op */ -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:56:34 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:56:34 -0400 Subject: [PATCH 6/8] run_test: prevent buffer overflow Message-ID: <20101001235634.GA31104@ecn.lan> Truncate buffers used to hold command line arguments. --- erts/etc/common/Makefile.in | 2 +- erts/etc/common/run_test.c | 19 +++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index 7f502b2..a965ac2 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -351,7 +351,7 @@ $(OBJDIR)/escript.o: escript.c $(CC) $(CFLAGS) -o $@ -c escript.c $(BINDIR)/run_test@REDACTED@: $(OBJDIR)/run_test.o - $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/run_test.o -L$(OBJDIR) $(LIBS) + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/run_test.o -L$(OBJDIR) $(LIBS) $(ERTS_INTERNAL_LIBS) $(OBJDIR)/run_test.o: run_test.c $(CC) $(CFLAGS) -o $@ -c run_test.c diff --git a/erts/etc/common/run_test.c b/erts/etc/common/run_test.c index 016d9c6..042b857 100644 --- a/erts/etc/common/run_test.c +++ b/erts/etc/common/run_test.c @@ -164,11 +164,13 @@ main(int argc, char** argv) erl_args = cnt; } else if (strcmp(argv[1], "-sname") == 0) { - strcpy(nodename, argv[2]); + strncpy(nodename, argv[2], sizeof(nodename)); + nodename[sizeof(nodename)-1] = '\0'; cnt++, argv++; } else if (strcmp(argv[1], "-name") == 0) { - strcpy(nodename, argv[2]); + strncpy(nodename, argv[2], sizeof(nodename)); + nodename[sizeof(nodename)-1] = '\0'; dist_mode = FULL_NAME; cnt++, argv++; } @@ -178,7 +180,8 @@ main(int argc, char** argv) ct_mode = VTS_MODE; } else if (strcmp(argv[1], "-browser") == 0) { - strcpy(browser, argv[2]); + strncpy(browser, argv[2], sizeof(browser)); + browser[sizeof(browser)-1] = '\0'; cnt++, argv++; } else if (strcmp(argv[1], "-shell") == 0) { @@ -189,7 +192,8 @@ main(int argc, char** argv) ct_mode = MASTER_MODE; } else if (strcmp(argv[1], "-ctname") == 0) { - strcpy(nodename, argv[2]); + strncpy(nodename, argv[2], sizeof(nodename)); + nodename[sizeof(nodename)-1] = '\0'; ct_mode = ERL_SHELL_MODE; cnt++, argv++; } @@ -273,7 +277,7 @@ main(int argc, char** argv) static void push_words(char* src) { - char sbuf[1024]; + char sbuf[MAXPATHLEN]; char* dst; dst = sbuf; @@ -405,7 +409,7 @@ error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "run_test: %s\n", sbuf); exit(1); @@ -434,6 +438,9 @@ get_default_emulator(char* progname) char sbuf[MAXPATHLEN]; char* s; + if (strlen(progname) >= sizeof(sbuf)) + return ERL_NAME; + strcpy(sbuf, progname); for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { if (IS_DIRSEP(*s)) { -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:56:52 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:56:52 -0400 Subject: [PATCH 7/8] typer: prevent buffer overflows Message-ID: <20101001235652.GA31119@ecn.lan> --- erts/etc/common/Makefile.in | 2 +- erts/etc/common/typer.c | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index a965ac2..e866511 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -339,7 +339,7 @@ $(OBJDIR)/dialyzer.o: dialyzer.c $(CC) $(CFLAGS) -o $@ -c dialyzer.c $(BINDIR)/typer@REDACTED@: $(OBJDIR)/typer.o - $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/typer.o -L$(OBJDIR) $(LIBS) + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/typer.o -L$(OBJDIR) $(LIBS) $(ERTS_INTERNAL_LIBS) $(OBJDIR)/typer.o: typer.c $(CC) $(CFLAGS) -o $@ -c typer.c diff --git a/erts/etc/common/typer.c b/erts/etc/common/typer.c index c2567cb..de48daf 100644 --- a/erts/etc/common/typer.c +++ b/erts/etc/common/typer.c @@ -175,7 +175,7 @@ main(int argc, char** argv) static void push_words(char* src) { - char sbuf[1024]; + char sbuf[MAXPATHLEN]; char* dst; dst = sbuf; @@ -307,7 +307,7 @@ error(char* format, ...) va_list ap; va_start(ap, format); - vsprintf(sbuf, format, ap); + erts_vsnprintf(sbuf, sizeof(sbuf), format, ap); va_end(ap); fprintf(stderr, "typer: %s\n", sbuf); exit(1); @@ -336,6 +336,9 @@ get_default_emulator(char* progname) char sbuf[MAXPATHLEN]; char* s; + if (strlen(progname) >= sizeof(sbuf)) + return ERL_NAME; + strcpy(sbuf, progname); for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { if (IS_DIRSEP(*s)) { -- 1.7.0.4 From michael.santos@REDACTED Sat Oct 2 01:57:08 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 19:57:08 -0400 Subject: [PATCH 8/8] erlc: remove unused variable Message-ID: <20101001235708.GA31133@ecn.lan> --- erts/etc/common/erlc.c | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/erts/etc/common/erlc.c b/erts/etc/common/erlc.c index b59453a..cd13743 100644 --- a/erts/etc/common/erlc.c +++ b/erts/etc/common/erlc.c @@ -148,10 +148,6 @@ int main(int argc, char** argv) { char cwd[MAXPATHLEN]; /* Current working directory. */ - char** rpc_eargv; /* Pointer to the beginning of arguments - * if calling a running Erlang system - * via erl_rpc(). - */ int eargv_size; int eargc_base; /* How many arguments in the base of eargv. */ char* emulator; @@ -192,7 +188,6 @@ main(int argc, char** argv) PUSH2("-mode", "minimal"); PUSH2("-boot", "start_clean"); PUSH3("-s", "erl_compile", "compile_cmdline"); - rpc_eargv = eargv+eargc; /* * Push standard arguments to Erlang. -- 1.7.0.4 From kostis@REDACTED Sat Oct 2 16:57:06 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 02 Oct 2010 17:57:06 +0300 Subject: Type of ets:update_counter/3 Message-ID: <4CA74842.5030807@cs.ntua.gr> Fix type of ets:update_counter/3 BIF: git fetch git://github.com/kostis/otp.git ets-update-counter Kostis From tuncer.ayaz@REDACTED Sat Oct 2 19:38:22 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 2 Oct 2010 19:38:22 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: <19612.39061.424713.26823@pilspetsen.it.uu.se> References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> Message-ID: On Fri, Sep 24, 2010 at 2:24 PM, Mikael Pettersson wrote: > My planned fix is to make remote calls link to the target's Export* instead, > just like BEAM does, which should solve the problems. ?This will however > require HiPE to use different kinds of trap-to-beam stubs for remote and local > calls, since local calls must not and often cannot go via Export entries. > > A simpler workaround for the error_handler issue (which I couldn't reproduce) > is to just never compile error_handler to native code. ?It's not like there's > a lot to gain by doing that. ?Please try the patch below. > > /Mikael > > --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ ? 2010-09-13 > 19:00:22.000000000 +0200 > +++ otp_src_R14B/lib/kernel/src/error_handler.erl ? ? ? 2010-09-24 > 13:44:09.000000000 +0200 > @@ -17,6 +17,7 @@ > ?%% %CopyrightEnd% > ?%% > ?-module(error_handler). > +-compile(no_native). > > ?%% A simple error handler. Any objections to applying this fix to dev? Otherwise let's include it as a trivial workaround that makes enable-native-libs useable. From pguyot@REDACTED Sun Oct 3 09:30:35 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 3 Oct 2010 09:30:35 +0200 Subject: Fix crash with erlang:system_info({cpu_topology,junk}) Message-ID: Hello, There is a bug causing a crash if erlang:system_info is called with {cpu_topology, Atom} where Atom is an unrecognized Atom (e.g. misspelled). The function should fail with badarg, but the emulator would segfault instead. A fix with a non-regression test is available here: git fetch git://github.com/pguyot/otp.git fix-system_info-cpu_topology-segfault Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From pguyot@REDACTED Sun Oct 3 18:20:56 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 3 Oct 2010 18:20:56 +0200 Subject: Support for CPU affinity & topology detection on FreeBSD 8 Message-ID: <76D94529-389A-46F4-9221-82D8C3CF76C7@kallisys.net> Hello, Support for CPU affinity and topology detection on FreeBSD is available on this branch: git fetch git://github.com/pguyot/otp.git freebsd-cpu-affinity-and-topology CPU affinity code is based on configure-time detection of FreeBSD's native CPU affinity API which appeared on FreeBSD 7.1. CPU topology detection code is based on the parsing of what the ULE scheduler detected at boot time and saved in a sysctl variable. This variable appeared on FreeBSD 8.0. The format of this specification is documented in SMP(4) and I believe this is the only way to know about the topology (except talking to the CPU directly). Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From pguyot@REDACTED Sun Oct 3 18:37:45 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 3 Oct 2010 18:37:45 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> Message-ID: >> --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ 2010-09-13 >> 19:00:22.000000000 +0200 >> +++ otp_src_R14B/lib/kernel/src/error_handler.erl 2010-09-24 >> 13:44:09.000000000 +0200 >> @@ -17,6 +17,7 @@ >> %% %CopyrightEnd% >> %% >> -module(error_handler). >> +-compile(no_native). >> >> %% A simple error handler. > > Any objections to applying this fix to dev? Otherwise let's include it > as a trivial workaround that makes enable-native-libs useable. This is no trivial workaround and it works by sheer luck. What this bug reveals is that there is a major design flaw with on_load and HiPE, as illustrated by the fact that the emulator goes into an infinite loop if error_handler is natively compiled: every time there is a remote call to a function in a module that has an on_load attribute, the emulator goes through error_handler:undefined_function! I think it doesn't infinite loop when error_handler is not native because it goes through the apply BIF which works around the linkage issue. A quick benchmark shows that (on dev, without --enable-native-libs) calling a remote function in an on_load attribute currently takes 100+ times longer from HiPE than from BEAM. The way things work for BEAM is that final_touch disables export entries if there is an on_load handler, and finish_after_on_load_2 BIF, which is called from init/code_server, fixes them by using the saved address. The same should be done for HiPE, but of course it's more complicated because dynamic linking works differently in HiPE. I can only see three solutions: - fix the pointers in each callers (which doesn't sound doable); - change the way dynamic linking works in HiPE as Mikael suggests; - or add a small stub, as implemented in my patch -- this is still three to five times slower than BEAM. Simply ignoring the problem will not make it go away. Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From tuncer.ayaz@REDACTED Sun Oct 3 19:36:52 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 3 Oct 2010 19:36:52 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> Message-ID: On Sun, Oct 3, 2010 at 6:37 PM, Paul Guyot wrote: >>> --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ ? 2010-09-13 >>> 19:00:22.000000000 +0200 >>> +++ otp_src_R14B/lib/kernel/src/error_handler.erl ? ? ? 2010-09-24 >>> 13:44:09.000000000 +0200 >>> @@ -17,6 +17,7 @@ >>> ?%% %CopyrightEnd% >>> ?%% >>> ?-module(error_handler). >>> +-compile(no_native). >>> >>> ?%% A simple error handler. >> >> Any objections to applying this fix to dev? Otherwise let's include it >> as a trivial workaround that makes enable-native-libs useable. > [...] > I can only see three solutions: > - fix the pointers in each callers (which doesn't sound doable); > - change the way dynamic linking works in HiPE as Mikael suggests; > - or add a small stub, as implemented in my patch -- this is still three to > five times slower than BEAM. > > Simply ignoring the problem will not make it go away. I see your point that applying that one-liner may hide the issue and possibly let it sink into oblivion. I do agree with that. I wasn't trying to suggest to ignore the issue but see if there's any downside/objection against this patch. Your point is valid enough to not include the compiler directive. enable-native-libs has reduced dialyzer runtime by quite a margin and is therefore of significant benefit for my use cases. For anyone testing: On the systems I build I had to manually $ rm lib/hipe/ebin/{erl,erl_bif}_types.beam $ make hipe to ensure that these two are also native-compiled. Having these +native made a big difference in my dialyzer runs. From mikpe@REDACTED Mon Oct 4 14:28:44 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 4 Oct 2010 14:28:44 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> Message-ID: <19625.51324.386905.575172@pilspetsen.it.uu.se> Paul Guyot writes: > >> --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ 2010-09-13 > >> 19:00:22.000000000 +0200 > >> +++ otp_src_R14B/lib/kernel/src/error_handler.erl 2010-09-24 > >> 13:44:09.000000000 +0200 > >> @@ -17,6 +17,7 @@ > >> %% %CopyrightEnd% > >> %% > >> -module(error_handler). > >> +-compile(no_native). > >> > >> %% A simple error handler. > > > > Any objections to applying this fix to dev? Otherwise let's include it > > as a trivial workaround that makes enable-native-libs useable. > > > This is no trivial workaround and it works by sheer luck. The loops always involve a native-mode error_handler, so by eliminating that the loops are eliminated. That's not "by sheer luck". > What this bug reveals is that there is a major design flaw with on_load and HiPE, as illustrated by the fact that the emulator goes into an infinite loop if error_handler is natively compiled: every time there is a remote call to a function in a module that has an on_load attribute, the emulator goes through error_handler:undefined_function! I think it doesn't infinite loop when error_handler is not native because it goes through the apply BIF which works around the linkage issue. The short explanation is that BEAM-mode code doesn't loop in this case because BEAM has an additional level of indirection between caller and callee. The BEAM code loader updates the middle level (the code address in Export entries) which allows callers to immediately see the updates. > A quick benchmark shows that (on dev, without --enable-native-libs) calling a remote function in an on_load attribute currently takes 100+ times longer from HiPE than from BEAM. Modules with on_load attributes are supposed to be prevented from being compiled to native code, so the "from HiPE" case shouldn't exist. If you meant to write that a call from native code in module N1 to BEAM code in module B2, where B2 also has an on_load, is slow, then yes I know that and it's because of the extra roundtrip through error_handler. In the long term my change to how native code links to BEAM code will fix that. In the short term you can work around it by splitting modules into on_load and non-on_load bits. > The way things work for BEAM is that final_touch disables export entries if there is an on_load handler, and finish_after_on_load_2 BIF, which is called from init/code_server, fixes them by using the saved address. The same should be done for HiPE, but of course it's more complicated because dynamic linking works differently in HiPE. > > I can only see three solutions: > - fix the pointers in each callers (which doesn't sound doable); Not doable in the short term. > - change the way dynamic linking works in HiPE as Mikael suggests; > - or add a small stub, as implemented in my patch -- this is still three to five times slower than BEAM. > > Simply ignoring the problem will not make it go away. It's not being ignored. From bgustavsson@REDACTED Mon Oct 4 14:52:13 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 14:52:13 +0200 Subject: [erlang-patches] [PATCH] Dialyzer: Fix dialyzer_plt:included_files/1 spec In-Reply-To: <4CA5E89D.1040807@cs.ntua.gr> References: <1285845742-847-1-git-send-email-tuncer.ayaz@gmail.com> <4CA5E89D.1040807@cs.ntua.gr> Message-ID: 2010/10/1 Kostis Sagonas : > > However, I would prefer that the following patch gets pushed to 'dev' > instead of his (ta/fix-dialyzer_plt-spec): > > git fetch git://github.com/kostis/otp.git dialyzer_plt-type > OK. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 4 14:53:20 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 14:53:20 +0200 Subject: [erlang-patches] [PATCH 1/8] dialyzer: prevent buffer overflows In-Reply-To: <20101001235451.GA31031@ecn.lan> References: <20101001235451.GA31031@ecn.lan> Message-ID: On Sat, Oct 2, 2010 at 1:54 AM, Michael Santos wrote: > Check length of buffers used with environment variables and debug > messages. > --- Thanks! Will include in pu, along with the other 7 seven patches. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Mon Oct 4 14:53:47 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Oct 2010 14:53:47 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: <19625.51324.386905.575172@pilspetsen.it.uu.se> References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> <19625.51324.386905.575172@pilspetsen.it.uu.se> Message-ID: On Mon, Oct 4, 2010 at 2:28 PM, Mikael Pettersson wrote: > Paul Guyot writes: > ?> >> --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ ? 2010-09-13 > ?> >> 19:00:22.000000000 +0200 > ?> >> +++ otp_src_R14B/lib/kernel/src/error_handler.erl ? ? ? 2010-09-24 > ?> >> 13:44:09.000000000 +0200 > ?> >> @@ -17,6 +17,7 @@ > ?> >> ?%% %CopyrightEnd% > ?> >> ?%% > ?> >> ?-module(error_handler). > ?> >> +-compile(no_native). > ?> >> > ?> >> ?%% A simple error handler. > ?> > > ?> > Any objections to applying this fix to dev? Otherwise let's > > > include it as a trivial workaround that makes enable-native-libs > > > useable. > ?> > ?> > ?> This is no trivial workaround and it works by sheer luck. > > The loops always involve a native-mode error_handler, so by > eliminating that the loops are eliminated. ? That's not "by sheer > luck". > >> What this bug reveals is that there is a major design flaw with >> on_load and HiPE, as illustrated by the fact that the emulator >> goes into an infinite loop if error_handler is natively compiled: >> every time there is a remote call to a function in a module that >> has an on_load attribute, the emulator goes through >> error_handler:undefined_function! I think it doesn't infinite loop >> when error_handler is not native because it goes through the apply >> BIF which works around the linkage issue. > > The short explanation is that BEAM-mode code doesn't loop in this > case because BEAM has an additional level of indirection between > caller and callee. ? The BEAM code loader updates the middle level > (the code address in Export entries) which allows callers to > immediately see the updates. > > ?> A quick benchmark shows that (on dev, without --enable-native-libs) > > calling a remote function in an on_load attribute currently takes > > 100+ times longer from HiPE than from BEAM. > > Modules with on_load attributes are supposed to be prevented from > being compiled to native code, so the "from HiPE" case shouldn't > exist. > > If you meant to write that a call from native code in module N1 to > BEAM code in module B2, where B2 also has an on_load, is slow, then > yes I know that and it's because of the extra roundtrip through > error_handler. In the long term my change to how native code links > to BEAM code will fix that. In the short term you can work around it > by splitting modules into on_load and non-on_load bits. Speaking of short term fixes, what would you prefer to see in the dev branch? S) split error_handler into on_load/non-on_load bits D) assuming you see no drawback add no_native directive to error_handler N) none of the above two options > ?> The way things work for BEAM is that final_touch disables > > export entries if there is an on_load handler, and > > finish_after_on_load_2 BIF, which is called from > > init/code_server, fixes them by using the saved address. > > The same should be done for HiPE, but of course it's more > > complicated because dynamic linking works differently in HiPE. > ?> > ?> I can only see three solutions: > ?> - fix the pointers in each callers (which doesn't sound doable); > > Not doable in the short term. > > ?> - change the way dynamic linking works in HiPE as Mikael suggests; > ?> - or add a small stub, as implemented in my patch -- this is > > still three to five times slower than BEAM. > ?> > ?> Simply ignoring the problem will not make it go away. > > It's not being ignored. From bgustavsson@REDACTED Mon Oct 4 14:56:06 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 14:56:06 +0200 Subject: [erlang-patches] Type of ets:update_counter/3 In-Reply-To: <4CA74842.5030807@cs.ntua.gr> References: <4CA74842.5030807@cs.ntua.gr> Message-ID: 2010/10/2 Kostis Sagonas : > Fix type of ets:update_counter/3 BIF: > > git fetch git://github.com/kostis/otp.git ets-update-counter > Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 4 14:59:48 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 14:59:48 +0200 Subject: [erlang-patches] Fix crash with erlang:system_info({cpu_topology,junk}) In-Reply-To: References: Message-ID: On Sun, Oct 3, 2010 at 9:30 AM, Paul Guyot wrote: > Hello, > > There is a bug causing a crash if erlang:system_info is called with {cpu_topology, Atom} where Atom is an unrecognized Atom (e.g. misspelled). The function should fail with badarg, but the emulator would segfault instead. > > A fix with a non-regression test is available here: > git fetch git://github.com/pguyot/otp.git fix-system_info-cpu_topology-segfault Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 4 15:02:32 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 15:02:32 +0200 Subject: [erlang-patches] Support for CPU affinity & topology detection on FreeBSD 8 In-Reply-To: <76D94529-389A-46F4-9221-82D8C3CF76C7@kallisys.net> References: <76D94529-389A-46F4-9221-82D8C3CF76C7@kallisys.net> Message-ID: On Sun, Oct 3, 2010 at 6:20 PM, Paul Guyot wrote: > Hello, > > Support for CPU affinity and topology detection on FreeBSD is available on this branch: > git fetch git://github.com/pguyot/otp.git freebsd-cpu-affinity-and-topology Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 4 15:22:14 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 15:22:14 +0200 Subject: [erlang-patches] erl_call can now use an IP address for remote node name In-Reply-To: <4C9B3E30.3070505@quicheaters.org> References: <4C9B3E30.3070505@quicheaters.org> Message-ID: On Thu, Sep 23, 2010 at 1:46 PM, Julien Barbot wrote: > Hi, > > erl_call was not able to connect to a remote node when the specified node > with -name contained an IP address. > > The previous inet_addr/htonl calls were not working with ip addresses. > > Here is the small patch: > > git fetch git://github.com/klyr/otp.git erl_call_get_hostent > The build fails on Windows like this: cc.sh -MD -O2 -Wall -I. -I../include -Iconnect -Iencode -Idecode -Imisc -Iepmd -Iregistry -Iwin32 -Ilegacy -DWIN32_THREADS -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -o /ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe prog/erl_call.c prog/erl_start.c \ -L/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/obj/win32 -lei_md -lsocket erl_call.o : error LNK2019: unresolved external symbol _inet_pton referenced in function _get_hostent C:/cygwin/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe : fatal error LNK1120: 1 unresolved externals I will drop the branch from pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From mikpe@REDACTED Mon Oct 4 16:15:17 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 4 Oct 2010 16:15:17 +0200 Subject: [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> <6AB0F18D-BF25-40BF-A658-BBBDE024F246@kallisys.net> <19612.39061.424713.26823@pilspetsen.it.uu.se> <19625.51324.386905.575172@pilspetsen.it.uu.se> Message-ID: <19625.57717.953835.446538@pilspetsen.it.uu.se> Tuncer Ayaz writes: > On Mon, Oct 4, 2010 at 2:28 PM, Mikael Pettersson wrote: > > Paul Guyot writes: > > ?> >> --- otp_src_R14B/lib/kernel/src/error_handler.erl.~1~ ? 2010-09-13 > > ?> >> 19:00:22.000000000 +0200 > > ?> >> +++ otp_src_R14B/lib/kernel/src/error_handler.erl ? ? ? 2010-09-24 > > ?> >> 13:44:09.000000000 +0200 > > ?> >> @@ -17,6 +17,7 @@ > > ?> >> ?%% %CopyrightEnd% > > ?> >> ?%% > > ?> >> ?-module(error_handler). > > ?> >> +-compile(no_native). > > ?> >> > > ?> >> ?%% A simple error handler. > > ?> > > > ?> > Any objections to applying this fix to dev? Otherwise let's > > > > include it as a trivial workaround that makes enable-native-libs > > > > useable. > > ?> > > ?> > > ?> This is no trivial workaround and it works by sheer luck. > > > > The loops always involve a native-mode error_handler, so by > > eliminating that the loops are eliminated. ? That's not "by sheer > > luck". > > > >> What this bug reveals is that there is a major design flaw with > >> on_load and HiPE, as illustrated by the fact that the emulator > >> goes into an infinite loop if error_handler is natively compiled: > >> every time there is a remote call to a function in a module that > >> has an on_load attribute, the emulator goes through > >> error_handler:undefined_function! I think it doesn't infinite loop > >> when error_handler is not native because it goes through the apply > >> BIF which works around the linkage issue. > > > > The short explanation is that BEAM-mode code doesn't loop in this > > case because BEAM has an additional level of indirection between > > caller and callee. ? The BEAM code loader updates the middle level > > (the code address in Export entries) which allows callers to > > immediately see the updates. > > > > ?> A quick benchmark shows that (on dev, without --enable-native-libs) > > > calling a remote function in an on_load attribute currently takes > > > 100+ times longer from HiPE than from BEAM. > > > > Modules with on_load attributes are supposed to be prevented from > > being compiled to native code, so the "from HiPE" case shouldn't > > exist. > > > > If you meant to write that a call from native code in module N1 to > > BEAM code in module B2, where B2 also has an on_load, is slow, then > > yes I know that and it's because of the extra roundtrip through > > error_handler. In the long term my change to how native code links > > to BEAM code will fix that. In the short term you can work around it > > by splitting modules into on_load and non-on_load bits. > > Speaking of short term fixes, what would you prefer to see in the dev > branch? > S) split error_handler into on_load/non-on_load bits The splitting I mentioned referred to the application module (B2 above), not error_handler. > D) assuming you see no drawback add no_native directive to > error_handler While it doesn't fix Paul's performance issue it does fix the correctness issue, which is why I suggested it in the first place some time ago. > N) none of the above two options error_handler gets no_native to fix the correctness issue. Application developers who hit the performance issue apply the splitting workaround. /Mikael From bgustavsson@REDACTED Mon Oct 4 16:39:11 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 4 Oct 2010 16:39:11 +0200 Subject: What's cooking in erlang/otp (2010-10-04) Message-ID: The next release will be R14B01, expected to be released in December. The next time I'll send out this email (probably next week), I intend to drop some branches that have been stalled for a long time. ------------------------------------------------------------ [Graduated] * ks/dialyzer-fixes (dialyzer, hipe) (2010-09-24) 1 commit + Fix two errors in dialyzer (3fbe522) * ks/hipe-cleanups (hipe) (2010-09-24) 1 commit + Cleanup and small fixes in hipe files (0ec4d12) * ks/stdlib-spec-fixes (stdlib) (2010-09-24) 1 commit + Correct erroneous specs; strengthen some others (7c64e54) * mh/dollar-font-lock (tools) (2010-09-15) 1 commit + Emacs erlang-mode: fix syntax highlighting of $ in two cases (b81b610) * mp/fix-hipe-write (erts) (2010-09-27) 1 commit + fix 64-bit writes to 32-bit struct field in HiPE runtime (3f367da) * pl/observer-installation (observer) (2010-09-16) 1 commit + Fix installation of example file in lib/observer (f825c9f) * rt/solaris-pty-handling-fixes (erts) (2010-08-06) 2 commits + Add corrected support for Solaris PTYs to run_erl (94399ba) + Fix a typo that leads to syntax errors with DEBUG defined in run_erl (9b6cd97) * ta/eliminate-warning (otp) (2010-09-23) 1 commit + emd2exml: Do not auto-import error/2 (137959a) * ta/eliminate-warnings (edoc, hipe, mnesia) (2010-09-12) 3 commits + mnesia: Do not auto-import error/2 (4276689) + hipe: Do not auto-import min/2 and max/2 (41693be) + edoc: Do not auto-import error/1 (fa669f5) * vd/otpmbox_hash (jinterface) (2010-09-20) 1 commit + add OtpMbox.hash() method (a117010) ------------------------------------------------------------ [New topics] * fm/httpc-upload-body-streaming (inets) (2010-09-26) 1 commit - httpc: allow streaming of PUT and POST request bodies (0ae050e) * jr/windows-file-append (erts) (2010-09-30) 1 commit - Fix appending to large files (>4GB) on Windows (6aae439) * ks/ets-update-counter (hipe) (2010-10-02) 1 commit - Fix type of ets:update_counter/3 to correspond with the documentation (56916f3) * ks/export-types (stdlib) (2010-09-24) 1 commit - Export opaque types so as to be used by other modules (4d60cb7) * ks/fix-dialyzer_plt-spec (dialyzer) (2010-10-01) 1 commit - Dialyzer: Fix erroneous type and use the same definition in two places (d158c01) * mh/dialyzer-full-path-option (dialyzer) (2010-09-15) 1 commit - Dialyzer: add --fullpath option for printing full paths in warnings (3703ea6) * ms/security-fixes (erts, stdlib) (2010-10-01) 8 commits - erlc: remove unused variable (0983ce5) - typer: prevent buffer overflows (b3904e7) - run_test: prevent buffer overflow (329d7a2) - heart: prevent buffer overflow (731755b) - escript: prevent buffer overflows (e5a231d) - erlexec: prevent buffer overflows (570dc4e) - erlc: prevent buffer overflows (be8759e) - dialyzer: prevent buffer overflows (3d430d0) * pg/fix-hibernate-with-hipe (erts, otp) (2010-09-27) 1 commit - Fix several bugs related to hibernate/3 and HiPE (646fd86) * pg/fix-system_info-cpu_topology-segfault (erts) (2010-10-02) 1 commit - Fix crash with erlang:system_info({cpu_topology,junk}) (f8201e5) * pg/freebsd-cpu-affinity-and-topology (erts) (2010-10-03) 1 commit - Add support for CPU affinity & topology detection on FreeBSD 8 (62fc607) * pg/hipe-on_load (erts) (2010-09-22) 1 commit - Fix hang when calling functions in a module with an on_load attribute from a native module (114f510) * sg/fix-diskless-booted-relup (sasl) (2010-09-19) 1 commit - Remove traces of release_handler reading from filesystem when it has Masters list (54cb5df) * ts/cover-with-export_all (tools) (2010-09-23) 1 commit - add user specified compiler options on form reloading (eb02beb) ------------------------------------------------------------ [Stalled] * dc/update-INSTALL-WIN32.md (otp) (2010-09-19) 1 commit - Doc update to latest dependencies & how to compile with free VS2008 Express (8e8061c) Waiting for an update by the topic author. * er/ei-decode_msg (erl_interface) (2010-03-11) 1 commit - Add function to decode distribution header (864091a) Needs documentation and test cases. Action expected from: Topic author * jb/erl_call-get_hostent (erl_interface) (2010-09-23) 1 commit . Teach erl_call to use an IP address for remote node name (d369555) Does not build on Windows. Action expected from: Topic author or anyone else with an interest to advance the topic * jn/gen_stream (stdlib) (2010-09-05) 5 commits - Use erlang:process_info(Pid, links) to determine if gen_stream managed worker processes are leaked (b2386df) - Move gen_stream internal process test to after termination of gen_stream parent (f1d8a97) - Add calls to code loader in every test to verify code loader still running (ac6cbba) - Track gen_stream processes directly for accurate test results (38a4083) - Patch dev to reproduce jn/gen_stream (7694a74) . Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) This topic branch has a failing test case: shell_SUITE:start_restricted_from_shell; We have now tested with and without this topic branch and when it is present that test case fails. We consider it proved to be guilty. The test case does not fail when run standalone so it seems to be tricky... Also, some of its own test cases fail sporadicly. Action expected from: Topic author. * jp/dependencies_makefile (compiler, erts) (2009-12-14) 1 commit - Add dependencies Makefile generation to erlc(1) and compile(3) (4945add) Approved in principle by the OTP Technical Board; now awaits a closer review of the implementation. Action expected from: Bj?rn Gustavsson (the reviewer) * kr/gs-browser (gs) (2010-02-10) 1 commit - Change lib/gs/src/tool_utils.erl to select multiple browsers other than Netscape (802d159) This topic branch introduces a new call to the deprecated regexp module. Action expected from: Topic author or anyone else with an interest to advance the topic * sf/erts_de_busy_limit (erts) (2010-06-08) 1 commit - Add flag-based setting for 'erts_de_busy_limit' constant (67ed9ab) This topic branch has multiple issues described in this email: http://www.erlang.org/cgi-bin/ezmlm-cgi?3:mss:1303:201008:ldciadjdmjednikemjch Action expected from: Topic author. * uw/epmd-fullnames (kernel) (2010-05-07) 1 commit . Allow the erlang nodename host part to differ from the hostname (e0774be) Must be rebased. Action expected from: Topic author. ------------------------------------------------------------ [Cooking] * bg/temp-hipe-fixes (compiler, hipe, kernel) (2010-03-15) 6 commits - Avoid crash by skipping compilation_SUITE:on_load/1 (8592a02) - Inline less aggressively for native-code compilation (e699ad4) - Crudely fix return type for the lists:key{search,find,member}/3 (2b72202) - seq_trace_SUITE: Don't native-compile (2712344) - Disable native code if on_load is used (e8d0ea4) - andor_SUITE: Don't native-compile (e25ef8c) This branch contains temporary workarounds to avoid failing test cases and is never intended to graduate. * gc/gen-format-status-improvements (stdlib) (2010-06-12) 1 commit - Fix format_status bug for unregistered gen_event processes (95ed86f) * jf/run_erl-disable-flow-control (erts) (2010-03-05) 1 commit - Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control (d0775cd) * km/pool_connect-to-running-nodes (stdlib) (2010-09-02) 1 commit - Change pool module to attempt to attach to nodes that are already running (a9f4cbc) * ml/add-os-pid-to-port-info (erts) (2010-08-19) 1 commit - Extend erlang:port_info/1,2 to show the OS pid of a spawned process (b2d85bf) * pg/honor-start-type-in-rel-files (sasl) (2010-06-06) 1 commit - Honor start type in .rel files when building relup files (7599900) * sa/callback-attr (compiler, inets, kernel, otp, stdlib) (2010-06-08) 7 commits - Add callback specs into 'application' module in kernel (eea750e) - Add callback specs to tftp module following internet documentation (d03554c) - Add callback specs to inets_service module following possibly deprecated comments (3d0bf65) - Add '-callback' attributes in stdlib's behaviours (b131583) - Update primary bootstrap (b64ffe6) - Automatical generation of 'behaviour_info' function from '-callback' attributes (879d6fc) - Add '-callback' attribute to language syntax (d63a448) * sb/make-files-like-erlc (tools) (2010-05-18) 1 commit - Change make:files to behave more like erlc (5e9d051) * sv/epmd-check-listen (erts) (2010-07-14) 1 commit - check return value of epmd server listen() call (c5c8baa) * ta/asn1-reporting (asn1) (2010-04-23) 1 commit - asn1ct: Make formatting of errors and warnings consistent (93face2) * ta/ensure-dot-ssh-exists (ssh) (2010-09-02) 1 commit - ssh: ensure ~/.ssh exists (2c14d90) * ta/sendfile (erts, kernel) (2010-09-06) 1 commit - Implement file:sendfile/4 and file:sendfile/2 (4918fcc) -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From andrew@REDACTED Mon Oct 4 17:07:13 2010 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 4 Oct 2010 11:07:13 -0400 Subject: [erlang-patches] erl_call can now use an IP address for remote node name In-Reply-To: References: <4C9B3E30.3070505@quicheaters.org> Message-ID: <20101004150713.GH31482@hijacked.us> On Mon, Oct 04, 2010 at 03:22:14PM +0200, Bj??rn Gustavsson wrote: > On Thu, Sep 23, 2010 at 1:46 PM, Julien Barbot wrote: > > Hi, > > > > erl_call was not able to connect to a remote node when the specified node > > with -name contained an IP address. > > > > The previous inet_addr/htonl calls were not working with ip addresses. > > > > Here is the small patch: > > > > git fetch git://github.com/klyr/otp.git erl_call_get_hostent > > > > The build fails on Windows like this: > > cc.sh -MD -O2 -Wall -I. -I../include -Iconnect -Iencode -Idecode > -Imisc -Iepmd -Iregistry -Iwin32 -Ilegacy -DWIN32_THREADS > -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -o > /ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe > prog/erl_call.c prog/erl_start.c \ > > -L/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/obj/win32 > -lei_md -lsocket > erl_call.o : error LNK2019: unresolved external symbol _inet_pton > referenced in function _get_hostent > C:/cygwin/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe > : fatal error LNK1120: 1 unresolved externals > > > I will drop the branch from pu. > This could probably be resolved with a simple ifdef for windows support. Windows Vista has inetPton, but on earlier versions you probably want to use WSAStringToAddress or something (which is available since win2k). Andrew From michael.santos@REDACTED Mon Oct 4 18:25:37 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 4 Oct 2010 12:25:37 -0400 Subject: [erlang-patches] erl_call can now use an IP address for remote node name In-Reply-To: <20101004150713.GH31482@hijacked.us> References: <4C9B3E30.3070505@quicheaters.org> <20101004150713.GH31482@hijacked.us> Message-ID: <20101004162537.GA855@ecn.lan> On Mon, Oct 04, 2010 at 11:07:13AM -0400, Andrew Thompson wrote: > On Mon, Oct 04, 2010 at 03:22:14PM +0200, Bj??rn Gustavsson wrote: > > On Thu, Sep 23, 2010 at 1:46 PM, Julien Barbot wrote: > > > Hi, > > > > > > erl_call was not able to connect to a remote node when the specified node > > > with -name contained an IP address. > > > > > > The previous inet_addr/htonl calls were not working with ip addresses. > > > > > > Here is the small patch: > > > > > > git fetch git://github.com/klyr/otp.git erl_call_get_hostent > > > > > > > The build fails on Windows like this: > > > > cc.sh -MD -O2 -Wall -I. -I../include -Iconnect -Iencode -Idecode > > -Imisc -Iepmd -Iregistry -Iwin32 -Ilegacy -DWIN32_THREADS > > -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -o > > /ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe > > prog/erl_call.c prog/erl_start.c \ > > > > -L/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/obj/win32 > > -lei_md -lsocket > > erl_call.o : error LNK2019: unresolved external symbol _inet_pton > > referenced in function _get_hostent > > C:/cygwin/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe > > : fatal error LNK1120: 1 unresolved externals > > > > > > I will drop the branch from pu. > > > > This could probably be resolved with a simple ifdef for windows support. > Windows Vista has inetPton, but on earlier versions you probably want to > use WSAStringToAddress or something (which is available since win2k). Here is a minimal patch that fixes Julien's problem. The return value of inet_addr() is always big endian. diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c index 448de9a..a18d29a 100644 --- a/lib/erl_interface/src/prog/erl_call.c +++ b/lib/erl_interface/src/prog/erl_call.c @@ -614,5 +614,4 @@ static struct hostent* get_hostent(char *host) struct in_addr ip_addr; int b1, b2, b3, b4; - long addr; /* FIXME: Use inet_aton() (or inet_pton() and get v6 for free). */ @@ -620,6 +619,5 @@ static struct hostent* get_hostent(char *host) return NULL; } - addr = inet_addr(host); - ip_addr.s_addr = htonl(addr); + ip_addr.s_addr = inet_addr(host); return ei_gethostbyaddr((char *)&ip_addr,sizeof(struct in_addr), AF_INET); From klyr@REDACTED Mon Oct 4 19:11:23 2010 From: klyr@REDACTED (Julien Barbot) Date: Mon, 04 Oct 2010 19:11:23 +0200 Subject: [erlang-patches] erl_call can now use an IP address for remote node name In-Reply-To: <20101004162537.GA855@ecn.lan> References: <4C9B3E30.3070505@quicheaters.org> <20101004150713.GH31482@hijacked.us> <20101004162537.GA855@ecn.lan> Message-ID: <4CAA0ABB.2040000@quicheaters.org> On 10/04/2010 06:25 PM, Michael Santos wrote: > On Mon, Oct 04, 2010 at 11:07:13AM -0400, Andrew Thompson wrote: >> On Mon, Oct 04, 2010 at 03:22:14PM +0200, Bj??rn Gustavsson wrote: >>> On Thu, Sep 23, 2010 at 1:46 PM, Julien Barbot wrote: >>>> erl_call was not able to connect to a remote node when the specified node >>>> with -name contained an IP address. >>>> >>>> The previous inet_addr/htonl calls were not working with ip addresses. >>>> >>> The build fails on Windows like this: >>> >>> cc.sh -MD -O2 -Wall -I. -I../include -Iconnect -Iencode -Idecode >>> -Imisc -Iepmd -Iregistry -Iwin32 -Ilegacy -DWIN32_THREADS >>> -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -o >>> /ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe >>> prog/erl_call.c prog/erl_start.c \ >>> >>> -L/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/obj/win32 >>> -lei_md -lsocket >>> erl_call.o : error LNK2019: unresolved external symbol _inet_pton >>> referenced in function _get_hostent >>> C:/cygwin/ldisk/daily_build/otp_norel_pu_win32_r14b01.2010-10-03_23/otp_src_R14B01/lib/erl_interface/bin/win32/erl_call.exe >>> : fatal error LNK1120: 1 unresolved externals Sorry for this, I though it was running on Windows Vista but did not test it on previous versions. >> This could probably be resolved with a simple ifdef for windows support. >> Windows Vista has inetPton, but on earlier versions you probably want to >> use WSAStringToAddress or something (which is available since win2k). > > Here is a minimal patch that fixes Julien's problem. The return value > of inet_addr() is always big endian. So be it :) This is more simple than my solution. I do not have time to check it now, nor time to check it on a Windows version less than Windows Vista. > diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c > index 448de9a..a18d29a 100644 > --- a/lib/erl_interface/src/prog/erl_call.c > +++ b/lib/erl_interface/src/prog/erl_call.c > @@ -614,5 +614,4 @@ static struct hostent* get_hostent(char *host) > struct in_addr ip_addr; > int b1, b2, b3, b4; > - long addr; > > /* FIXME: Use inet_aton() (or inet_pton() and get v6 for free). */ > @@ -620,6 +619,5 @@ static struct hostent* get_hostent(char *host) > return NULL; > } > - addr = inet_addr(host); > - ip_addr.s_addr = htonl(addr); > + ip_addr.s_addr = inet_addr(host); > > return ei_gethostbyaddr((char *)&ip_addr,sizeof(struct in_addr), AF_INET); From tuncer.ayaz@REDACTED Mon Oct 4 22:23:28 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Oct 2010 22:23:28 +0200 Subject: [PATCH] error_handler: add no_native compiler directive Message-ID: <1286223808-5456-1-git-send-email-tuncer.ayaz@gmail.com> As suggested by Mikael Pettersson: "I agree there _may_ be a recursion between the native-traps-to-beam mechanism and the error_handler module. However, the real problem is that the chosen mechanism (point to target MFA's BEAM code) isn't flexible enough to handle newer features like on_load or (apparently) a native-mode error_handler. My planned fix is to make remote calls link to the target's Export* instead, just like BEAM does, which should solve the problems. This will however require HiPE to use different kinds of trap-to-beam stubs for remote and local calls, since local calls must not and often cannot go via Export entries. A simpler workaround for the error_handler issue (which I couldn't reproduce) is to just never compile error_handler to native code. It's not like there's a lot to gain by doing that. Please try the patch below." Patch has been tested and confirmed to make --enable-native-libs useable. Signed-off-by: Mikael Pettersson Acked-by: Tuncer Ayaz --- lib/kernel/src/error_handler.erl | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/lib/kernel/src/error_handler.erl b/lib/kernel/src/error_handler.erl index 17dd02a..db35f41 100644 --- a/lib/kernel/src/error_handler.erl +++ b/lib/kernel/src/error_handler.erl @@ -17,6 +17,7 @@ %% %CopyrightEnd% %% -module(error_handler). +-compile(no_native). %% A simple error handler. From michael.santos@REDACTED Mon Oct 4 22:44:44 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 4 Oct 2010 16:44:44 -0400 Subject: [erlang-patches] erl_call can now use an IP address for remote node name In-Reply-To: <4CAA0ABB.2040000@quicheaters.org> References: <4C9B3E30.3070505@quicheaters.org> <20101004150713.GH31482@hijacked.us> <20101004162537.GA855@ecn.lan> <4CAA0ABB.2040000@quicheaters.org> Message-ID: <20101004204444.GA5061@ecn.lan> On Mon, Oct 04, 2010 at 07:11:23PM +0200, Julien Barbot wrote: >> Here is a minimal patch that fixes Julien's problem. The return value >> of inet_addr() is always big endian. > > So be it :) This is more simple than my solution. I do not have time to > check it now, nor time to check it on a Windows version less than > Windows Vista. Is the get_hostent() function even needed? On Unix and Windows, gethostbyname() works with IPv4 addresses as well as hostnames and getting rid of get_hostent() will fix the bug with hostnames that begin with a number mentioned in the comments. I did some light testing, but maybe I am missing something subtle. If no one sees a problem, I'll send in a properly formatted patch later. diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c index 448de9a..33ff6da 100644 --- a/lib/erl_interface/src/prog/erl_call.c +++ b/lib/erl_interface/src/prog/erl_call.c @@ -119,5 +119,4 @@ static void usage_error(const char *progname, const char *switchname); static void usage(const char *progname); static int get_module(char **mbuf, char **mname); -static struct hostent* get_hostent(char *host); static int do_connect(ei_cnode *ec, char *nodename, struct call_flags *flags); static int read_stdin(char **buf); @@ -368,6 +367,6 @@ int erl_call(int argc, char **argv) */ /* FIXME better error string */ - if ((hp = get_hostent(host)) == 0) { - fprintf(stderr,"erl_call: can't get_hostent(%s)\n", host); + if ((hp = ei_gethostbyname(host)) == 0) { + fprintf(stderr,"erl_call: can't ei_gethostbyname(%s)\n", host); exit(1); } @@ -605,30 +604,4 @@ int erl_call(int argc, char **argv) ***************************************************************************/ -/* - * Get host entry (by address or name) - */ -/* FIXME: will fail on names like '2fun4you'. */ -static struct hostent* get_hostent(char *host) -{ - if (isdigit((int)*host)) { - struct in_addr ip_addr; - int b1, b2, b3, b4; - long addr; - - /* FIXME: Use inet_aton() (or inet_pton() and get v6 for free). */ - if (sscanf(host, "%d.%d.%d.%d", &b1, &b2, &b3, &b4) != 4) { - return NULL; - } - addr = inet_addr(host); - ip_addr.s_addr = htonl(addr); - - return ei_gethostbyaddr((char *)&ip_addr,sizeof(struct in_addr), AF_INET); - } - - return ei_gethostbyname(host); -} /* get_hostent */ - - - /* From tuncer.ayaz@REDACTED Mon Oct 4 23:34:06 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 4 Oct 2010 23:34:06 +0200 Subject: [PATCH] error_handler: add no_native compiler directive In-Reply-To: <1286223808-5456-1-git-send-email-tuncer.ayaz@gmail.com> References: <1286223808-5456-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: <1286228046-24669-1-git-send-email-tuncer.ayaz@gmail.com> As suggested by Mikael Pettersson: "I agree there _may_ be a recursion between the native-traps-to-beam mechanism and the error_handler module. However, the real problem is that the chosen mechanism (point to target MFA's BEAM code) isn't flexible enough to handle newer features like on_load or (apparently) a native-mode error_handler. My planned fix is to make remote calls link to the target's Export* instead, just like BEAM does, which should solve the problems. This will however require HiPE to use different kinds of trap-to-beam stubs for remote and local calls, since local calls must not and often cannot go via Export entries. A simpler workaround for the error_handler issue (which I couldn't reproduce) is to just never compile error_handler to native code. It's not like there's a lot to gain by doing that. Please try the patch below." Patch has been tested and confirmed to make --enable-native-libs useable. Signed-off-by: Mikael Pettersson Acked-by: Tuncer Ayaz --- lib/kernel/src/error_handler.erl | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/lib/kernel/src/error_handler.erl b/lib/kernel/src/error_handler.erl index 17dd02a..885eeb2 100644 --- a/lib/kernel/src/error_handler.erl +++ b/lib/kernel/src/error_handler.erl @@ -17,6 +17,11 @@ %% %CopyrightEnd% %% -module(error_handler). +%% FIXME: remove no_native directive after HiPE has been changed to make +%% remote calls link to the target's Export* like BEAM does. +%% For a detailed explanation see the commit titled +%% "error_handler: add no_native compiler directive" +-compile(no_native). %% A simple error handler. From BWhittle@REDACTED Tue Oct 5 00:39:53 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Mon, 4 Oct 2010 15:39:53 -0700 Subject: patch to enable both hard and symbolic linking on win32 Message-ID: This patch adds support for both hard and symbolic linking for Windows. This patch is a resubmission via a patch originally submitted on 9/2/2009 by Alex Tearse-Doyle. Hard linking requires XP or later and a NTFS formatted drive. Symbolic linking on Windows requires either Windows Vista, Windows Server 2008, or Windows 7. The Windows API call for creating symbolic links is dynamically executed via LoadLibrary, which allows a single binary to handle both the OS supported and OS not-supported cases. For example calling file:make_symlink on a Windows XP will return {error, enotsup}, however the same call should work as expected on Windows Vista. All the file module functions are supported: make_link/2, make_sym_link/2, read_link/1 and read_link_info/1. Code has been added so that read_info/1 could return the number of hard links on the file. However, this code has been commented out as it severally affects the performance of read_info/1 on windows. We left the code in the patch to allow anyone to enable this feature if needed. On a side note, the file.erl man page, {error, exdev} isn't listed on file's man page as a valid return type for make_link, however the error is (correctly) returned when attempting to create a hard link to file on another volume (which should result in {error, exdev}). Patch is located at git fetch git://github.com/bwhittle/otp.git win32_file_linking From fdmanana@REDACTED Tue Oct 5 01:38:40 2010 From: fdmanana@REDACTED (Filipe David Manana) Date: Tue, 5 Oct 2010 00:38:40 +0100 Subject: [erlang-patches] [PATCH] httpc module - allow streaming of bodies for PUT and POST requests In-Reply-To: References: Message-ID: Hi, I just made another change to that patch so that it allows as well for chunking (chunked transfer-encoding) the results of the streaming function. This is specially useful when the client doesn't know in advance the length of the payload. git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming Example given in the comment of the following commit: http://github.com/fdmanana/otp/commit/f15ee821a34536f73d6cf1658bb7ca789fe1ac2d cheers 2010/9/27 Bj?rn Gustavsson : > On Sun, Sep 26, 2010 at 1:04 PM, Filipe David Manana > wrote: >> Hi, >> >> The following patch, as a github commit, allows for streaming of HTTP PUT >> and POST requests. >> >> http://github.com/fdmanana/otp/commit/2d0e66d9d41c813b6fb09179bdcd29910bf0d296 > > Thanks! Will include in 'pu'. > > For your next submission, please provide the command needed to fetch > your branch. > For example: > > git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > > -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From kostis@REDACTED Tue Oct 5 13:09:19 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 05 Oct 2010 14:09:19 +0300 Subject: Add types to syntax_tools records Message-ID: <4CAB075F.4080003@cs.ntua.gr> git fetch git://github.com/kostis/otp.git syntax_tools-types Kostis From bgustavsson@REDACTED Tue Oct 5 14:33:54 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 14:33:54 +0200 Subject: [erlang-patches] [PATCH] error_handler: add no_native compiler directive In-Reply-To: <1286228046-24669-1-git-send-email-tuncer.ayaz@gmail.com> References: <1286223808-5456-1-git-send-email-tuncer.ayaz@gmail.com> <1286228046-24669-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: On Mon, Oct 4, 2010 at 11:34 PM, Tuncer Ayaz wrote: > As suggested by Mikael Pettersson: > "I agree there _may_ be a recursion between the native-traps-to-beam > mechanism and the error_handler module. However, the real problem is > that the chosen mechanism (point to target MFA's BEAM code) isn't > flexible enough to handle newer features like on_load or (apparently) > a native-mode error_handler. > > My planned fix is to make remote calls link to the target's Export* > instead, just like BEAM does, which should solve the problems. This > will however require HiPE to use different kinds of trap-to-beam stubs > for remote and local calls, since local calls must not and often > cannot go via Export entries. > > A simpler workaround for the error_handler issue (which I couldn't > reproduce) is to just never compile error_handler to native code. It's > not like there's a lot to gain by doing that. Please try the patch > below." > > Patch has been tested and confirmed to make --enable-native-libs > useable. > > Signed-off-by: Mikael Pettersson > Acked-by: Tuncer Ayaz Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 5 14:43:43 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 14:43:43 +0200 Subject: [erlang-patches] [PATCH] Fix hang when calling functions in a module with an on_load attribute from a native module In-Reply-To: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> References: <1285162705-5320-1-git-send-email-pguyot@kallisys.net> Message-ID: On Wed, Sep 22, 2010 at 3:38 PM, wrote: > From: Paul Guyot > > The fix consists in adding a new opcode for hipe stub calls that > is used to replace the call to error_handler:undefined_function/3. This > stub directly calls the function: there is no need to go through > error_handler:undefined_function/3 once we know the module is loaded, > and, more importantly, when error_handler module itself was natively > compiled (with native code loaded), the emulator did loop infinitely. > I will drop this branch from 'pu' now. The immediate reason is that it crashes certain build flavors, such as a debug build on Windows or a valgrind build. When adding new instructions, they MUST be listed in ops.tab (or in one of the other similar files for HiPE instructions). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 5 14:53:00 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 14:53:00 +0200 Subject: [erlang-patches] patch to enable both hard and symbolic linking on win32 In-Reply-To: References: Message-ID: On Tue, Oct 5, 2010 at 12:39 AM, Blaine Whittle wrote: > This patch adds support for both hard and symbolic linking for Windows. > > This patch is a resubmission via a patch originally submitted on 9/2/2009 by Alex Tearse-Doyle. > > Hard linking requires XP or later and a NTFS formatted drive. > > Symbolic linking on Windows requires either Windows Vista, Windows Server 2008, or Windows 7. The Windows API call for creating symbolic links is dynamically executed via LoadLibrary, which allows a single binary to handle both the OS supported and OS not-supported cases. For example calling file:make_symlink on a Windows XP will return {error, enotsup}, however the same call should work as expected on Windows Vista. > > All the file module functions are supported: make_link/2, make_sym_link/2, read_link/1 and read_link_info/1. Code has been added so that read_info/1 could return the number of hard links on the file. However, this code has been commented out as it severally affects the performance of read_info/1 on windows. We left the code in the patch to allow anyone to enable this feature if needed. > > On a side note, the file.erl man page, {error, exdev} isn't listed on file's man page as a valid return type for make_link, however the error is (correctly) returned when attempting to create a hard link to file on another volume (which should result in {error, exdev}). > > Patch is located at git fetch git://github.com/bwhittle/otp.git win32_file_linking > > Please read http://github.com/erlang/otp/wiki/Writing-good-commit-messages and fix your commit message (especially the first line and the overlong lines). Also, the coding style (especially the placement of opening braces) is different from the rest of the file. The indentation also seems to be off (but I have only looked at the diff, so I could be wrong). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 5 14:54:47 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 14:54:47 +0200 Subject: [erlang-patches] patch to enable both hard and symbolic linking on win32 In-Reply-To: References: Message-ID: 2010/10/5 Bj?rn Gustavsson : > Please read > > http://github.com/erlang/otp/wiki/Writing-good-commit-messages > > and fix your commit message (especially the first line and the overlong lines). > > Also, the coding style (especially the placement of opening braces) is > different from the rest of the file. The indentation also seems to be off (but > I have only looked at the diff, so I could be wrong). > One more thing: Please also fix your author information. The author of a commit should be a real name. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 5 14:56:42 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 14:56:42 +0200 Subject: [erlang-patches] Add types to syntax_tools records In-Reply-To: <4CAB075F.4080003@cs.ntua.gr> References: <4CAB075F.4080003@cs.ntua.gr> Message-ID: 2010/10/5 Kostis Sagonas : > > git fetch git://github.com/kostis/otp.git syntax_tools-types Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 5 15:13:32 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 5 Oct 2010 15:13:32 +0200 Subject: [erlang-patches] [PATCH] httpc module - allow streaming of bodies for PUT and POST requests In-Reply-To: References: Message-ID: 2010/10/5 Filipe David Manana : > Hi, > > I just made another change to that patch so that it allows as well for > chunking (chunked transfer-encoding) the results of the streaming > function. This is specially useful when the client doesn't know in > advance the length of the payload. > > git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming > Thanks! I will include in pu with minor clean-ups of the commit messages. Next time, please review our guidelines for commit message: http://github.com/erlang/otp/wiki/Writing-good-commit-messages -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From BWhittle@REDACTED Tue Oct 5 21:15:20 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Tue, 5 Oct 2010 12:15:20 -0700 Subject: [erlang-patches] patch to enable both hard and symbolic linking on win32 In-Reply-To: References: Message-ID: Fixed, please check again. -----Original Message----- From: Bj?rn Gustavsson [mailto:bgustavsson@REDACTED] Sent: Tuesday, October 05, 2010 5:55 AM To: Blaine Whittle Cc: erlang-patches@REDACTED Subject: Re: [erlang-patches] patch to enable both hard and symbolic linking on win32 2010/10/5 Bj?rn Gustavsson : > Please read > > http://github.com/erlang/otp/wiki/Writing-good-commit-messages > > and fix your commit message (especially the first line and the overlong lines). > > Also, the coding style (especially the placement of opening braces) is > different from the rest of the file. The indentation also seems to be off (but > I have only looked at the diff, so I could be wrong). > One more thing: Please also fix your author information. The author of a commit should be a real name. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From BWhittle@REDACTED Tue Oct 5 21:33:52 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Tue, 5 Oct 2010 12:33:52 -0700 Subject: Added native OS file system copy to efile Message-ID: prim_file:copy has a comment that reads " xxx Should be moved down to the driver for optimization". This patch does just that, however to ensure compatibility I haven't changed the existing implementation of prim_file:copy/3; I've added two new functions prim_file:raw_copy/2, and prim_file:raw_copy/3 which perform the copy in the driver. git fetch git://github.com/bwhittle/otp.git prim_file_copy From michael.santos@REDACTED Tue Oct 5 22:46:44 2010 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 5 Oct 2010 16:46:44 -0400 Subject: [PATCH] cerl: prevent gdb hang at prompt Message-ID: <20101005204644.GA10210@ecn.lan> On Ubuntu 10.04 with emacs versions 22 and 23, gdb hangs at the prompt after being started. Change the load order and, for v23, use a command line switch to set annotation mode. --- erts/etc/unix/cerl.src | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diff --git a/erts/etc/unix/cerl.src b/erts/etc/unix/cerl.src index 9dab9fc..de34ffc 100644 --- a/erts/etc/unix/cerl.src +++ b/erts/etc/unix/cerl.src @@ -65,6 +65,7 @@ cxargs_add() { core= GDB= +GDBARGS= GDBBP= TYPE= EMU_TYPE= @@ -277,9 +278,15 @@ else ;; esac + gdbcmd="$gdbcmd $GDBBP \ + (insert-string \"source $ROOTDIR/erts/etc/unix/etp-commands\") \ + (comint-send-input)" + # Set annotation level for gdb in emacs 22 and higher. emacs_major=`$EMACS --version | head -1 | sed 's,^[^0-9]*\([0-9]*\).*,\1,g'` - if [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then + if [ '!' -z "$emacs_major" -a $emacs_major -gt 22 ]; then + GDBARGS="--annotate=3" + elif [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then # Hack - wait for etp-commands to be loaded and then set # annotation level, could be done more beautifully than with sit-for... gdbcmd="$gdbcmd \ @@ -287,9 +294,6 @@ else (insert-string \"set annotate 3\") \ (comint-send-input)" fi - gdbcmd="$gdbcmd $GDBBP \ - (insert-string \"source $ROOTDIR/erts/etc/unix/etp-commands\") \ - (comint-send-input)" # Fire up gdb in emacs... - exec $EMACS --eval "(progn (gdb \"gdb $EMU\") $gdbcmd)" + exec $EMACS --eval "(progn (gdb \"gdb $GDBARGS $EMU\") $gdbcmd)" fi -- 1.7.0.4 From michael.santos@REDACTED Tue Oct 5 22:47:24 2010 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 5 Oct 2010 16:47:24 -0400 Subject: [PATCH] erl_call: remove get_hostent Message-ID: <20101005204724.GA10237@ecn.lan> get_hostent does not properly handle IPv4 addresses on little endian platforms and fails with hostnames beginning with a number. Remove get_hostent and use ei_gethostbyname directly since gethostbyname supports IPv4 addresses. Reported-By: Julien Barbot --- lib/erl_interface/src/prog/erl_call.c | 31 ++----------------------------- 1 files changed, 2 insertions(+), 29 deletions(-) diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c index 448de9a..33ff6da 100644 --- a/lib/erl_interface/src/prog/erl_call.c +++ b/lib/erl_interface/src/prog/erl_call.c @@ -118,7 +118,6 @@ static void usage_arg(const char *progname, const char *switchname); static void usage_error(const char *progname, const char *switchname); static void usage(const char *progname); static int get_module(char **mbuf, char **mname); -static struct hostent* get_hostent(char *host); static int do_connect(ei_cnode *ec, char *nodename, struct call_flags *flags); static int read_stdin(char **buf); static void split_apply_string(char *str, char **mod, @@ -367,8 +366,8 @@ int erl_call(int argc, char **argv) * Expand name to a real name (may be ip-address) */ /* FIXME better error string */ - if ((hp = get_hostent(host)) == 0) { - fprintf(stderr,"erl_call: can't get_hostent(%s)\n", host); + if ((hp = ei_gethostbyname(host)) == 0) { + fprintf(stderr,"erl_call: can't ei_gethostbyname(%s)\n", host); exit(1); } /* If shortnames, cut off the name at first '.' */ @@ -604,32 +603,6 @@ int erl_call(int argc, char **argv) * ***************************************************************************/ -/* - * Get host entry (by address or name) - */ -/* FIXME: will fail on names like '2fun4you'. */ -static struct hostent* get_hostent(char *host) -{ - if (isdigit((int)*host)) { - struct in_addr ip_addr; - int b1, b2, b3, b4; - long addr; - - /* FIXME: Use inet_aton() (or inet_pton() and get v6 for free). */ - if (sscanf(host, "%d.%d.%d.%d", &b1, &b2, &b3, &b4) != 4) { - return NULL; - } - addr = inet_addr(host); - ip_addr.s_addr = htonl(addr); - - return ei_gethostbyaddr((char *)&ip_addr,sizeof(struct in_addr), AF_INET); - } - - return ei_gethostbyname(host); -} /* get_hostent */ - - - /* * This function does only return on success. -- 1.7.0.4 From michael.santos@REDACTED Tue Oct 5 22:47:52 2010 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 5 Oct 2010 16:47:52 -0400 Subject: [PATCH] registry: fix zero byte allocation Message-ID: <20101005204752.GA10253@ecn.lan> Pasto in code results in 0 bytes (instead of length bytes) being allocated when the length of the key exceeds the static buffer. --- lib/erl_interface/src/registry/reg_dump.c | 4 ++-- lib/erl_interface/src/registry/reg_restore.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/erl_interface/src/registry/reg_dump.c b/lib/erl_interface/src/registry/reg_dump.c index 50a6949..dfec96b 100644 --- a/lib/erl_interface/src/registry/reg_dump.c +++ b/lib/erl_interface/src/registry/reg_dump.c @@ -157,7 +157,7 @@ static int mn_send_delete(int fd, erlang_pid *mnesia, const char *key) int len = strlen(key) + 32; /* 32 is a slight overestimate */ if (len > EISMALLBUF) - if (!(dbuf = malloc(index))) + if (!(dbuf = malloc(len))) return -1; msgbuf = (dbuf ? dbuf : sbuf); @@ -187,7 +187,7 @@ static int mn_send_write(int fd, erlang_pid *mnesia, const char *key, ei_reg_obj int len = 32 + keylen + obj->size; if (len > EISMALLBUF) - if (!(dbuf = malloc(index))) + if (!(dbuf = malloc(len))) return -1; msgbuf = (dbuf ? dbuf : sbuf); diff --git a/lib/erl_interface/src/registry/reg_restore.c b/lib/erl_interface/src/registry/reg_restore.c index 27918d2..aeb33c7 100644 --- a/lib/erl_interface/src/registry/reg_restore.c +++ b/lib/erl_interface/src/registry/reg_restore.c @@ -266,7 +266,7 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) /* make sure receive buffer can handle largest expected message */ len = maxkey + maxobj + 512; if (len > EISMALLBUF) - if (!(dbuf = malloc(index))) { + if (!(dbuf = malloc(len))) { ei_send_exit(fd,&self,&mnesia,"cannot allocate space for incoming data"); return -1; } -- 1.7.0.4 From kostis@REDACTED Wed Oct 6 00:20:46 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 06 Oct 2010 01:20:46 +0300 Subject: Avoid erl_lint internal errors for badly formed export_type declarations Message-ID: <4CABA4BE.5040109@cs.ntua.gr> Avoid errors for a badly formed export_type declarations In the following program, erl_lint crashed with an erl_lint internal error. With this patch it does not, but prints "bad export_type declaration" errors instead. -module(baz). -export([test/0]). -export_type(t/0). -export_type([3.14]). -type t() :: any(). test() -> 42. I have not added the program in the erl_lint tests (somebody might want to do that) but the patch that fixes this is here: git fetch git://github.com/kostis/otp.git bad-export_type Kostis From BWhittle@REDACTED Wed Oct 6 02:29:21 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Tue, 5 Oct 2010 17:29:21 -0700 Subject: Added win32 specific build artifacts to .gitignore Message-ID: Simple additions to the .gitignore file so that the build directory is a little more manageable via the git tools on win32. From BWhittle@REDACTED Wed Oct 6 02:18:31 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Tue, 5 Oct 2010 17:18:31 -0700 Subject: Added native OS file system copy to efile In-Reply-To: References: Message-ID: Fixed an issue, and included the .beam file since prim_file is preloaded. New location is at git fetch git://github.com/bwhittle/otp.git prim_file_copy2 -----Original Message----- From: erlang-patches@REDACTED [mailto:erlang-patches@REDACTED] On Behalf Of Blaine Whittle Sent: Tuesday, October 05, 2010 12:34 PM To: erlang-patches@REDACTED Subject: [erlang-patches] Added native OS file system copy to efile prim_file:copy has a comment that reads " xxx Should be moved down to the driver for optimization". This patch does just that, however to ensure compatibility I haven't changed the existing implementation of prim_file:copy/3; I've added two new functions prim_file:raw_copy/2, and prim_file:raw_copy/3 which perform the copy in the driver. git fetch git://github.com/bwhittle/otp.git prim_file_copy From cristian@REDACTED Wed Oct 6 04:19:11 2010 From: cristian@REDACTED (Cristian Greco) Date: Wed, 6 Oct 2010 04:19:11 +0200 Subject: Fix a bug in the implementation of the pseudo-random number generator Message-ID: <20101006041911.5cf4247e@regolo> Hi, the current implementation of the Wichmann-Hill prng contains an error, which is fixed in the following branch. git fetch git@REDACTED:cristiangreco/otp.git cg/fix-prng More details provided in the following mail. This commit fixes an error in the mathematical formula of the Wichmann-Hill pseudo-random number generator. In particular, the implementation used until now produces sequences which differ from the expected ones by an extra starting number, which is instead the very last value of the sequence. This bug amplified the effect of extremely correlated initial numbers when seeding different generators with very similar seed values. --- lib/stdlib/src/random.erl | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/stdlib/src/random.erl b/lib/stdlib/src/random.erl index 01227c2..d94722c 100644 --- a/lib/stdlib/src/random.erl +++ b/lib/stdlib/src/random.erl @@ -86,7 +86,7 @@ uniform() -> B2 = (A2*172) rem 30307, B3 = (A3*170) rem 30323, put(random_seed, {B1,B2,B3}), - R = A1/30269 + A2/30307 + A3/30323, + R = B1/30269 + B2/30307 + B3/30323, R - trunc(R). %% uniform(N) -> I @@ -110,7 +110,7 @@ uniform_s({A1, A2, A3}) -> B1 = (A1*171) rem 30269, B2 = (A2*172) rem 30307, B3 = (A3*170) rem 30323, - R = A1/30269 + A2/30307 + A3/30323, + R = B1/30269 + B2/30307 + B3/30323, {R - trunc(R), {B1,B2,B3}}. %% uniform_s(N, State) -> {I, NewState} -- 1.7.1 Thanks, -- Cristian Greco GPG key ID: 0xCF4D32E4 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From cristian@REDACTED Wed Oct 6 04:32:49 2010 From: cristian@REDACTED (Cristian Greco) Date: Wed, 6 Oct 2010 04:32:49 +0200 Subject: [erlang-patches] Fix a bug in the implementation of the pseudo-random number generator In-Reply-To: <20101006041911.5cf4247e@regolo> References: <20101006041911.5cf4247e@regolo> Message-ID: <20101006043249.4056afb4@regolo> [ sorry for the very long mail ] On Wed, 6 Oct 2010 04:19:11 +0200 Cristian Greco wrote: > Hi, > > the current implementation of the Wichmann-Hill prng contains an > error, which is fixed in the following branch. > > git fetch git@REDACTED:cristiangreco/otp.git cg/fix-prng > > More details provided in the following mail. > > > This commit fixes an error in the mathematical formula of the > Wichmann-Hill pseudo-random number generator. In particular, the > implementation used until now produces sequences which differ from the > expected ones by an extra starting number, which is instead the very > last value of the sequence. This bug amplified the effect of extremely > correlated initial numbers when seeding different generators with very > similar seed values. The first number emitted from the functions in module 'random' (either via uniform/0, uniform/1, uniform_s/1 or uniform_s/2) is not the first number of the real sequence generated by the Wichmann-Hill prng. This kind of generator is a combination of three simple ones (LCGs): X_i = (171 * X_i-1) mod 30269 Y_i = (172 * Y_i-1) mod 30307 Z_i = (170 * Z_i-1) mod 30323 with U_i = (X_i / 30269 + Y_i / 30307 + Z_i / 30323) mod 1 while the actual implementation in random.erl is something like: U_i = (X_i-1 / 30269 + Y_i-1 / 30307 + Z_i-1 / 30323) mod 1 Please compare the following outputs: the first _6_ float numbers emitted in Erlang with the default seed of {3172, 9814, 20125}, 1> [ random:uniform() || _ <- lists:seq(1,6) ]. [0.09230089279334841,0.4435846174457203,0.7230402056221108, 0.94581636451987,0.5014907142064751,0.311326754804393] with the first _5_ floats in Python, $ python -c "import random; r = random.WichmannHill(); r.setstate((1, (3172, 9814, 20125), None)); print [ r.random() for n in range(0,5) ]" [0.44358461744572031, 0.7230402056221108, 0.94581636451986995, 0.50149071420647506, 0.31132675480439298] or in R, > .Random.seed <- c(400L, 3172L, 9814L, 20125L) > runif(5) [1] 0.4435846 0.7230402 0.9458164 0.5014907 0.3113268 where the first element of the vector in .Random.seed is the code of the WH prng. It is clear that the sequence of values produced by the implementation in Erlang is somehow shifted compared to that produced by other implementation. We get the same results sampling random integers in (1, 1000000) from the generator: Erlang: 1> [ random:uniform(1000000) || _ <- lists:seq(1,6) ]. [92301,443585,723041,945817,501491,311327] Python: $ python -c "import random; r = random.WichmannHill(); r.setstate((1, (3172, 9814, 20125), None)); print [ r.randint(1,1000000) for n in range(0,5) ]" [443585, 723041, 945817, 501491, 311327] R: > .Random.seed <- c(400L, 3172L, 9814L, 20125L) > sample(1:1000000, 5, replace=T) [1] 443585 723041 945817 501491 311327 To be more precise, given the math involved in the Wichmann-Hill generator and the kind of error of the implementation, the first number emitted in Erlang is not completely unrelated to the real sequence generated by the seed {3172, 9814, 20125}. Indeed it is the _very last_ number of the sequence, as demonstrated by: $ python -c "import random; r = random.WichmannHill(); r.setstate((1, (3172, 9814, 20125), None)); r.jumpahead(6953607871643); print r.random()" 0.0923008927933 where the jumpahead(N) function from the Python library changes the internal state of the generator by "jumping" N states away from the current one, while the number passed as argument is `P - 1`, where P = 6953607871644 ~ 6.95e+12 is the period of this generator. For the sake of completeness, the following command prints the state of the generator before and after emitting the last number of the sequence, and then loops again: $ python -c "import random; r = random.WichmannHill(); r.setstate((1, (3172, 9814, 20125), None)); r.jumpahead(6953607871643); print r.getstate(); print r.random(); print r.getstate(); print r.random()" (1, (21968, 17325, 20631), None) 0.0923008927933 (1, (3172, 9814, 20125), None) 0.443584617446 and it is worth noting that: 1> F = fun({A1,A2,A3}) -> {(A1*171) rem 30269, (A2*172) rem 30307, (A3*170) rem 30323} end. #Fun 2> F({21968, 17325, 20631}). {3172,9814,20125} I ran some tests using the dieharder tool and it _seems_ this bug does not considerably affects the randomness of the generated sequence; in any case, I'm not a theorist so can't judge any possible weakness of this implementation. On the other hand, this error clearly explains a common misconception about the pseudo-random number generator in Erlang. Initializing a number of parallel generators (here I mean concurrent processes, not independent parallel streams) using very similar seeds from subsequent calls to erlang:now/0 produces sequences whose first numbers are extremely correlated (e.g. slightly increasing values, or even identical values for sufficiently small values of N in random:uniform/1) but then seem to "diverge pretty rapidly". The problem here is that the first number within each sequence is not the first one emitted by the prng, it is instead just like a "poor hash" of the starting seeds. This behavior seems to be well known and has been discussed on the lists: http://www.erlang.org/cgi-bin/ezmlm-cgi?4:53165:201009 http://www.erlang.org/cgi-bin/ezmlm-cgi?4:39751:200811 http://www.erlang.org/cgi-bin/ezmlm-cgi?4:39752:200811 http://www.erlang.org/cgi-bin/ezmlm-cgi?4:39756:200811 This is an example using a fixed module called 'fixrand': 1> S = lists:map(fun() -> {A,B,C} = erlang:now(), random:seed(A,B,C), io:format("~p~n", [[random:uniform(10) || _ <- lists:seq(1,10)]]), {A,B,C} end, lists:seq(1,5)). [4,1,2,2,10,9,1,7,4,6] [5,7,2,4,10,9,3,9,8,2] [5,4,2,2,9,10,9,8,10,10] [5,1,5,9,9,2,6,6,6,10] [5,6,4,9,4,8,10,9,4,10] [{1286,310211,426712}, {1286,310211,428248}, {1286,310211,428375}, {1286,310211,428497}, {1286,310211,428585}] 2> lists:map(fun({A,B,C}) -> fixrand:seed(A,B,C), io:format("~p~n", [[fixrand:uniform(10) || _ <- lists:seq(1,10)]]) end, S). [1,2,2,10,9,1,7,4,6,9] [7,2,4,10,9,3,9,8,2,4] [4,2,2,9,10,9,8,10,10,8] [1,5,9,9,2,6,6,6,10,5] [6,4,9,4,8,10,9,4,10,10] [ok,ok,ok,ok,ok] As a final remark, please note that a fixed version of the Wichmann-Hill generator, although still considered acceptable nowadays, indeed shows the same flaw when seeding with very very similar values and sampling from small integer ranges, for example: 1> S = [ erlang:now() || _ <- lists:seq(1,5) ]. [{1286,310474,823741}, {1286,310474,823744}, {1286,310474,823746}, {1286,310474,823748}, {1286,310474,823750}] 2> lists:map(fun({A,B,C}) -> fixrand:seed(A,B,C), io:format("~p~n", [[fixrand:uniform(10) || _ <- lists:seq(1,10)]]) end, S). [5,2,10,8,7,6,9,6,1,7] [5,10,10,1,8,5,4,7,2,6] [5,9,1,7,5,5,10,2,9,5] [5,8,1,2,3,4,6,6,6,4] [5,7,2,8,10,4,2,10,3,3] [ok,ok,ok,ok,ok] 3> lists:map(fun({A,B,C}) -> fixrand:seed(A,B,C), io:format("~p~n", [[fixrand:uniform(10000) || _ <- lists:seq(1,10)]]) end, S). [4284,1169,9327,7355,6266,5247,8976,5920,525,6819] [4452,9761,9994,714,7380,4684,3276,6982,1104,5266] [4564,8823,439,6287,4789,4309,9477,1024,8157,4230] [4676,7884,883,1860,2199,3934,5677,5065,5209,3195] [4788,6946,1328,7433,9609,3558,1877,9107,2262,2159] [ok,ok,ok,ok,ok] or when using seeds with congruent numbers, such as in the following example, where 510934 = 450288 mod 30323 http://www.erlang.org/cgi-bin/ezmlm-cgi?4:39986:200811 This bug seems to be around since at least 11 years, which is the release year of the oldest available official release (R6B-0, released 1999) at http://erlang.org/download. Thanks, -- Cristian Greco GPG key ID: 0xCF4D32E4 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From bgustavsson@REDACTED Wed Oct 6 14:26:35 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 14:26:35 +0200 Subject: [erlang-patches] Added win32 specific build artifacts to .gitignore In-Reply-To: References: Message-ID: On Wed, Oct 6, 2010 at 2:29 AM, Blaine Whittle wrote: > Simple additions to the .gitignore file so that the build directory is a little more manageable via the git tools on win32. > > But where is the branch? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 14:37:37 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 14:37:37 +0200 Subject: [erlang-patches] Avoid erl_lint internal errors for badly formed export_type declarations In-Reply-To: <4CABA4BE.5040109@cs.ntua.gr> References: <4CABA4BE.5040109@cs.ntua.gr> Message-ID: 2010/10/6 Kostis Sagonas : > > git fetch git://github.com/kostis/otp.git bad-export_type > Thanks! Will include in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 14:55:13 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 14:55:13 +0200 Subject: [erlang-patches] [PATCH] cerl: prevent gdb hang at prompt In-Reply-To: <20101005204644.GA10210@ecn.lan> References: <20101005204644.GA10210@ecn.lan> Message-ID: On Tue, Oct 5, 2010 at 10:46 PM, Michael Santos wrote: > On Ubuntu 10.04 with emacs versions 22 and 23, gdb hangs at the prompt > after being started. Change the load order and, for v23, use a command > line switch to set annotation mode. Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 14:59:22 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 14:59:22 +0200 Subject: [erlang-patches] [PATCH] erl_call: remove get_hostent In-Reply-To: <20101005204724.GA10237@ecn.lan> References: <20101005204724.GA10237@ecn.lan> Message-ID: On Tue, Oct 5, 2010 at 10:47 PM, Michael Santos wrote: > get_hostent does not properly handle IPv4 addresses on little endian > platforms and fails with hostnames beginning with a number. Remove > get_hostent and use ei_gethostbyname directly since gethostbyname supports > IPv4 addresses. Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 15:05:40 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 15:05:40 +0200 Subject: [erlang-patches] [PATCH] registry: fix zero byte allocation In-Reply-To: <20101005204752.GA10253@ecn.lan> References: <20101005204752.GA10253@ecn.lan> Message-ID: On Tue, Oct 5, 2010 at 10:47 PM, Michael Santos wrote: > Pasto in code results in 0 bytes (instead of length bytes) being allocated > when the length of the key exceeds the static buffer. Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 15:14:20 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 15:14:20 +0200 Subject: [erlang-patches] Fix a bug in the implementation of the pseudo-random number generator In-Reply-To: <20101006041911.5cf4247e@regolo> References: <20101006041911.5cf4247e@regolo> Message-ID: On Wed, Oct 6, 2010 at 4:19 AM, Cristian Greco wrote: > Hi, > > the current implementation of the Wichmann-Hill prng contains an error, > which is fixed in the following branch. > > git fetch git@REDACTED:cristiangreco/otp.git cg/fix-prng > Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 15:45:03 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 15:45:03 +0200 Subject: [erlang-patches] Added native OS file system copy to efile In-Reply-To: References: Message-ID: On Tue, Oct 5, 2010 at 9:33 PM, Blaine Whittle wrote: > prim_file:copy has a comment that reads " xxx Should be moved down to the driver for optimization". This patch does just that, however to ensure compatibility I haven't changed the existing implementation of prim_file:copy/3; I've added two new functions prim_file:raw_copy/2, and prim_file:raw_copy/3 which perform the copy in the driver. > > git fetch git://github.com/bwhittle/otp.git prim_file_copy > As currently written, copying a huge file will block the thread for entire duration of the copy operation. That is not OK. If the copy function is to be moved to the driver, there must be a limit to the number of bytes that the driver is allowed to copy at one time. When that number has been exceeded, the driver should return control to the Erlang process to allow other Erlang processes to be run. That was the original idea behind the comment, but it is not really clear that it would improve performance much. Do you have a real need for a faster file copy operation or is it just nice to have? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 6 16:23:34 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 6 Oct 2010 16:23:34 +0200 Subject: [erlang-patches] patch to enable both hard and symbolic linking on win32 In-Reply-To: References: Message-ID: 2010/10/5 Blaine Whittle : > Fixed, please check again. Thanks! That's better. This time, I have fixed the remaining issues in the commit message (broken the over-long line). For your next submission, please follow the guidelines (make sure to break lines to appr. 70 characters so that the commit message can be easily read in gitk). Make sure to use your real name in commits by running this command: git config --global user.name "Blaine Whittle" I have also fixed some more code-style issues (in a separate commit for now so that you can review my changes). You can use this link http://erlang.github.com/otp/ to examine the commits in your branch as included in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From BWhittle@REDACTED Wed Oct 6 17:48:18 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Wed, 6 Oct 2010 08:48:18 -0700 Subject: [erlang-patches] Added win32 specific build artifacts to .gitignore In-Reply-To: References: Message-ID: Not sure why I forgot that part, it's at. git fetch git://github.com/bwhittle/otp.git git_ignore -----Original Message----- From: Bj?rn Gustavsson [mailto:bgustavsson@REDACTED] Sent: Wednesday, October 06, 2010 5:27 AM To: Blaine Whittle Cc: erlang-patches@REDACTED Subject: Re: [erlang-patches] Added win32 specific build artifacts to .gitignore On Wed, Oct 6, 2010 at 2:29 AM, Blaine Whittle wrote: > Simple additions to the .gitignore file so that the build directory is a little more manageable via the git tools on win32. > > But where is the branch? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From vinoski@REDACTED Wed Oct 6 17:54:09 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 6 Oct 2010 11:54:09 -0400 Subject: fix ei_x_format to handle unary numeric operators Message-ID: Passing a negative numeric constant or a positive constant with an explicit leading plus sign within the format string to ei_x_format or ei_x_format_wo_ver failed due to the number parser not handling the unary operators. This patch fixes that and includes an augmented unit test that catches the problem. git fetch git://github.com/vinoski/otp.git ei-number-parsing Simon Hillerbring found this problem by noticing that this failed: erl_call -c COOKIE -sname test@REDACTED -a 'erlang integer_to_list [-1]' See the original erlang-questions thread here: --steve From BWhittle@REDACTED Wed Oct 6 18:24:06 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Wed, 6 Oct 2010 09:24:06 -0700 Subject: [erlang-patches] Added native OS file system copy to efile In-Reply-To: References: Message-ID: Understood... This change along led to a 30% performance increase for our app (yes we do a bit file coping). The reason for implementing the OS copy in efile (or a standalone driver) instead of a NIF is so that async threads using the +A command line option (which don't block the scheduler). I implemented this in efile as the prim_file.erl comment suggested that there was a preference for a driver implementation of copy. What are your thoughts about adding a check to see if Async threads are enabled and returning enotsup otherwise? On semi unrelated note, I'm about to implement a function to quickly verify if a file exists. At the moment, I'm just using prim_file:read_file_info/1 for this. The problem is that read_file_info is a little too heavy for this simple check as it collects extra info about the file. This means that if a file exists, the file existence check takes 2 or 3 times longer than when the file doesn't exist. For most cases, I'm sure this is fine but when you are doing a few million of checks, it adds up. If there is any interest in having this as part of the efile API, I can add this function to efile + the posix and win32 implementations (fyi there already is a efile_may_openfile function which is close to what I want, the problem is that it's not exposed to prim_file), otherwise I'll implement a light weight file exists check as a NIF. -----Original Message----- From: Bj?rn Gustavsson [mailto:bgustavsson@REDACTED] Sent: Wednesday, October 06, 2010 6:45 AM To: Blaine Whittle Cc: erlang-patches@REDACTED Subject: Re: [erlang-patches] Added native OS file system copy to efile On Tue, Oct 5, 2010 at 9:33 PM, Blaine Whittle wrote: > prim_file:copy has a comment that reads " xxx Should be moved down to the driver for optimization". This patch does just that, however to ensure compatibility I haven't changed the existing implementation of prim_file:copy/3; I've added two new functions prim_file:raw_copy/2, and prim_file:raw_copy/3 which perform the copy in the driver. > > git fetch git://github.com/bwhittle/otp.git prim_file_copy > As currently written, copying a huge file will block the thread for entire duration of the copy operation. That is not OK. If the copy function is to be moved to the driver, there must be a limit to the number of bytes that the driver is allowed to copy at one time. When that number has been exceeded, the driver should return control to the Erlang process to allow other Erlang processes to be run. That was the original idea behind the comment, but it is not really clear that it would improve performance much. Do you have a real need for a faster file copy operation or is it just nice to have? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From BWhittle@REDACTED Wed Oct 6 20:21:36 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Wed, 6 Oct 2010 11:21:36 -0700 Subject: Added win32 large address aware linker option Message-ID: Added win32 large address aware linker option This link option just sets a flag in the binary and doesn't change OS compatibility. This allows the Erlang VM to use up to 3 gigs of address space instead of the default of 2 gigs. git fetch git://github.com/bwhittle/otp.git win32_laa From kostis@REDACTED Wed Oct 6 21:55:09 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 06 Oct 2010 22:55:09 +0300 Subject: Changes to supervisor.erl Message-ID: <4CACD41D.8020106@cs.ntua.gr> Type and spec related changes - Export two more types so that they can be used in other modules - Correct some types and specs - Add spec for behaviour_info/1 git fetch git://github.com/kostis/otp.git supervisor-export-types Kostis From BWhittle@REDACTED Wed Oct 6 22:02:58 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Wed, 6 Oct 2010 13:02:58 -0700 Subject: Change to allow table viewer to display refs, ports and small binaries Message-ID: Trivial formatting change that improves the usability of tv. Table viewer displayed #Port, #Ref, or #Bin as place holders for their respective object types in ets and mnesia tables. This can make table viewer difficult to use when viewing tables containing those data types. It doesn't make sense to render large binaries so #Bin will still be used for binaries that exceed 100 bytes. git fetch git://github.com/bwhittle/otp.git tv_render_fix From kostis@REDACTED Thu Oct 7 01:14:21 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 07 Oct 2010 02:14:21 +0300 Subject: Strengthen specs of lists Message-ID: <4CAD02CD.50101@cs.ntua.gr> Strengthen and modernize specs Strengthen some specs that were a bit more liberal than they should. While at it, change uses of the is_subtype guard to :: which is more modern and compact. git fetch it://github.com/kostis/otp.git strengthen-lists-specs Kostis From bgustavsson@REDACTED Thu Oct 7 14:10:25 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 14:10:25 +0200 Subject: [erlang-patches] Added win32 large address aware linker option In-Reply-To: References: Message-ID: On Wed, Oct 6, 2010 at 8:21 PM, Blaine Whittle wrote: > Added win32 large address aware linker option > > This link option just sets a flag in the binary and doesn't change > OS compatibility. This allows the Erlang VM to use up to 3 gigs of > address space instead of the default of 2 gigs. > > git fetch git://github.com/bwhittle/otp.git win32_laa Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 14:47:01 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 14:47:01 +0200 Subject: [erlang-patches] Added win32 specific build artifacts to .gitignore In-Reply-To: References: Message-ID: 2010/10/6 Blaine Whittle : > Not sure why I forgot that part, it's at. > > git fetch git://github.com/bwhittle/otp.git git_ignore Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 14:50:49 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 14:50:49 +0200 Subject: [erlang-patches] fix ei_x_format to handle unary numeric operators In-Reply-To: References: Message-ID: On Wed, Oct 6, 2010 at 5:54 PM, Steve Vinoski wrote: > > git fetch git://github.com/vinoski/otp.git ei-number-parsing Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 15:21:38 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 15:21:38 +0200 Subject: [erlang-patches] Added native OS file system copy to efile In-Reply-To: References: Message-ID: We will think more about the first part of your email. Here is the answer for the second part... 2010/10/6 Blaine Whittle : > On semi unrelated note, I'm about to implement a function to quickly verify if a file exists. At the moment, I'm just using prim_file:read_file_info/1 for this. The problem is that read_file_info is a little too heavy for this simple check as it collects extra info about the file. This means that if a file exists, the file existence check takes 2 or 3 times longer than when the file doesn't exist. For most cases, I'm sure this is fine but when you are doing a few million of checks, it adds up. > > If there is any interest in having this as part of the efile API, I can add this function to efile + the posix and win32 implementations (fyi there already is a efile_may_openfile function which is close to what I want, the problem is that it's not exposed to prim_file), otherwise I'll implement a light weight file exists check as a NIF. I think there is good chance that we may include such an extension in a future release, provided that your patch is OTP-worthy. See "Making a patch OTP-worthy" in: http://github.com/erlang/otp/wiki/Submitting-patches -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 15:24:18 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 15:24:18 +0200 Subject: [erlang-patches] Changes to supervisor.erl In-Reply-To: <4CACD41D.8020106@cs.ntua.gr> References: <4CACD41D.8020106@cs.ntua.gr> Message-ID: 2010/10/6 Kostis Sagonas : > Type and spec related changes > > - Export two more types so that they can be used in other modules > - Correct some types and specs > - Add spec for behaviour_info/1 > > > git fetch git://github.com/kostis/otp.git supervisor-export-types Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 15:32:19 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 15:32:19 +0200 Subject: [erlang-patches] Strengthen specs of lists In-Reply-To: <4CAD02CD.50101@cs.ntua.gr> References: <4CAD02CD.50101@cs.ntua.gr> Message-ID: 2010/10/7 Kostis Sagonas : > Strengthen and modernize specs > > Strengthen some specs that were a bit more liberal than they should. > While at it, change uses of the is_subtype guard to :: which is more > modern and compact. > > git fetch it://github.com/kostis/otp.git strengthen-lists-specs Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 7 15:53:05 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 7 Oct 2010 15:53:05 +0200 Subject: [erlang-patches] Change to allow table viewer to display refs, ports and small binaries In-Reply-To: References: Message-ID: On Wed, Oct 6, 2010 at 10:02 PM, Blaine Whittle wrote: > Trivial formatting change that improves the usability of tv. > > Table viewer displayed #Port, #Ref, or #Bin as place holders for their > respective object types in ets and mnesia tables. This can make table > viewer difficult to use when viewing tables containing those data types. > It doesn't make sense to render large binaries so #Bin will still be > used for binaries that exceed 100 bytes. > > git fetch git://github.com/bwhittle/otp.git tv_render_fix > Thanks! I will include it in with the following line: write(Term, _D) when is_binary(Term) andalso size(Term) > 100 -> "#Bin"; changed to: write(Term, _D) when is_binary(Term), byte_size(Term) > 100 -> "#Bin"; (Coding style) Points to ponder: Wouldn't it be more useful to truncate long binaries and end them with "..."? Shouldn't bitstrings also be handled? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From BWhittle@REDACTED Fri Oct 8 01:52:49 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Thu, 7 Oct 2010 16:52:49 -0700 Subject: prim_file:exists/{1,2} a new lightweight file existence check Message-ID: This function is meant to be faster / simpler check for the existence of a file or directory. Most all erlang functions that do this end up calling prim_file:read_file_info at some point. When read_file_info is used as a simple existence check, time is wasted collecting file stats data only to turn around and throw the data away. If a file exists, read_file_info can take three times longer to return than when the file doesn't exist. Return values are true, false, or {error, eisdir}. The low level implementation of this function was already defined via efile_may_openfile, this patch just exposes it to prim_file. filelib:is_dir/1, filelib:is_file/1, filelib:is_regular/1 could be changed from calling read_file_info to prim_file:exists or better yet to calling to a yet unwritten file:exists function. Patch located at git fetch git://github.com/bwhittle/otp.git efile_exists Note that I mapped the API function to 33 instead of 32. This is just for my own convenience as I use the raw_copy patch which currently defines raw_copy at 32. If the raw_copy patch submission doesn't work out, the FILE_EXISTS macro can be easily refactored from 33 to 32 by changing efile_drv.c and prim_file.erl. From BWhittle@REDACTED Fri Oct 8 03:57:13 2010 From: BWhittle@REDACTED (Blaine Whittle) Date: Thu, 7 Oct 2010 18:57:13 -0700 Subject: [erlang-patches] prim_file:exists/{1,2} a new lightweight file existence check In-Reply-To: References: Message-ID: I haven't tested the function on pipes or block devices. Although this was written with the assumption of testing for files, the function can be used to check for directories by matching the result against {error, eisdir}. -----Original Message----- From: dave.peticolas@REDACTED [mailto:dave.peticolas@REDACTED] On Behalf Of Dave Peticolas Sent: Thursday, October 07, 2010 6:40 PM To: Blaine Whittle Cc: erlang-patches@REDACTED Subject: Re: [erlang-patches] prim_file:exists/{1,2} a new lightweight file existence check On Thu, Oct 7, 2010 at 4:52 PM, Blaine Whittle wrote: > This function is meant to be faster / simpler check for the existence > of a file or directory. ?Most all erlang functions that do this end > up calling prim_file:read_file_info at some point. ? When > read_file_info is used as a simple existence check, time is wasted > collecting file stats data only to turn around and throw the data > away. ?If a file exists, read_file_info can take three times longer > to return than when the file doesn't exist. ?Return values are true, > false, or {error, eisdir}. ?The low level implementation of this > function was already defined via efile_may_openfile, this patch just > exposes it to prim_file. ?filelib:is_dir/1, filelib:is_file/1, > filelib:is_regular/1 could be changed from calling read_file_info to > prim_file:exists or better yet to calling to a yet unwritten > file:exists function. But efile_may_openfile isn't just testing existence is it? As I read it it requires the path to be a regular file. What if you want to test for the existence of a pipe or a block device? > Patch located at > git fetch git://github.com/bwhittle/otp.git efile_exists > > Note that I mapped the API function to 33 instead of 32. ? This is just > for my own convenience as I use the raw_copy patch which currently > defines raw_copy at 32. ? If the raw_copy patch submission doesn't > work out, the FILE_EXISTS macro can be easily refactored from 33 to > 32 by changing efile_drv.c and prim_file.erl. > > From tuncer.ayaz@REDACTED Sat Oct 9 21:02:16 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 9 Oct 2010 21:02:16 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/9/15 Bj?rn Gustavsson: > On Sun, Sep 12, 2010 at 2:00 AM, Tuncer Ayaz wrote: >> >> git fetch git://github.com/tuncer/otp.git sendfile > > Thanks! Included in 'pu'. Please refetch updated patch with sendfile/3 for chunked send and minor cleanups: git fetch git://github.com/tuncer/otp.git sendfile From pguyot@REDACTED Sun Oct 10 15:16:34 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 10 Oct 2010 15:16:34 +0200 Subject: Fix hibernate with HiPE Message-ID: Hello, The "Fix hibernate with HiPE" patch I previously submitted and that is currently on 'pu' can actually lead to a segfault. The 'pu' commit is here: http://github.com/erlang/otp/commit/b1d738d7548151b37d264757ea7830b79275edad Please also apply this commit: http://github.com/pguyot/otp/commit/57f4c9ccac45f06ec47846a117908a2dd54962b3 Or just fetch the two commits at once: git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch Indeed, the initial commits empties the stack when hibernating by setting the stack pointer to the beginning of the stack (or the end, depending on the native stack direction). But popping arguments when returning from the TRAP condition can actually lead to a segfault because it would access memory beyond the stack zone. With the new commit: - memory used by the native stack is freed (since this is the point of hibernating), and all stack-related pointers are set to NULL; - when returning from TRAP, if the native stack pointer (nsp) is NULL, no argument is popped (since there is nothing to pop). Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From tuncer.ayaz@REDACTED Sun Oct 10 18:20:15 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 10 Oct 2010 18:20:15 +0200 Subject: [PATCH] lists: fix foreach/2's documented spec Message-ID: <1286727615-10698-1-git-send-email-tuncer.ayaz@gmail.com> --- lib/stdlib/doc/src/lists.xml | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml index 92c4eb4..ac19a0d 100644 --- a/lib/stdlib/doc/src/lists.xml +++ b/lib/stdlib/doc/src/lists.xml @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> - foreach(Fun, List) -> void() + foreach(Fun, List) -> ok Apply a function to each element of a list - Fun = fun(Elem) -> void() + Fun = fun(Elem) -> _  Elem = term() List = [term()] From tuncer.ayaz@REDACTED Sun Oct 10 19:01:25 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 10 Oct 2010 19:01:25 +0200 Subject: [PATCH] lists: fix foreach/2's documented spec In-Reply-To: <1286727615-10698-1-git-send-email-tuncer.ayaz@gmail.com> References: <1286727615-10698-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: On Sun, Oct 10, 2010 at 6:20 PM, Tuncer Ayaz wrote: > --- > ?lib/stdlib/doc/src/lists.xml | ? ?4 ++-- > ?1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml > index 92c4eb4..ac19a0d 100644 > --- a/lib/stdlib/doc/src/lists.xml > +++ b/lib/stdlib/doc/src/lists.xml > @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> > ? ? ? > ? ? > ? ? > - ? ? ?foreach(Fun, List) -> void() > + ? ? ?foreach(Fun, List) -> ok > ? ? ? Apply a function to each element of a list > ? ? ? > - ? ? ? ?Fun = fun(Elem) -> void() > + ? ? ? ?Fun = fun(Elem) -> _ > ? ? ? ?  Elem = term() > ? ? ? ? List = [term()] > ? ? ? > Please ignore this patch. I will mail a corrected patch in a minute. Thanks to Kostis for noticing that it should be term() and not _. From tuncer.ayaz@REDACTED Sun Oct 10 19:02:10 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 10 Oct 2010 19:02:10 +0200 Subject: [PATCH] Fix lists:foreach/2's documented spec Message-ID: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> --- lib/stdlib/doc/src/lists.xml | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml index 92c4eb4..0cf88fe 100644 --- a/lib/stdlib/doc/src/lists.xml +++ b/lib/stdlib/doc/src/lists.xml @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> - foreach(Fun, List) -> void() + foreach(Fun, List) -> ok Apply a function to each element of a list - Fun = fun(Elem) -> void() + Fun = fun(Elem) -> term()  Elem = term() List = [term()] From mikpe@REDACTED Sun Oct 10 19:32:20 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Sun, 10 Oct 2010 19:32:20 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: References: Message-ID: <19633.63652.923839.994702@pilspetsen.it.uu.se> Paul Guyot writes: > Hello, > > The "Fix hibernate with HiPE" patch I previously submitted and that is currently on 'pu' can actually lead to a segfault. > The 'pu' commit is here: > http://github.com/erlang/otp/commit/b1d738d7548151b37d264757ea7830b79275edad > > Please also apply this commit: > http://github.com/pguyot/otp/commit/57f4c9ccac45f06ec47846a117908a2dd54962b3 > > Or just fetch the two commits at once: > git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch > > Indeed, the initial commits empties the stack when hibernating by setting the stack pointer to the beginning of the stack (or the end, depending on the native stack direction). But popping arguments when returning from the TRAP condition can actually lead to a segfault because it would access memory beyond the stack zone. > > With the new commit: > - memory used by the native stack is freed (since this is the point of hibernating), and all stack-related pointers are set to NULL; > - when returning from TRAP, if the native stack pointer (nsp) is NULL, no argument is popped (since there is nothing to pop). Looking at the patch via github's awful web interface it looks like it will cause the 'is_recursive' variable to be conditionally defined and possibly used uninitialized (at least as far as GCC will be able to tell). Thus, NAK for now. I would have provided the fix but you didn't include the patch in the email so I can't. From pguyot@REDACTED Sun Oct 10 20:01:43 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 10 Oct 2010 20:01:43 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: <19633.63652.923839.994702@pilspetsen.it.uu.se> References: <19633.63652.923839.994702@pilspetsen.it.uu.se> Message-ID: <042BA426-8235-4724-8781-78B65C5F80A2@kallisys.net> Le 10 oct. 2010 ? 19:32, Mikael Pettersson a ?crit : > Looking at the patch via github's awful web interface it looks like > it will cause the 'is_recursive' variable to be conditionally defined > and possibly used uninitialized (at least as far as GCC will be able > to tell). Thus, NAK for now. > > I would have provided the fix but you didn't include the patch in the > email so I can't. Mikael, Thank you for your e-mail and review. While the is_recursive variable will currently not be used uninitialized (if stack is emptied by erts_garbage_collect_hibernate, status is set to P_WAITING), I agree this is a mistake and I have amended the patch accordingly. It is now available here: http://github.com/pguyot/otp/commit/75e39faaa3180a78e8dab8d7bd4bef066dae579b Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From mikpe@REDACTED Sun Oct 10 23:25:51 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Sun, 10 Oct 2010 23:25:51 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: <042BA426-8235-4724-8781-78B65C5F80A2@kallisys.net> References: <19633.63652.923839.994702@pilspetsen.it.uu.se> <042BA426-8235-4724-8781-78B65C5F80A2@kallisys.net> Message-ID: <19634.12127.702407.423294@pilspetsen.it.uu.se> Paul Guyot writes: > > Le 10 oct. 2010 ? 19:32, Mikael Pettersson a ?crit : > > > Looking at the patch via github's awful web interface it looks like > > it will cause the 'is_recursive' variable to be conditionally defined > > and possibly used uninitialized (at least as far as GCC will be able > > to tell). Thus, NAK for now. > > > > I would have provided the fix but you didn't include the patch in the > > email so I can't. > > Mikael, > > Thank you for your e-mail and review. While the is_recursive variable will currently not be used uninitialized (if stack is emptied by erts_garbage_collect_hibernate, status is set to P_WAITING), I agree this is a mistake and I have amended the patch accordingly. It is now available here: > > http://github.com/pguyot/otp/commit/75e39faaa3180a78e8dab8d7bd4bef066dae579b This version is Ok. From bgustavsson@REDACTED Mon Oct 11 13:53:24 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 13:53:24 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: References: Message-ID: On Sun, Oct 10, 2010 at 3:16 PM, Paul Guyot wrote: > Or just fetch the two commits at once: > git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch > Thanks! I have combined the two commits to one (since the second commit fixes a bug introduced in the first commit). I also fixed the indentation in hipe_empty_nstack(). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 11 14:04:23 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 14:04:23 +0200 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: On Sun, Oct 10, 2010 at 7:02 PM, Tuncer Ayaz wrote: > --- > lib/stdlib/doc/src/lists.xml | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml > index 92c4eb4..0cf88fe 100644 > --- a/lib/stdlib/doc/src/lists.xml > +++ b/lib/stdlib/doc/src/lists.xml > @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> > > > > - foreach(Fun, List) -> void() > + foreach(Fun, List) -> ok > Apply a function to each element of a list > > - Fun = fun(Elem) -> void() > + Fun = fun(Elem) -> term() >  Elem = term() > List = [term()] > > void() is a convention used in the documentation to mean that the function has no useful return value and should be ignored. It is used in many places in the documentation. Try: git grep 'void()' | grep '[.]xml' -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 11 14:09:58 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 14:09:58 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/9 Tuncer Ayaz : > Please refetch updated patch with sendfile/3 for chunked > send and minor cleanups: > git fetch git://github.com/tuncer/otp.git sendfile > Thanks! Refetched. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Mon Oct 11 14:30:19 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 11 Oct 2010 14:30:19 +0200 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: 2010/10/11 Bj?rn Gustavsson: > On Sun, Oct 10, 2010 at 7:02 PM, Tuncer Ayaz wrote: >> --- >> ?lib/stdlib/doc/src/lists.xml | ? ?4 ++-- >> ?1 files changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml >> index 92c4eb4..0cf88fe 100644 >> --- a/lib/stdlib/doc/src/lists.xml >> +++ b/lib/stdlib/doc/src/lists.xml >> @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> >> ? ? ? >> ? ? >> ? ? >> - ? ? ?foreach(Fun, List) -> void() >> + ? ? ?foreach(Fun, List) -> ok >> ? ? ? Apply a function to each element of a list >> ? ? ? >> - ? ? ? ?Fun = fun(Elem) -> void() >> + ? ? ? ?Fun = fun(Elem) -> term() >> ? ? ? ?  Elem = term() >> ? ? ? ? List = [term()] >> ? ? ? >> > > void() is a convention used in the documentation to mean > that the function has no useful return value and should be > ignored. It is used in many places in the documentation. Both changes or only the 2nd (fun(Elem)) changed line? > Try: > > git grep 'void()' | grep '[.]xml' Is it wrong to rely on foreach/2 returning 'ok' even though the spec in the sources specifies that? From bgustavsson@REDACTED Mon Oct 11 14:55:25 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 14:55:25 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: References: Message-ID: 2010/10/11 Bj?rn Gustavsson : > On Sun, Oct 10, 2010 at 3:16 PM, Paul Guyot wrote: > >> Or just fetch the two commits at once: >> git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch >> > > Thanks! I have combined the two commits to one (since the > second commit fixes a bug introduced in the first commit). > > I also fixed the indentation in hipe_empty_nstack(). There is a compilation warning: beam/bif.c: In function ?hibernate_3?: beam/bif.c:1105: warning: assignment makes integer from pointer without a cast It's the following line: BIF_P->def_arg_reg[3] = BIF_P->i; -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kostis@REDACTED Mon Oct 11 15:04:18 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 11 Oct 2010 16:04:18 +0300 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: <4CB30B52.5060904@cs.ntua.gr> Bj?rn Gustavsson wrote: > On Sun, Oct 10, 2010 at 7:02 PM, Tuncer Ayaz wrote: >> --- >> lib/stdlib/doc/src/lists.xml | 4 ++-- >> 1 files changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml >> index 92c4eb4..0cf88fe 100644 >> --- a/lib/stdlib/doc/src/lists.xml >> +++ b/lib/stdlib/doc/src/lists.xml >> @@ -297,10 +297,10 @@ flatmap(Fun, List1) -> >> >> >> >> - foreach(Fun, List) -> void() >> + foreach(Fun, List) -> ok >> Apply a function to each element of a list >> >> - Fun = fun(Elem) -> void() >> + Fun = fun(Elem) -> term() >>  Elem = term() >> List = [term()] >> >> > > void() is a convention used in the documentation to mean > that the function has no useful return value and should be > ignored. It is used in many places in the documentation. > Try: > > git grep 'void()' | grep '[.]xml' Convention or not, the fact remains that a void() type does not exist in Erlang. The problem is that even if one is careful enough not to match against the result of the lists:foreach/2 call, the return from this function can propagate higher up in the function call chain. foo1() -> ... gazonk = foo2(...), % this one will fail here .... foo2(...) -> ... foo3(...). ... fooN_1(...) -> ... fooN(...). fooN() -> ... lists:foreach(...). It would be a much better world if all void() functions were made to return some consistent value such as 'ok'. Kostis From bgustavsson@REDACTED Mon Oct 11 15:22:28 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 15:22:28 +0200 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: <4CB30B52.5060904@cs.ntua.gr> References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> <4CB30B52.5060904@cs.ntua.gr> Message-ID: On Mon, Oct 11, 2010 at 3:04 PM, Kostis Sagonas wrote: > > It would be a much better world if all void() functions were made to return > some consistent value such as 'ok'. > I will accept for 'pu' a patch that changes all occurrences of void() to 'ok' (or a patch that only keeps void() for funs and callback function to indicate that the return value is not important). However, I think that the decision to graduate such a branch should be taken by the OTP Technical Board. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Mon Oct 11 15:26:12 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 15:26:12 +0200 Subject: [erlang-patches] prim_file:exists/{1,2} a new lightweight file existence check In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 1:52 AM, Blaine Whittle wrote: > This function is meant to be faster / simpler check for the existence > of a file or directory. Most all erlang functions that do this end > up calling prim_file:read_file_info at some point. When > read_file_info is used as a simple existence check, time is wasted > collecting file stats data only to turn around and throw the data > away. If a file exists, read_file_info can take three times longer > to return than when the file doesn't exist. Return values are true, > false, or {error, eisdir}. The low level implementation of this > function was already defined via efile_may_openfile, this patch just > exposes it to prim_file. filelib:is_dir/1, filelib:is_file/1, > filelib:is_regular/1 could be changed from calling read_file_info to > prim_file:exists or better yet to calling to a yet unwritten > file:exists function. > > Patch located at > git fetch git://github.com/bwhittle/otp.git efile_exists > Thanks! I will include it in 'pu'. Two source files had execution permission set in your commit. I have fixed that. I have also removed the commit of prim_file.beam for practical reasons (I have put the update in a separate commit in the 'pu' branch). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Mon Oct 11 15:34:23 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 11 Oct 2010 15:34:23 +0200 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> <4CB30B52.5060904@cs.ntua.gr> Message-ID: 2010/10/11 Bj?rn Gustavsson: > On Mon, Oct 11, 2010 at 3:04 PM, Kostis Sagonas wrote: >> >> It would be a much better world if all void() functions were made to return >> some consistent value such as 'ok'. >> > > I will accept for 'pu' a patch that changes all > occurrences of void() to 'ok' (or a patch that > only keeps void() for funs and callback function > to indicate that the return value is not important). > However, I think that the decision to graduate > such a branch should be taken by the OTP > Technical Board. I have to re-ask: Is it wrong to match ok=foreach/2 even though the spec says that and it's what the current implementation does? From bgustavsson@REDACTED Mon Oct 11 15:47:40 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Oct 2010 15:47:40 +0200 Subject: [erlang-patches] [PATCH] Fix lists:foreach/2's documented spec In-Reply-To: References: <1286730130-15496-1-git-send-email-tuncer.ayaz@gmail.com> <4CB30B52.5060904@cs.ntua.gr> Message-ID: 2010/10/11 Tuncer Ayaz : > > I have to re-ask: > Is it wrong to match ok=foreach/2 even though the spec says > that and it's what the current implementation does? > In practice, no, since we have no intention to change the return value. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From rdoering@REDACTED Tue Oct 12 11:29:26 2010 From: rdoering@REDACTED (Ralf Doering) Date: Tue, 12 Oct 2010 11:29:26 +0200 Subject: [PATCH] Update hint where to find epmd protocol spec in epmd_srv.c. Message-ID: <1286875766-21085-1-git-send-email-rdoering@netsys-it.de> With newer releases erts/emulator/internal_doc/erl_ext_dist.txt just forwards the reader to the ERTS User's Guide. Adjust the comment in epmd_srv.c to point to this location too. Signed-off-by: Ralf Doering --- erts/epmd/src/epmd_srv.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/erts/epmd/src/epmd_srv.c b/erts/epmd/src/epmd_srv.c index df4d1a5..ef471a4 100644 --- a/erts/epmd/src/epmd_srv.c +++ b/erts/epmd/src/epmd_srv.c @@ -39,8 +39,10 @@ * server keeps the socket open where the request for registration was * made. * - * The protocol is briefly documented in "erl_ext_dist.txt". All requests - * to this server are done with a packet + * The protocol is briefly documented in the ERTS User's Guide, see + * http://www.erlang.org/doc/apps/erts/erl_dist_protocol.html + * + * All requests to this server are done with a packet * * 2 n * +--------+---------+ -- 1.7.3.15.g442cb From kostis@REDACTED Tue Oct 12 13:48:58 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 12 Oct 2010 14:48:58 +0300 Subject: Add missing types to file:open/2 mode() Message-ID: <4CB44B2A.7020401@cs.ntua.gr> Add the {encoding, _} options to file:open/2's modes Although the {encoding, encoding()} set of options is documented in the manual page of the 'file' module, they do not appear in the Mode description on that page nor in the mode() type declaration. The patch adds this information in both the code of the module and the documentation of the module. To avoid duplication, the declaration of the encoding() type is added to the 'unicode' module where it most probably belongs. While at it, added a proper declaration for posix(), took out the now superfluous information about the types of file:open/2 from the 'erl_bif_types' module, and corrected the return type of file:open/2 so that it corresponds to its published documentation. git fetch git://github.com/kostis/otp.git file_open-mode Kostis PS. Among other things, this corrects some erroneous dialyzer warning reported by Alexey Romanov. Thanks! From fdmanana@REDACTED Tue Oct 12 22:10:38 2010 From: fdmanana@REDACTED (Filipe David Manana) Date: Tue, 12 Oct 2010 21:10:38 +0100 Subject: [erlang-patches] [PATCH] httpc module - allow streaming of bodies for PUT and POST requests In-Reply-To: References: Message-ID: Hi, In the previous patch I forgot to add the documentation for the new chunkify function body parameter. I updated my git branch with it: git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming thanks 2010/10/5 Bj?rn Gustavsson : > 2010/10/5 Filipe David Manana : >> Hi, >> >> I just made another change to that patch so that it allows as well for >> chunking (chunked transfer-encoding) the results of the streaming >> function. This is specially useful when the client doesn't know in >> advance the length of the payload. >> >> git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming >> > > Thanks! I will include in pu with minor clean-ups of the commit messages. > Next time, please review our guidelines for commit message: > > http://github.com/erlang/otp/wiki/Writing-good-commit-messages > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-patches (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED > > -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From tuncer.ayaz@REDACTED Wed Oct 13 13:48:02 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 13 Oct 2010 13:48:02 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/11 Bj?rn Gustavsson: > 2010/10/9 Tuncer Ayaz: >> Please refetch updated patch with sendfile/3 for chunked >> send and minor cleanups: >> git fetch git://github.com/tuncer/otp.git sendfile >> > > Thanks! Refetched. Please refetch for cleanups and comments. From bgustavsson@REDACTED Wed Oct 13 14:41:27 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 13 Oct 2010 14:41:27 +0200 Subject: [erlang-patches] [PATCH] Update hint where to find epmd protocol spec in epmd_srv.c. In-Reply-To: <1286875766-21085-1-git-send-email-rdoering@netsys-it.de> References: <1286875766-21085-1-git-send-email-rdoering@netsys-it.de> Message-ID: On Tue, Oct 12, 2010 at 11:29 AM, Ralf Doering wrote: > With newer releases erts/emulator/internal_doc/erl_ext_dist.txt just > forwards the reader to the ERTS User's Guide. Adjust the comment in > epmd_srv.c to point to this location too. > Thanks! Will apply directly to 'dev'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 13 14:44:07 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 13 Oct 2010 14:44:07 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/13 Tuncer Ayaz : > Please refetch for cleanups and comments. Refetched. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 13 14:47:53 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 13 Oct 2010 14:47:53 +0200 Subject: [erlang-patches] Add missing types to file:open/2 mode() In-Reply-To: <4CB44B2A.7020401@cs.ntua.gr> References: <4CB44B2A.7020401@cs.ntua.gr> Message-ID: 2010/10/12 Kostis Sagonas : > > git fetch git://github.com/kostis/otp.git file_open-mode Thanks! Will include in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 13 14:59:22 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 13 Oct 2010 14:59:22 +0200 Subject: [erlang-patches] [PATCH] httpc module - allow streaming of bodies for PUT and POST requests In-Reply-To: References: Message-ID: 2010/10/12 Filipe David Manana : > Hi, > > In the previous patch I forgot to add the documentation for the new > chunkify function body parameter. > I updated my git branch with it: > > git fetch git://github.com/fdmanana/otp.git httpc_upload_body_streaming > Thanks! Updated. I combined your new commit with the previous commit. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kostis@REDACTED Thu Oct 14 14:11:00 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 14 Oct 2010 15:11:00 +0300 Subject: Deeper unfolding of recursive types Message-ID: <4CB6F354.7060607@cs.ntua.gr> git fetch git://github.com/kostis/otp.git rec-erl_types Kostis From bgustavsson@REDACTED Thu Oct 14 15:59:05 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 14 Oct 2010 15:59:05 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/13 Bj?rn Gustavsson : > 2010/10/13 Tuncer Ayaz : >> Please refetch for cleanups and comments. > > Refetched. > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > There are some problems with the specs (they were there before the latest update too): ./file.erl:293: file.erl:293: The call file:sendfile(File::pid(),SockFD::string() | non_neg_integer(),Offset::any(),Bytes::any()) breaks the contract (File::io_device() | fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) -> {'ok',non_neg_integer()} | {'error',posix()} ./file.erl:305: file.erl:305: The specification for file:sendfile/3 states that the function might also return {'error',atom()} | {'ok',non_neg_integer()} but the inferred return is none() ./file.erl:312: file.erl:312: The call file:sendfile_chunked_loop(Fd::pid() | #file_descriptor{module::atom()},SockFD::string() | non_neg_integer(),Offset::0,Bytes::'undefined' | non_neg_integer(),Bytes::'undefined' | non_neg_integer(),ChunkSize::any()) will never return since it differs in the 2nd argument from the success typing arguments: (pid() | #file_descriptor{module::atom() | tuple()},port(),non_neg_integer(),any(),non_neg_integer(),non_neg_integer()) ./file.erl:323: file.erl:323: Invalid type specification for function file:sendfile_chunked_loop/6. The success typing is (pid() | #file_descriptor{module::atom() | tuple()},port(),non_neg_integer(),_,non_neg_integer(),non_neg_integer()) -> {'error',atom()} | {'ok',_} ./file.erl:331: file.erl:331: The call file:sendfile(Fd::pid() | #file_descriptor{module::atom()},SockFD::string() | non_neg_integer(),Offset0::0,ToWrite::non_neg_integer()) breaks the contract (File::io_device() | fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) -> {'ok',non_neg_integer()} | {'error',posix()} ./file.erl:344: file.erl:344: Invalid type specification for function file:sendfile/2. The success typing is (atom() | [atom() | [any()] | char()],port()) -> {'error',atom()} | {'ok',non_neg_integer()} -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Thu Oct 14 16:26:36 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 14 Oct 2010 16:26:36 +0200 Subject: [erlang-patches] Deeper unfolding of recursive types In-Reply-To: <4CB6F354.7060607@cs.ntua.gr> References: <4CB6F354.7060607@cs.ntua.gr> Message-ID: 2010/10/14 Kostis Sagonas : > > git fetch git://github.com/kostis/otp.git rec-erl_types Thanks! Included in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tuncer.ayaz@REDACTED Fri Oct 15 00:04:36 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 15 Oct 2010 00:04:36 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/14 Bj?rn Gustavsson: > 2010/10/13 Bj?rn Gustavsson: >> 2010/10/13 Tuncer Ayaz: >>> Please refetch for cleanups and comments. >> >> Refetched. > > There are some problems with the specs > (they were there before the latest update > too): Corrected the obvious discrepancies in commit c9b2d0a. Also changed file:F/A calls to be local F/A calls. Feel free to refetch until I fix the remaining issues. > ./file.erl:293: file.erl:293: The call > file:sendfile(File::pid(),SockFD::string() | > non_neg_integer(),Offset::any(),Bytes::any()) breaks the contract > (File::io_device() | > fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) > -> {'ok',non_neg_integer()} | {'error',posix()} > > ./file.erl:305: file.erl:305: The specification for file:sendfile/3 > states that the function might also return {'error',atom()} | > {'ok',non_neg_integer()} but the inferred return is none() > > ./file.erl:312: file.erl:312: The call > file:sendfile_chunked_loop(Fd::pid() | > #file_descriptor{module::atom()},SockFD::string() | > non_neg_integer(),Offset::0,Bytes::'undefined' | > non_neg_integer(),Bytes::'undefined' | > non_neg_integer(),ChunkSize::any()) will never return since it differs > in the 2nd argument from the success typing arguments: (pid() | > #file_descriptor{module::atom() | > tuple()},port(),non_neg_integer(),any(),non_neg_integer(),non_neg_integer()) > > ./file.erl:323: file.erl:323: Invalid type specification for function > file:sendfile_chunked_loop/6. The success typing is (pid() | > #file_descriptor{module::atom() | > tuple()},port(),non_neg_integer(),_,non_neg_integer(),non_neg_integer()) > -> {'error',atom()} | {'ok',_} > > ./file.erl:331: file.erl:331: The call file:sendfile(Fd::pid() | > #file_descriptor{module::atom()},SockFD::string() | > non_neg_integer(),Offset0::0,ToWrite::non_neg_integer()) breaks the > contract (File::io_device() | > fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) > -> {'ok',non_neg_integer()} | {'error',posix()} > > ./file.erl:344: file.erl:344: Invalid type specification for function > file:sendfile/2. The success typing is (atom() | [atom() | [any()] | > char()],port()) -> {'error',atom()} | {'ok',non_neg_integer()} From tuncer.ayaz@REDACTED Fri Oct 15 00:42:58 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 15 Oct 2010 00:42:58 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: 2010/10/14 Bj?rn Gustavsson: > 2010/10/13 Bj?rn Gustavsson: >> 2010/10/13 Tuncer Ayaz: >>> Please refetch for cleanups and comments. >> >> Refetched. >> >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB >> > > There are some problems with the specs > (they were there before the latest update > too): Corrected all specs in 296ee42. Please refetch. > ./file.erl:293: file.erl:293: The call > file:sendfile(File::pid(),SockFD::string() | > non_neg_integer(),Offset::any(),Bytes::any()) breaks the contract > (File::io_device() | > fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) > -> {'ok',non_neg_integer()} | {'error',posix()} > > ./file.erl:305: file.erl:305: The specification for file:sendfile/3 > states that the function might also return {'error',atom()} | > {'ok',non_neg_integer()} but the inferred return is none() > > ./file.erl:312: file.erl:312: The call > file:sendfile_chunked_loop(Fd::pid() | > #file_descriptor{module::atom()},SockFD::string() | > non_neg_integer(),Offset::0,Bytes::'undefined' | > non_neg_integer(),Bytes::'undefined' | > non_neg_integer(),ChunkSize::any()) will never return since it differs > in the 2nd argument from the success typing arguments: (pid() | > #file_descriptor{module::atom() | > tuple()},port(),non_neg_integer(),any(),non_neg_integer(),non_neg_integer()) > > ./file.erl:323: file.erl:323: Invalid type specification for function > file:sendfile_chunked_loop/6. The success typing is (pid() | > #file_descriptor{module::atom() | > tuple()},port(),non_neg_integer(),_,non_neg_integer(),non_neg_integer()) > -> {'error',atom()} | {'ok',_} > > ./file.erl:331: file.erl:331: The call file:sendfile(Fd::pid() | > #file_descriptor{module::atom()},SockFD::string() | > non_neg_integer(),Offset0::0,ToWrite::non_neg_integer()) breaks the > contract (File::io_device() | > fd(),Sock::port(),Offset::non_neg_integer(),Bytes::non_neg_integer()) > -> {'ok',non_neg_integer()} | {'error',posix()} > > ./file.erl:344: file.erl:344: Invalid type specification for function > file:sendfile/2. The success typing is (atom() | [atom() | [any()] | > char()],port()) -> {'error',atom()} | {'ok',non_neg_integer()} From kostis@REDACTED Fri Oct 15 19:29:01 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 15 Oct 2010 20:29:01 +0300 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: <4CB853A1.3070303@cs.ntua.gr> References: <1287122265.5114.0.camel@seasc1137> <4CB853A1.3070303@cs.ntua.gr> Message-ID: <4CB88F5D.9070703@cs.ntua.gr> Kostis Sagonas wrote: > Dan Kelley wrote: >> I can't, but this minimal reproducer shows the problem: >> >> ----- >> -module(example). >> -export([bar/1]). >> >> bar (Baz) -> >> binary_part(Baz, 0, 1). > > .... SNIP > > and there is no case for BIFs with more than two arguments. I will look > into this. This can be fixed by the patch that appears in: http://github.com/kostis/otp/commit/c0933e3d8158a2e783548070e4edd2f0bbacd6bd the repository is: http://github.com/kostis/otp/tree/hipe-fixes Since this is very simple I trust that somebody from the OTP team will include it directly to 'dev'. Kostis From pguyot@REDACTED Sat Oct 16 19:14:18 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 16 Oct 2010 19:14:18 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: References: Message-ID: Le 11 oct. 2010 ? 14:55, Bj?rn Gustavsson a ?crit : > There is a compilation warning: > > beam/bif.c: In function ?hibernate_3?: > beam/bif.c:1105: warning: assignment makes integer from pointer without a cast > > It's the following line: > > BIF_P->def_arg_reg[3] = BIF_P->i; Thank you for your e-mail. I have fixed this warning by using a proper cast. Since such casts are normally found in macros defined in bif.h, I added a new macro for hibernate_3 (that just sets BIF_P->i and leaves arity untouched), so all TRAP-related code is in a single place. I believe this should ease halfword emulator work. The new (combined) commit is here: http://github.com/pguyot/otp/commit/aa60538c7f68b4c657d8ead45ffb4e44164f9561 git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch Also, I noticed in the "Currently in the 'pu' branch" page (http://erlang.github.com/otp/) that there is "1 line with whitespace errors.". While I guess what a line with whitespace errors could be, how can I figure out which line it is ? Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From pguyot@REDACTED Sun Oct 17 09:43:25 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 17 Oct 2010 09:43:25 +0200 Subject: DES and DES3 in CFB mode Message-ID: <1335BC38-0B14-44DA-B8AF-52B47A846483@kallisys.net> Hello, crypto application exposes DES and Triple DES in ECB and CBC mode, but not in CFB mode which is used in some protocols (Nagios' NSCA protocol for example). The following patch adds support for CFB mode: git fetch git://github.com/pguyot/otp.git des-cfb-functions It also fixes a typo in the crypto documentation and adds a test case for Triple DES in CBC mode. Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From bgustavsson@REDACTED Wed Oct 20 16:00:41 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 20 Oct 2010 16:00:41 +0200 Subject: [erlang-patches] sendfile() In-Reply-To: References: Message-ID: Refetched. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 20 16:05:50 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 20 Oct 2010 16:05:50 +0200 Subject: [erlang-patches] Fix hibernate with HiPE In-Reply-To: References: Message-ID: 2010/10/16 Paul Guyot : > > Le 11 oct. 2010 ? 14:55, Bj?rn Gustavsson a ?crit : > >> There is a compilation warning: >> >> beam/bif.c: In function ?hibernate_3?: >> beam/bif.c:1105: warning: assignment makes integer from pointer without a cast >> >> It's the following line: >> >> BIF_P->def_arg_reg[3] = BIF_P->i; > > Thank you for your e-mail. I have fixed this warning by using a proper cast. Since such casts are normally found in macros defined in bif.h, I added a new macro for hibernate_3 (that just sets BIF_P->i and leaves arity untouched), so all TRAP-related code is in a single place. I believe this should ease halfword emulator work. > > The new (combined) commit is here: > http://github.com/pguyot/otp/commit/aa60538c7f68b4c657d8ead45ffb4e44164f9561 > git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe branch Thanks! Updated. > Also, I noticed in the "Currently in the 'pu' branch" page (http://erlang.github.com/otp/) that there is "1 line with whitespace errors.". While I guess what a line with whitespace errors could be, how can I figure out which line it is ? git diff --check HEAD^ (To check for whitespace errors in one commit.) To fix: git rebase --whitespace=fix HEAD^ -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Wed Oct 20 16:07:55 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 20 Oct 2010 16:07:55 +0200 Subject: [erlang-patches] DES and DES3 in CFB mode In-Reply-To: <1335BC38-0B14-44DA-B8AF-52B47A846483@kallisys.net> References: <1335BC38-0B14-44DA-B8AF-52B47A846483@kallisys.net> Message-ID: On Sun, Oct 17, 2010 at 9:43 AM, Paul Guyot wrote: > > git fetch git://github.com/pguyot/otp.git des-cfb-functions Thanks! Will include in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From fritchie@REDACTED Fri Oct 22 23:18:52 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 22 Oct 2010 16:18:52 -0500 Subject: Patch for configurable ERTS_DE_BUSY_LIMIT In-Reply-To: Message of "Thu, 12 Aug 2010 17:38:01 +0200." <4C641559.5030100@erlang.org> Message-ID: <56931.1287782332@snookles.snookles.com> Rickard Green wrote: rg> I've had a look at your patch, and it needs some improvements before rg> we can accept it. Rickard, after sitting on the shelf for quite a while, please see the erts_de_busy_limit_take2 branch at git://github.com/slfritchie/otp.git. -Scott From tuncer.ayaz@REDACTED Sat Oct 23 00:13:24 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 23 Oct 2010 00:13:24 +0200 Subject: [PATCH] Add missing Dialyzer warning options to doc Message-ID: <1287785604-52515-1-git-send-email-tuncer.ayaz@gmail.com> -Wno_opaque, -Wrace_conditions and -Wbehaviours are only documented in the Dialyzer command line help text. Add them to the documentation. --- lib/dialyzer/doc/src/dialyzer.xml | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index 1ec2ce8..b977a44 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -158,9 +158,16 @@ Suppress warnings for patterns that are unused or cannot match. + + Suppress warnings for violations of opaqueness of data types. *** Include warnings for functions that only return by means of an exception. + *** + Include warnings for possible race conditions. + *** + Include warnings about behaviour callbacks which drift from the + published recommended interfaces. *** Include warnings for function calls which ignore a structured return value or do not match against one of many possible return value(s). @@ -215,8 +222,11 @@ WarnOpts : no_return | no_improper_lists | no_fun_app | no_match + | no_opaque | no_fail_call | error_handling + | race_conditions + | behaviours | unmatched_returns | overspecs | underspecs From michael.santos@REDACTED Sun Oct 24 14:37:27 2010 From: michael.santos@REDACTED (Michael Santos) Date: Sun, 24 Oct 2010 08:37:27 -0400 Subject: [PATCH] Fix crash in string:copies/2 Message-ID: <20101024123727.GA936@ecn.lan> Using a float for the number of copies results in an infinite loop. Check the argument is an integer. Reported-By: Eric Pailleau --- lib/stdlib/src/string.erl | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/lib/stdlib/src/string.erl b/lib/stdlib/src/string.erl index 6636a03..c987c22 100644 --- a/lib/stdlib/src/string.erl +++ b/lib/stdlib/src/string.erl @@ -201,7 +201,7 @@ chars(C, 0, Tail) when is_integer(C) -> -spec copies(string(), non_neg_integer()) -> string(). -copies(CharList, Num) when is_list(CharList), Num >= 0 -> +copies(CharList, Num) when is_list(CharList), is_integer(Num), Num >= 0 -> copies(CharList, Num, []). copies(_CharList, 0, R) -> -- 1.7.0.4 From michael.santos@REDACTED Sun Oct 24 15:09:19 2010 From: michael.santos@REDACTED (Michael Santos) Date: Sun, 24 Oct 2010 09:09:19 -0400 Subject: [PATCH] ei: check the length of the node name Message-ID: <20101024130919.GA1040@ecn.lan> Check the length of the node name to prevent an overflow. --- lib/erl_interface/src/epmd/epmd_publish.c | 6 ++++++ lib/erl_interface/src/epmd/epmd_unpublish.c | 5 +++++ 2 files changed, 11 insertions(+), 0 deletions(-) diff --git a/lib/erl_interface/src/epmd/epmd_publish.c b/lib/erl_interface/src/epmd/epmd_publish.c index a9b8727..d45fe64 100644 --- a/lib/erl_interface/src/epmd/epmd_publish.c +++ b/lib/erl_interface/src/epmd/epmd_publish.c @@ -69,6 +69,12 @@ static int ei_epmd_r4_publish (int port, const char *alive, unsigned ms) int n; int res, creation; + if (len > sizeof(buf)-2) + { + erl_errno = ERANGE; + return -1; + } + s = buf; put16be(s,len); diff --git a/lib/erl_interface/src/epmd/epmd_unpublish.c b/lib/erl_interface/src/epmd/epmd_unpublish.c index 08662fe..495cbab 100644 --- a/lib/erl_interface/src/epmd/epmd_unpublish.c +++ b/lib/erl_interface/src/epmd/epmd_unpublish.c @@ -59,6 +59,11 @@ int ei_unpublish_tmo(const char *alive, unsigned ms) int len = 1 + strlen(alive); int fd, res; + if (len > sizeof(buf)-3) { + erl_errno = ERANGE; + return -1; + } + put16be(s,len); put8(s,EI_EPMD_STOP_REQ); strcpy(s, alive); -- 1.7.0.4 From kostis@REDACTED Mon Oct 25 16:41:56 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 25 Oct 2010 17:41:56 +0300 Subject: compiler: Strengthen some specs Message-ID: <4CC59734.5020308@cs.ntua.gr> Strengthen some specs to shut off dialyzer warnings git fetch git://github.com/kostis/otp.git cerl-specs Kostis From ulf.wiger@REDACTED Mon Oct 25 17:36:16 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Oct 2010 17:36:16 +0200 Subject: Track parents when expanding #xmlElement records Message-ID: git fetch git://github.com/uwiger/otp.git xmerl-expand-simple Track parents when expanding #xmlElement records BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From kostis@REDACTED Tue Oct 26 11:40:55 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 26 Oct 2010 12:40:55 +0300 Subject: stdlib: Export ordset/1 type and strengthen specs Message-ID: <4CC6A227.2070807@cs.ntua.gr> git fetch git://github.com/kostis/otp.git ordsets-types Kostis From kostis@REDACTED Tue Oct 26 12:21:36 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 26 Oct 2010 13:21:36 +0300 Subject: syntax_tools: Fix an erroneous type and strengthen some specs Message-ID: <4CC6ABB0.8090908@cs.ntua.gr> The declaration of the stubDescriptop() type in 'igor' was erroneous, both in the -type and in the published documentation of the module. While fixing this some specs where strengthened and used a remote type to refer to ordsets:ordset(T). Consequently, this patch depends on the ordsets module exporting the ordset/1 type. git fetch git://github.com/kostis/otp.git syntax_tools-fixes Kostis From bgustavsson@REDACTED Tue Oct 26 15:15:10 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 15:15:10 +0200 Subject: [erlang-patches] Re: Patch for configurable ERTS_DE_BUSY_LIMIT In-Reply-To: <56931.1287782332@snookles.snookles.com> References: <4C641559.5030100@erlang.org> <56931.1287782332@snookles.snookles.com> Message-ID: On Fri, Oct 22, 2010 at 11:18 PM, Scott Lystig Fritchie wrote: > Rickard Green wrote: > > rg> I've had a look at your patch, and it needs some improvements before > rg> we can accept it. > > Rickard, after sitting on the shelf for quite a while, please see the > erts_de_busy_limit_take2 branch at git://github.com/slfritchie/otp.git. > I will combine your commits to one commit (because the two last commits are only corrections to the first commit) and update the branch in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 26 15:39:36 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 15:39:36 +0200 Subject: [erlang-patches] [PATCH] Add missing Dialyzer warning options to doc In-Reply-To: <1287785604-52515-1-git-send-email-tuncer.ayaz@gmail.com> References: <1287785604-52515-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: On Sat, Oct 23, 2010 at 12:13 AM, Tuncer Ayaz wrote: > -Wno_opaque, -Wrace_conditions and -Wbehaviours are only documented > in the Dialyzer command line help text. Add them to the documentation. Thanks! Will include in 'pu'. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 26 15:59:09 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 15:59:09 +0200 Subject: [erlang-patches] [PATCH] Fix crash in string:copies/2 In-Reply-To: <20101024123727.GA936@ecn.lan> References: <20101024123727.GA936@ecn.lan> Message-ID: On Sun, Oct 24, 2010 at 2:37 PM, Michael Santos wrote: > Using a float for the number of copies results in an infinite loop. > Check the argument is an integer. > > Reported-By: Eric Pailleau Thanks! Will in include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 26 16:01:23 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 16:01:23 +0200 Subject: [erlang-patches] [PATCH] ei: check the length of the node name In-Reply-To: <20101024130919.GA1040@ecn.lan> References: <20101024130919.GA1040@ecn.lan> Message-ID: On Sun, Oct 24, 2010 at 3:09 PM, Michael Santos wrote: > Check the length of the node name to prevent an overflow. Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 26 16:14:33 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 16:14:33 +0200 Subject: [erlang-patches] syntax_tools: Fix an erroneous type and strengthen some specs In-Reply-To: <4CC6ABB0.8090908@cs.ntua.gr> References: <4CC6ABB0.8090908@cs.ntua.gr> Message-ID: 2010/10/26 Kostis Sagonas : > The declaration of the stubDescriptop() type in 'igor' was erroneous, > both in the -type and in the published documentation of the module. > > While fixing this some specs where strengthened and used a remote > type to refer to ordsets:ordset(T). Consequently, this patch depends > on the ordsets module exporting the ordset/1 type. > Because of this dependency, I have put this commit on the same branch as the commit that exports the ordset/1 type. I also put the other commits that strengthen specs onto the same branch (still as separate commits). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Tue Oct 26 16:16:58 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 26 Oct 2010 16:16:58 +0200 Subject: [erlang-patches] Track parents when expanding #xmlElement records In-Reply-To: References: Message-ID: On Mon, Oct 25, 2010 at 5:36 PM, Ulf Wiger wrote: > > git fetch git://github.com/uwiger/otp.git xmerl-expand-simple > Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From matthias@REDACTED Thu Oct 28 11:22:48 2010 From: matthias@REDACTED (Matthias Lang) Date: Thu, 28 Oct 2010 11:22:48 +0200 Subject: improve the 're' manpage by correcting typos and rewording Message-ID: <20101028092248.GA5365@corelatus.se> Hi, git fetch git://github.com/matthiasl/otp.git re-manpage-cleanup I took a couple of passes through the manpage and fixed all the typos I saw. I also shortened some sentences. (Branch looks wonky because I just learned that 'git commit --amend' doesn't work if you've already done a 'git push'. So I merged and pushed again.) Matt From michael.santos@REDACTED Fri Oct 29 02:45:09 2010 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 28 Oct 2010 20:45:09 -0400 Subject: [PATCH 1/2] ei: error if return value of ei_alloc_big is NULL Message-ID: <20101029004509.GA17650@ecn.lan> --- lib/erl_interface/src/decode/decode_big.c | 2 +- lib/erl_interface/src/legacy/erl_marshal.c | 7 +++++-- lib/erl_interface/src/misc/ei_printterm.c | 3 ++- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/erl_interface/src/decode/decode_big.c b/lib/erl_interface/src/decode/decode_big.c index efe9c6e..b5e9b45 100644 --- a/lib/erl_interface/src/decode/decode_big.c +++ b/lib/erl_interface/src/decode/decode_big.c @@ -74,7 +74,7 @@ erlang_big *ei_alloc_big(unsigned int digit_bytes) { memset(b,(char)0,sizeof(erlang_big)); if ( (b->digits = malloc(2*n)) == NULL) { free(b); - return 0; + return NULL; } b->arity = digit_bytes; diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index 18315bf..5084c65 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -1890,8 +1890,11 @@ static int cmp_big_big(unsigned char**e1, unsigned char **e2) ei_get_type((char *)*e1,&i1,&t1,&n1); ei_get_type((char *)*e2,&i2,&t2,&n2); - b1 = ei_alloc_big(n1); - b2 = ei_alloc_big(n2); + if ( (b1 = ei_alloc_big(n1)) == NULL) return -1; + if ( (b2 = ei_alloc_big(n2)) == NULL) { + ei_free_big(b1); + return 1; + } ei_decode_big((char *)*e1,&i1,b1); ei_decode_big((char *)*e2,&i2,b2); diff --git a/lib/erl_interface/src/misc/ei_printterm.c b/lib/erl_interface/src/misc/ei_printterm.c index 98473f7..5fc6b35 100644 --- a/lib/erl_interface/src/misc/ei_printterm.c +++ b/lib/erl_interface/src/misc/ei_printterm.c @@ -253,7 +253,8 @@ static int print_term(FILE* fp, ei_x_buff* x, erlang_big *b; char *ds; - b = ei_alloc_big(n); + if ( (b = ei_alloc_big(n)) == NULL) goto err; + if (ei_decode_big(buf, index, b) < 0) { ei_free_big(b); goto err; -- 1.7.0.4 From michael.santos@REDACTED Fri Oct 29 02:45:35 2010 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 28 Oct 2010 20:45:35 -0400 Subject: [PATCH 2/2] ei: check memory was allocated Message-ID: <20101029004535.GA17669@ecn.lan> --- lib/erl_interface/src/misc/ei_portio.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/lib/erl_interface/src/misc/ei_portio.c b/lib/erl_interface/src/misc/ei_portio.c index c4e397f..a3f6f63 100644 --- a/lib/erl_interface/src/misc/ei_portio.c +++ b/lib/erl_interface/src/misc/ei_portio.c @@ -166,6 +166,9 @@ int ei_writev_fill_t(int fd, const struct iovec *iov, int iovcnt, unsigned if (done < sum) { if (iov_base == NULL) { iov_base = malloc(sizeof(struct iovec) * iovcnt); + if (iov_base == NULL) { + return -1; + } memcpy(iov_base, iov, sizeof(struct iovec) * iovcnt); current_iov = iov_base; } -- 1.7.0.4 From michal.ptaszek@REDACTED Fri Oct 29 13:19:29 2010 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Fri, 29 Oct 2010 13:19:29 +0200 Subject: Supervisor transient workers restart strategy fix Message-ID: <638DB44D-BDED-428F-A376-BBA37A1ADEFA@erlang-solutions.com> Hi, we have found the following inconsistency in the supervisor.erl code: when supervisor's child restart strategy is set to transient and the child exits with reason shutdown, it is never restarted. The patches include fix for the issue and the test suite used for reproducing the error, and later, after patching, for verifying the correctness of the code. The patches are available here: git fetch git@REDACTED:paulgray/otp.git supervisor_transient_children_fix Best regards, Michal Ptaszek From tuncer.ayaz@REDACTED Fri Oct 29 13:46:01 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 29 Oct 2010 13:46:01 +0200 Subject: [PATCH] ethread: do not return from void ethr_atomic_set_relb Message-ID: <1288352761-3675-1-git-send-email-tuncer.ayaz@gmail.com> Reported-by: Patrick Baggett --- erts/include/internal/ethread.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/erts/include/internal/ethread.h b/erts/include/internal/ethread.h index 4a20569..fcac7df 100644 --- a/erts/include/internal/ethread.h +++ b/erts/include/internal/ethread.h @@ -757,7 +757,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_atomic_set_relb)(ethr_atomic_t *var, long val) #ifdef ETHR_HAVE_NATIVE_ATOMICS ethr_native_atomic_set_relb(var, val); #else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(var, val); + ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(var, val); #endif } From bgustavsson@REDACTED Fri Oct 29 14:57:42 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 29 Oct 2010 14:57:42 +0200 Subject: What's cooking in erlang/otp (2010-10-29) Message-ID: The next release will be R14B01, expected to be released in December. ------------------------------------------------------------ [Graduated] * ks/bad-export_type (stdlib) (2010-10-06) 1 commit + Avoid errors for a badly formed export_type declarations (e041319) * ks/ets-update-counter (hipe) (2010-10-02) 1 commit + Fix type of ets:update_counter/3 to correspond with the documentation (56916f3) * ks/export-types (stdlib) (2010-09-24) 1 commit + Export opaque types so as to be used by other modules (4d60cb7) * ks/fix-dialyzer_plt-spec (dialyzer) (2010-10-01) 1 commit + Dialyzer: Fix erroneous type and use the same definition in two places (d158c01) * ks/rec-erl_types (hipe) (2010-10-14) 1 commit + Deeper unfolding of recursive types (0c18ec9) * ks/strengthen-specs (compiler, stdlib, syntax_tools) (2010-10-07) 4 commits + lists: Strengthen and modernize specs (54c54dc) + syntax_tools: Fix an erroneous type and strengthen some specs (44eb815) + ordsets: Export ordset/1 type and strengthen specs (251a255) + compiler: Strengthen some specs to shut off dialyzer warnings (31e4428) * ks/supervisor-export-types (stdlib) (2010-10-06) 1 commit + supervisor: Type and spec related changes (759f90b) * ks/syntax_tools-types (syntax_tools) (2010-10-05) 1 commit + syntax_tools: Add types to record definitions (44d6f54) * ms/fix-erl_call (erl_interface) (2010-10-05) 1 commit + erl_call: remove get_hostent (d9edfa8) * ms/fix-string-copies (stdlib) (2010-10-29) 3 commits + string_SUITE: Add test for string:copies(Str, Float) (b9fc6c5) + string_SUITE: Fix copy and paste error in test of string:copies/2 (0af8aef) + Fix crash in string:copies/2 (0deea46) * ms/fix-zero-byte-allocation (erl_interface) (2010-10-05) 1 commit + registry: fix zero byte allocation (98a1edf) * ms/security-fixes (erts, stdlib) (2010-10-01) 8 commits + erlc: remove unused variable (0983ce5) + typer: prevent buffer overflows (b3904e7) + run_test: prevent buffer overflow (329d7a2) + heart: prevent buffer overflow (731755b) + escript: prevent buffer overflows (e5a231d) + erlexec: prevent buffer overflows (570dc4e) + erlc: prevent buffer overflows (be8759e) + dialyzer: prevent buffer overflows (3d430d0) * sv/ei-number-parsing (erl_interface) (2010-10-06) 1 commit + teach ei_x_format to handle unary - and + (386050e) * ta/fix-dialyzer-documentation (dialyzer) (2010-10-23) 1 commit + Add missing Dialyzer warning options to doc (1b0f5a2) ------------------------------------------------------------ [New topics] * bw/efile_exists (erts) (2010-10-07) 1 commit - prim_file:exists/{1,2} which is lightweight file existence check (7203932) * bw/gitignore (otp) (2010-10-05) 1 commit - Add win32 specific build artifacts to .gitignore (834cd98) * bw/tv-render-fix (tv) (2010-10-06) 1 commit - tv: Allow table viewer to display refs, ports and small binaries (5eff630) * bw/win32-file-linking (erts) (2010-10-04) 1 commit - Add Win32 support for hard and symbolic file system links (1f6d7c8) * bw/win32-laa (erts) (2010-10-06) 1 commit - Added win32 large address aware link option (8bb6f48) * cg/fix-prng (stdlib) (2010-10-06) 1 commit - Fix a bug in the implementation of the pseudo-random number generator (863ec4d) * ks/file_open-mode (hipe, kernel, stdlib) (2010-10-12) 1 commit - Add the {encoding, _} options to file:open/2's modes (6751329) * ms/ei-fix-node-name-overflow (erl_interface) (2010-10-24) 1 commit - ei: check the length of the node name (cea0698) * pg/des-cfb-functions (crypto) (2010-10-16) 1 commit - Add DES and Triple DES cipher feedback (CFB) mode functions (2c79ae7) * ta/fix-native-looping (kernel) (2010-10-04) 1 commit - error_handler: add no_native compiler directive (fb5e332) * uw/xmerl-expand-simple (xmerl) (2010-10-25) 1 commit - Track parents when expanding #xmlElement records (9f4c1ee) ------------------------------------------------------------ [Stalled] * dc/update-INSTALL-WIN32.md (otp) (2010-09-19) 1 commit - Doc update to latest dependencies & how to compile with free VS2008 Express (8e8061c) Waiting for an update by the topic author. * jn/gen_stream (stdlib) (2010-09-05) 5 commits - Use erlang:process_info(Pid, links) to determine if gen_stream managed worker processes are leaked (b2386df) - Move gen_stream internal process test to after termination of gen_stream parent (f1d8a97) - Add calls to code loader in every test to verify code loader still running (ac6cbba) - Track gen_stream processes directly for accurate test results (38a4083) - Patch dev to reproduce jn/gen_stream (7694a74) . Add new gen_stream behaviour for efficiently consuming serial streams (65f3db8) This topic branch has a failing test case: shell_SUITE:start_restricted_from_shell; We have now tested with and without this topic branch and when it is present that test case fails. We consider it proved to be guilty. The test case does not fail when run standalone so it seems to be tricky... Also, some of its own test cases fail sporadicly. Action expected from: Topic author. * jp/dependencies_makefile (compiler, erts) (2009-12-14) 1 commit - Add dependencies Makefile generation to erlc(1) and compile(3) (4945add) Approved in principle by the OTP Technical Board; now awaits a closer review of the implementation. Action expected from: Bj?rn Gustavsson (the reviewer) * sf/erts_de_busy_limit (erts) (2010-10-22) 1 commit - Add flag-based setting for 'erts_de_busy_limit' constant (e3da5a0) This topic branch has multiple issues described in this email: http://www.erlang.org/cgi-bin/ezmlm-cgi?3:mss:1303:201008:ldciadjdmjednikemjch Action expected from: Topic author. ------------------------------------------------------------ [Cooking] * bg/temp-hipe-fixes (compiler, hipe, kernel) (2010-03-15) 6 commits - Avoid crash by skipping compilation_SUITE:on_load/1 (8592a02) - Inline less aggressively for native-code compilation (e699ad4) - Crudely fix return type for the lists:key{search,find,member}/3 (2b72202) - seq_trace_SUITE: Don't native-compile (2712344) - Disable native code if on_load is used (e8d0ea4) - andor_SUITE: Don't native-compile (e25ef8c) This branch contains temporary workarounds to avoid failing test cases and is never intended to graduate. * fm/httpc-upload-body-streaming (inets) (2010-10-05) 2 commits - httpc: add option to do automatic chunked transfer-encoding (6951ed1) - httpc: allow streaming of PUT and POST request bodies (f9ec3cb) * gc/gen-format-status-improvements (stdlib) (2010-06-12) 1 commit - Fix format_status bug for unregistered gen_event processes (95ed86f) * jf/run_erl-disable-flow-control (erts) (2010-03-05) 1 commit - Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control (d0775cd) * jr/windows-file-append (erts) (2010-09-30) 1 commit - Fix appending to large files (>4GB) on Windows (6aae439) * km/pool_connect-to-running-nodes (stdlib) (2010-09-02) 1 commit - Change pool module to attempt to attach to nodes that are already running (a9f4cbc) * mh/dialyzer-full-path-option (dialyzer) (2010-09-15) 1 commit - Dialyzer: add --fullpath option for printing full paths in warnings (3703ea6) * ml/add-os-pid-to-port-info (erts) (2010-08-19) 1 commit - Extend erlang:port_info/1,2 to show the OS pid of a spawned process (b2d85bf) * pg/fix-hibernate-with-hipe (erts, otp) (2010-09-27) 1 commit - Fix several bugs related to hibernate/3 and HiPE (2b5ac3e) * pg/fix-system_info-cpu_topology-segfault (erts) (2010-10-02) 1 commit - Fix crash with erlang:system_info({cpu_topology,junk}) (f8201e5) * pg/freebsd-cpu-affinity-and-topology (erts) (2010-10-03) 1 commit - Add support for CPU affinity & topology detection on FreeBSD 8 (c55fd8f) * pg/honor-start-type-in-rel-files (sasl) (2010-06-06) 1 commit - Honor start type in .rel files when building relup files (7599900) * sa/callback-attr (compiler, inets, kernel, otp, stdlib) (2010-06-08) 7 commits - Add callback specs into 'application' module in kernel (420d28a) - Add callback specs to tftp module following internet documentation (711bcce) - Add callback specs to inets_service module following possibly deprecated comments (cffbf06) - Add '-callback' attributes in stdlib's behaviours (e4d46c9) - Update primary bootstrap (2706b2d) - Automatical generation of 'behaviour_info' function from '-callback' attributes (2e224ed) - Add '-callback' attribute to language syntax (075b15d) * sb/make-files-like-erlc (tools) (2010-05-18) 1 commit - Change make:files to behave more like erlc (5e9d051) * sg/fix-diskless-booted-relup (sasl) (2010-09-19) 1 commit - Remove traces of release_handler reading from filesystem when it has Masters list (54cb5df) * sv/epmd-check-listen (erts) (2010-07-14) 1 commit - check return value of epmd server listen() call (c5c8baa) * ta/asn1-reporting (asn1) (2010-04-23) 1 commit - asn1ct: Make formatting of errors and warnings consistent (93face2) * ta/ensure-dot-ssh-exists (ssh) (2010-09-02) 1 commit - ssh: ensure ~/.ssh exists (2c14d90) * ta/sendfile (erts, kernel) (2010-09-06) 1 commit - Implement file:sendfile (633c1e7) * ts/cover-with-export_all (tools) (2010-09-23) 1 commit - add user specified compiler options on form reloading (eb02beb) ------------------------------------------------------------ [Dropped] * er/ei-decode_msg (erl_interface) (2010-03-11) 1 commit . Add function to decode distribution header (864091a) Stalled for a long time with missing documentation and test cases. * jb/erl_call-get_hostent (erl_interface) (2010-09-23) 1 commit . Teach erl_call to use an IP address for remote node name (d369555) * kr/gs-browser (gs) (2010-02-10) 1 commit . Change lib/gs/src/tool_utils.erl to select multiple browsers other than Netscape (802d159) Stalled for a long time. * ms/prevent-gdb-hang (erts) (2010-10-05) 1 commit . cerl: prevent gdb hang at prompt (e687f88) * uw/epmd-fullnames (kernel) (2010-05-07) 1 commit . Allow the erlang nodename host part to differ from the hostname (e0774be) Needs to be rebased before it can be included in 'pu' again. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Fri Oct 29 15:30:10 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 29 Oct 2010 15:30:10 +0200 Subject: [erlang-patches] improve the 're' manpage by correcting typos and rewording In-Reply-To: <20101028092248.GA5365@corelatus.se> References: <20101028092248.GA5365@corelatus.se> Message-ID: On Thu, Oct 28, 2010 at 11:22 AM, Matthias Lang wrote: > Hi, > > git fetch git://github.com/matthiasl/otp.git re-manpage-cleanup > > I took a couple of passes through the manpage and fixed all the typos > I saw. I also shortened some sentences. > > (Branch looks wonky because I just learned that 'git commit --amend' > doesn't work if you've already done a 'git push'. So I merged and pushed > again.) If you do 'git commit --amend' you will have force the push using the -f option. Are you sure that you have pushed the correct branch? The branch does not look wonky to me and I don't see any merge commit made by you. You should check 'pu' when I'll have pushed to make sure that it has the expected version of your branch. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Fri Oct 29 15:33:49 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 29 Oct 2010 15:33:49 +0200 Subject: [erlang-patches] [PATCH 2/2] ei: check memory was allocated In-Reply-To: <20101029004535.GA17669@ecn.lan> References: <20101029004535.GA17669@ecn.lan> Message-ID: On Fri, Oct 29, 2010 at 2:45 AM, Michael Santos wrote: > --- > lib/erl_interface/src/misc/ei_portio.c | 3 +++ > 1 files changed, 3 insertions(+), 0 deletions(-) > Thanks! Will include this and the previous patch in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Fri Oct 29 15:36:48 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 29 Oct 2010 15:36:48 +0200 Subject: [erlang-patches] Supervisor transient workers restart strategy fix In-Reply-To: <638DB44D-BDED-428F-A376-BBA37A1ADEFA@erlang-solutions.com> References: <638DB44D-BDED-428F-A376-BBA37A1ADEFA@erlang-solutions.com> Message-ID: On Fri, Oct 29, 2010 at 1:19 PM, Michal Ptaszek wrote: > Hi, > > we have found the following inconsistency in the supervisor.erl code: > when supervisor's child restart strategy is set to transient and the child exits with reason shutdown, it is never restarted. > > The patches include fix for the issue and the test suite used for reproducing the error, and later, after patching, for verifying the correctness of the code. > > The patches are available here: > git fetch git@REDACTED:paulgray/otp.git > supervisor_transient_children_fix Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Fri Oct 29 15:39:16 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 29 Oct 2010 15:39:16 +0200 Subject: [erlang-patches] [PATCH] ethread: do not return from void ethr_atomic_set_relb In-Reply-To: <1288352761-3675-1-git-send-email-tuncer.ayaz@gmail.com> References: <1288352761-3675-1-git-send-email-tuncer.ayaz@gmail.com> Message-ID: On Fri, Oct 29, 2010 at 1:46 PM, Tuncer Ayaz wrote: > Reported-by: Patrick Baggett > --- > erts/include/internal/ethread.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) Thanks! Will include in pu. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From rickard@REDACTED Fri Oct 29 16:31:27 2010 From: rickard@REDACTED (Rickard Green) Date: Fri, 29 Oct 2010 16:31:27 +0200 Subject: Patch for configurable ERTS_DE_BUSY_LIMIT In-Reply-To: <56931.1287782332@snookles.snookles.com> References: <56931.1287782332@snookles.snookles.com> Message-ID: <4CCADABF.6070201@erlang.org> I've made a number of changes in a separate commit after your commit. If you approve these changes, I'll squash the commits with you as the author of the resulting commit. If so (and no unexpected problems show up in our daily builds), the resulting commit will be merged into dev some time next week. My changes is available at: git fetch git://github.com/rickard-green/otp.git sf/erts_de_busy_limit Regards, Rickard Scott Lystig Fritchie wrote: > Rickard Green wrote: > > rg> I've had a look at your patch, and it needs some improvements before > rg> we can accept it. > > Rickard, after sitting on the shelf for quite a while, please see the > erts_de_busy_limit_take2 branch at git://github.com/slfritchie/otp.git. > > -Scott -- Rickard Green, Erlang/OTP, Ericsson AB. From tuncer.ayaz@REDACTED Fri Oct 29 21:21:42 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 29 Oct 2010 21:21:42 +0200 Subject: [PATCH] Fix two typos in Dialyzer documentation Message-ID: <1288380102-10632-1-git-send-email-tuncer.ayaz@gmail.com> --- lib/dialyzer/doc/src/dialyzer.xml | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index b977a44..2930888 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -101,9 +101,9 @@ Store the PLT at the specified location after building it. - Use the specified plt as the initial persistent lookup table. + Use the specified PLT as the initial persistent lookup table. - a family of option which selectively turn on/off warnings. + a family of options which selectively turn on/off warnings. (for help on the names of warnings use ) do not disable the Erlang shell while running the GUI