From saleyn@REDACTED Fri Jul 11 03:22:53 2008 From: saleyn@REDACTED (Serge Aleynikov) Date: Thu, 10 Jul 2008 21:22:53 -0400 Subject: [erlang-patches] ei patch to handle NEW_FLOAT_EXT Message-ID: <4876B5ED.7010402@gmail.com> Ever since the emulator support of the compact IEEE 754 double encoding in external binary format was introduced I've been patiently waiting for ei to support this feature. Finally I gave up and implemented this myself as I recently needed to marshal lots of doubles between Erlang and C++ and size of the binary stream did matter. Attached is a patch for R12B-2 that implements this feature. The only thing I wasn't sure about was whether backward compatibility of ei_{encode,decode}_double() is truly required. It exists in the emulator, where the new functionality is controlled by term_to_binary(Term, [{minor_version, 1}]), and is essential indeed, but is it really needed for ei? In order to be properly backward compatible, it would likely require to introduce yet another pair of functions, such as: int ei_encode_double_new(char *buf, int *index, double p) int ei_x_encode_double_new(ei_x_buff* x, const void *p, long len) which gets a little messy as there are already a fair amount of ei functions. Since the internals of the external binary format are hidden behind the facade of: int ei_encode_double(char *buf, int *index, double p) int ei_decode_double(const char *buf, int *index, double *p) Perhaps retaining the compatibility of lower-level implementation is not that crucial? In the patch the old functionality is controlled by setting USE_OLD_FLOAT_ENCODER macro, but this is left discretionary to the OTP team. Regards, Serge -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ei.R12B-2.patch URL: From egil@REDACTED Fri Jul 11 11:59:38 2008 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Fri, 11 Jul 2008 11:59:38 +0200 Subject: [erlang-patches] ei patch to handle NEW_FLOAT_EXT In-Reply-To: <4876B5ED.7010402@gmail.com> References: <4876B5ED.7010402@gmail.com> Message-ID: <48772F0A.1000703@erix.ericsson.se> Hi Serge, Thank you for sending a patch and underlining this problem. Compatibility is a problem, and I am not sure yet if this is the solution. I will look into the matter. *adding it to my todo list* Regards, Bj?rn-Egil Erlang/OTP Serge Aleynikov wrote: > Ever since the emulator support of the compact IEEE 754 double encoding > in external binary format was introduced I've been patiently waiting for > ei to support this feature. Finally I gave up and implemented this > myself as I recently needed to marshal lots of doubles between Erlang > and C++ and size of the binary stream did matter. > > Attached is a patch for R12B-2 that implements this feature. The only > thing I wasn't sure about was whether backward compatibility of > ei_{encode,decode}_double() is truly required. It exists in the > emulator, where the new functionality is controlled by > term_to_binary(Term, [{minor_version, 1}]), and is essential indeed, but > is it really needed for ei? In order to be properly backward > compatible, it would likely require to introduce yet another pair of > functions, such as: > > int ei_encode_double_new(char *buf, int *index, double p) > int ei_x_encode_double_new(ei_x_buff* x, const void *p, long len) > > which gets a little messy as there are already a fair amount of ei > functions. > > Since the internals of the external binary format are hidden behind the > facade of: > > int ei_encode_double(char *buf, int *index, double p) > int ei_decode_double(const char *buf, int *index, double *p) > > Perhaps retaining the compatibility of lower-level implementation is not > that crucial? In the patch the old functionality is controlled by > setting USE_OLD_FLOAT_ENCODER macro, but this is left discretionary to > the OTP team. > > Regards, > > Serge > > > ------------------------------------------------------------------------ > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://www.erlang.org/mailman/listinfo/erlang-patches From saleyn@REDACTED Fri Jul 11 13:01:52 2008 From: saleyn@REDACTED (Serge Aleynikov) Date: Fri, 11 Jul 2008 07:01:52 -0400 Subject: [erlang-patches] ei patch to handle NEW_FLOAT_EXT In-Reply-To: <48772F0A.1000703@erix.ericsson.se> References: <4876B5ED.7010402@gmail.com> <48772F0A.1000703@erix.ericsson.se> Message-ID: <48773DA0.5000702@gmail.com> Another thought that crossed my mind on the ei's low-level old style decoding compatibility of doubles was to control it via introducing ei_init() with some global flags controlling fine-grain behavior as opposed to introducing more functions of ei_encode_double family. Serge Bj?rn-Egil Dahlberg wrote: > Hi Serge, > > Thank you for sending a patch and underlining this problem. > > Compatibility is a problem, and I am not sure yet if this is the solution. > > I will look into the matter. *adding it to my todo list* > > Regards, > Bj?rn-Egil > Erlang/OTP > > Serge Aleynikov wrote: >> Ever since the emulator support of the compact IEEE 754 double >> encoding in external binary format was introduced I've been patiently >> waiting for ei to support this feature. Finally I gave up and >> implemented this myself as I recently needed to marshal lots of >> doubles between Erlang and C++ and size of the binary stream did matter. >> >> Attached is a patch for R12B-2 that implements this feature. The only >> thing I wasn't sure about was whether backward compatibility of >> ei_{encode,decode}_double() is truly required. It exists in the >> emulator, where the new functionality is controlled by >> term_to_binary(Term, [{minor_version, 1}]), and is essential indeed, >> but is it really needed for ei? In order to be properly backward >> compatible, it would likely require to introduce yet another pair of >> functions, such as: >> >> int ei_encode_double_new(char *buf, int *index, double p) >> int ei_x_encode_double_new(ei_x_buff* x, const void *p, long len) >> >> which gets a little messy as there are already a fair amount of ei >> functions. >> >> Since the internals of the external binary format are hidden behind >> the facade of: >> >> int ei_encode_double(char *buf, int *index, double p) >> int ei_decode_double(const char *buf, int *index, double *p) >> >> Perhaps retaining the compatibility of lower-level implementation is >> not that crucial? In the patch the old functionality is controlled by >> setting USE_OLD_FLOAT_ENCODER macro, but this is left discretionary to >> the OTP team. >> >> Regards, >> >> Serge >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> erlang-patches mailing list >> erlang-patches@REDACTED >> http://www.erlang.org/mailman/listinfo/erlang-patches > > From jan@REDACTED Thu Jul 17 20:45:38 2008 From: jan@REDACTED (Jan Lehnardt) Date: Thu, 17 Jul 2008 20:45:38 +0200 Subject: [erlang-patches] Patch for file::sync() to use fcntl(F_FULLFSYNC) on Darwin / Mac OS X Message-ID: Dear OTP Team, traditionally, on UNIX systems the fsync() system call is used to flush any filesystem buffers to disk. The file::sync() function triggers a fsync() call in OTP R12B-3 On Darwin (and Mac OS X) systems, the fsync() system call exists, but does not guarantee the writing to disk. The fcntl(F_FULLFSYNC) function call exists to achieve this. See http://developer.apple.com/documentation/Darwin/Reference/ManPages/man2/fsync.2.html and http://developer.apple.com/documentation/Darwin/Reference/ManPages/man2/fcntl.2.html for reference. The patch below adds some #ifdefs to figure out if it is compiled on a Darwin system and uses the fcntl() function instead of the fsync() function. Please note my poor understanding of how your build system actually detects the target host. My way of detecting Darwin was shamelessly ripped from elsewhere in the source tree and might be wrong in this case (although testing was successful). Please feel free to make any necessary changes in case you choose to integrate the patch. I'm working on the CouchDB project (yes, a database in Erlang), and we require a reliable file::sync()-behaviour for data consistency. Adding this to OTP would be highly appreciated. Cheers Jan & the CouchDB Team -- --- erts/emulator/drivers/unix/unix_efile.c.orig 2008-07-17 20:44:23.000000000 +0200 +++ erts/emulator/drivers/unix/unix_efile.c 2008-07-17 20:44:21.000000000 +0200 @@ -44,6 +44,14 @@ #endif #endif /* _OSE_ */ +#if defined(__APPLE__) && defined(__MACH__) && !defined(__DARWIN__) +#define DARWIN 1 +#endif + +#ifdef DARWIN +#include +#endif /* DARWIN */ + #ifdef VXWORKS #include #include @@ -818,7 +826,11 @@ undefined fsync #endif /* VXWORKS */ #else +#if defined(DARWIN) && defined(F_FULLFSYNC) + return check_error(fcntl(fd, F_FULLFSYNC), errInfo); +#else return check_error(fsync(fd), errInfo); +#endif /* DARWIN */ #endif /* NO_FSYNC */ } From oscar@REDACTED Fri Jul 25 20:16:36 2008 From: oscar@REDACTED (=?ISO-8859-1?Q?Oscar_Hellstr=F6m?=) Date: Fri, 25 Jul 2008 19:16:36 +0100 Subject: [erlang-patches] R11B SSL patch Message-ID: <488A1884.1000408@erlang-consulting.com> Hi, I've noticed quite a few problems with the SSL driver exiting after printing "esock: Got connect request while PASSIVE". This is probably more easily triggered when the ssl:accept/2 is called with a timeout in a loop (since I suspect that it then sends the NOACCEPT command to the port quite often). My suspicion is that if a connection attempt have been received, and polled from the listener socket, when the is an ESOCK_NOACCEPT_CMD waiting we'll end up receiving the connect event in ESOCK_PASSIVE_LISTENING further down in the loop. I don't quite understand why we need to exit whenever we poll a connection on a listener socket in ESOCK_PASSIVE_LISTENING state, but I guess it's because there shouldn't be an event there. To avoid this I've added a function to the esock_poll.c which can clear events for the fd in question. This should avoid the driver exiting with the "esock: Got connect request while PASSIVE" message and thus taking down all the SSL connections. Please note that this has ONLY been tested with the poll interface and not even complied with select. There has also been two emails describing this problem: http://www.erlang.org/pipermail/erlang-questions/2006-September/022755.html http://www.erlang.org/pipermail/erlang-bugs/2008-June/000829.html -- Oscar Hellstr?m, oscar@REDACTED Office: +44 20 7655 0337 Mobile: +44 798 45 44 773 Erlang Training and Consulting http://www.erlang-consulting.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-r11b-3.ssl.patch Type: text/x-patch Size: 1375 bytes Desc: not available URL: From ingela@REDACTED Mon Jul 28 14:14:24 2008 From: ingela@REDACTED (Ingela Anderton Andin) Date: Mon, 28 Jul 2008 14:14:24 +0200 Subject: [erlang-patches] R11B SSL patch In-Reply-To: <488A1884.1000408@erlang-consulting.com> References: <488A1884.1000408@erlang-consulting.com> Message-ID: <488DB820.2000004@erix.ericsson.se> Hi, thank you for the patch. I have created a ticket for this and we will try it out when we can find some time for it. However we are in the process of making a new ssl-implementation in erlang that will replace the ssl-driver version as soon as it is complete enough. A trail-version is already available in R12B but it does not yet support all necessary features and ssl/tls versions to replace the old ssl. Regards - Ingela, Erlang/OTP Ericsson Oscar Hellstr?m wrote: > Hi, > > I've noticed quite a few problems with the SSL driver exiting after > printing "esock: Got connect request while PASSIVE". This is probably > more easily triggered when the ssl:accept/2 is called with a timeout > in a loop (since I suspect that it then sends the NOACCEPT command to > the port quite often). > > My suspicion is that if a connection attempt have been received, and > polled from the listener socket, when the is an ESOCK_NOACCEPT_CMD > waiting we'll end up receiving the connect event in > ESOCK_PASSIVE_LISTENING further down in the loop. I don't quite > understand why we need to exit whenever we poll a connection on a > listener socket in ESOCK_PASSIVE_LISTENING state, but I guess it's > because there shouldn't be an event there. To avoid this I've added a > function to the esock_poll.c which can clear events for the fd in > question. This should avoid the driver exiting with the "esock: Got > connect request while PASSIVE" message and thus taking down all the > SSL connections. > > Please note that this has ONLY been tested with the poll interface and > not even complied with select. > > There has also been two emails describing this problem: > http://www.erlang.org/pipermail/erlang-questions/2006-September/022755.html > http://www.erlang.org/pipermail/erlang-bugs/2008-June/000829.html > ------------------------------------------------------------------------ > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://www.erlang.org/mailman/listinfo/erlang-patches