From stefan.zegenhagen@REDACTED  Mon Jul  2 15:03:14 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Mon, 02 Jul 2012 15:03:14 +0200
Subject: [erlang-patches] error in crypto application
In-Reply-To: <4FEC6825.2060508@erix.ericsson.se>
References: <1340874032.3506.2.camel@ax-sze>
 <4FEC6825.2060508@erix.ericsson.se>
Message-ID: <1341234194.3506.30.camel@ax-sze>

Dear Sverker,

here comes an updated version of the patch that extends the previous
version by the following:
 - fix the same issue for aes_cbc_*_{en|de}crypt
 - fix the documentation

I didn't find any relevant test case in the test suite that would deal
with input data of arbitrary length. Unfortunately, I'm insufficiently
familiar with eunit to feel fit to create a new test case. I hope the
patch can get accepted nevertheless.


Kind regards,


--- snip ---

>From c97b647babc0b5e0a0645c9b929c8af7dd8a1061 Mon Sep 17 00:00:00 2001
From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
Date: Thu, 28 Jun 2012 10:50:40 +0200
Subject: [PATCH] CRYPTO: aes_cfb_128_crypt can handle arbitrary text length

The OpenSSL implementation of AES_cfb128_encrypt can handle data to
en-/decrypt with arbitrary length.

The restriction that the data length needs to be a multiple of 16 bytes
is unnecessary and breaks the SNMP application (causing decryption and
encryption errors in SNMPv3 AES-encrypted packets).

Fix the same issue for aes_cbc_{128|256}_{en|de}crypt, the OpenSSL
version of which can also handle arbitrary text lengths.
---
 lib/crypto/c_src/crypto.c     |    6 ++----
 lib/crypto/doc/src/crypto.xml |    6 ++----
 2 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c
index 4be593e..0d19d59 100644
--- a/lib/crypto/c_src/crypto.c
+++ b/lib/crypto/c_src/crypto.c
@@ -954,8 +954,7 @@ static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TE
 
     if (!enif_inspect_iolist_as_binary(env, argv[0], &key) || key.size != 16
 	|| !enif_inspect_binary(env, argv[1], &ivec) || ivec.size != 16
-	|| !enif_inspect_iolist_as_binary(env, argv[2], &text)
-	|| text.size % 16 != 0) {
+	|| !enif_inspect_iolist_as_binary(env, argv[2], &text) ) {
 	return enif_make_badarg(env);
     }
 
@@ -1340,8 +1339,7 @@ static ERL_NIF_TERM aes_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM a
 	|| (key_bin.size != 16 && key_bin.size != 32)
 	|| !enif_inspect_binary(env, argv[1], &ivec_bin)
 	|| ivec_bin.size != 16
-	|| !enif_inspect_iolist_as_binary(env, argv[2], &data_bin)
-	|| data_bin.size % 16 != 0) {
+	|| !enif_inspect_iolist_as_binary(env, argv[2], &data_bin)) {
 
 	return enif_make_badarg(env);
     }
diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml
index 19db6c9..06aa213 100644
--- a/lib/crypto/doc/src/crypto.xml
+++ b/lib/crypto/doc/src/crypto.xml
@@ -651,8 +651,7 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
       </type>
       <desc>
         <p>Encrypts <c>Text</c> according to AES in Cipher Feedback 
-          mode (CFB) or Cipher Block Chaining mode (CBC). <c>Text</c>
-          must be a multiple of 128 bits (16 bytes). <c>Key</c> is the
+          mode (CFB) or Cipher Block Chaining mode (CBC). <c>Key</c> is the
           AES key, and <c>IVec</c> is an arbitrary initializing vector.
           The lengths of <c>Key</c> and <c>IVec</c> must be 128 bits
           (16 bytes).</p>
@@ -671,8 +670,7 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
           or Cipher Block Chaining mode (CBC).
           <c>Key</c> is the AES key, and <c>IVec</c> is an arbitrary
           initializing vector. <c>Key</c> and <c>IVec</c> must have
-          the same values as those used when encrypting. <c>Cipher</c>
-          must be a multiple of 128 bits (16 bytes).  The lengths of
+          the same values as those used when encrypting. The lengths of
           <c>Key</c> and <c>IVec</c> must be 128 bits (16 bytes).</p>
       </desc>
     </func>
-- 
1.7.9.5


-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From tuncer.ayaz@REDACTED  Tue Jul  3 11:25:57 2012
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Tue, 3 Jul 2012 11:25:57 +0200
Subject: [erlang-patches] observer: add missing modules to app resource file
Message-ID: <CAOvwQ4gFE4mTwxL=NbKBPbrikZT2FMtdN8KANxdaMVjZsR6uTQ@mail.gmail.com>

git fetch git://github.com/tuncer/otp.git observer-app

https://github.com/tuncer/otp/compare/observer-app
https://github.com/tuncer/otp/compare/observer-app.patch


From tuncer.ayaz@REDACTED  Tue Jul  3 11:38:05 2012
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Tue, 3 Jul 2012 11:38:05 +0200
Subject: [erlang-patches] observer: add missing modules to app resource
	file
In-Reply-To: <CAOvwQ4gFE4mTwxL=NbKBPbrikZT2FMtdN8KANxdaMVjZsR6uTQ@mail.gmail.com>
References: <CAOvwQ4gFE4mTwxL=NbKBPbrikZT2FMtdN8KANxdaMVjZsR6uTQ@mail.gmail.com>
Message-ID: <CAOvwQ4hyTJYi5kc-ZuJe-WXxieqa0MbnZoiWayb_xwChTvHE1w@mail.gmail.com>

On Tue, Jul 3, 2012 at 11:25 AM, Tuncer Ayaz wrote:
> git fetch git://github.com/tuncer/otp.git observer-app
>
> https://github.com/tuncer/otp/compare/observer-app
> https://github.com/tuncer/otp/compare/observer-app.patch

Rewrote the commit message:
==
observer: fix app file (Noticed-by: Motiejus Jakstys)

Add missing observer modules to observer.app.src.
==


From henrik@REDACTED  Tue Jul  3 11:43:40 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 3 Jul 2012 11:43:40 +0200
Subject: [erlang-patches] observer: add missing modules to app resource
 file
In-Reply-To: <CAOvwQ4hyTJYi5kc-ZuJe-WXxieqa0MbnZoiWayb_xwChTvHE1w@mail.gmail.com>
References: <CAOvwQ4gFE4mTwxL=NbKBPbrikZT2FMtdN8KANxdaMVjZsR6uTQ@mail.gmail.com>
 <CAOvwQ4hyTJYi5kc-ZuJe-WXxieqa0MbnZoiWayb_xwChTvHE1w@mail.gmail.com>
Message-ID: <4FF2BECC.5040204@erlang.org>

Thank you, I have included it in 'pu'

On 07/03/2012 11:38 AM, Tuncer Ayaz wrote:
> On Tue, Jul 3, 2012 at 11:25 AM, Tuncer Ayaz wrote:
>> git fetch git://github.com/tuncer/otp.git observer-app
>>
>> https://github.com/tuncer/otp/compare/observer-app
>> https://github.com/tuncer/otp/compare/observer-app.patch
> Rewrote the commit message:
> ==
> observer: fix app file (Noticed-by: Motiejus Jakstys)
>
> Add missing observer modules to observer.app.src.
> ==
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From stefan.zegenhagen@REDACTED  Tue Jul  3 11:59:13 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Tue, 03 Jul 2012 11:59:13 +0200
Subject: [erlang-patches] error in crypto application
In-Reply-To: <1341234194.3506.30.camel@ax-sze>
References: <1340874032.3506.2.camel@ax-sze>
 <4FEC6825.2060508@erix.ericsson.se> <1341234194.3506.30.camel@ax-sze>
Message-ID: <1341309553.3506.67.camel@ax-sze>

Dear all,

here, comes a third version of the patch that
      * fixes the aes_cfb_128_crypt() function to not require the
        plaintext/cipher data to be a multiple of 16 bytes in length
      * adjusts the documentation accordingly, and
      * leaves the aes_cbc_crypt() function untouched because its
        current API in erlang requires the plaintext/cipher data to be a
        multiple of 16 bytes.


Kind regards,


--- snip ---
>From c9f86ea4a2244deeae4efd2955b329857d91e340 Mon Sep 17 00:00:00 2001
From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
Date: Mon, 2 Jul 2012 22:00:09 +0200
Subject: [PATCH] CRYPTO: aes_cfb_128_crypt can handle arbitrary text length

The OpenSSL implementation of AES_cfb128_encrypt can handle data to
en-/decrypt with arbitrary length.

The restriction that the data length needs to be a multiple of 16 bytes
is unnecessary and breaks the SNMP application (causing decryption and
encryption errors in SNMPv3 AES-encrypted packets).
---
 lib/crypto/c_src/crypto.c     |    3 +--
 lib/crypto/doc/src/crypto.xml |   42 ++++++++++++++++++++++++++++++++++-------
 2 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c
index 4be593e..a6a81d6 100644
--- a/lib/crypto/c_src/crypto.c
+++ b/lib/crypto/c_src/crypto.c
@@ -954,8 +954,7 @@ static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TE
 
     if (!enif_inspect_iolist_as_binary(env, argv[0], &key) || key.size != 16
 	|| !enif_inspect_binary(env, argv[1], &ivec) || ivec.size != 16
-	|| !enif_inspect_iolist_as_binary(env, argv[2], &text)
-	|| text.size % 16 != 0) {
+	|| !enif_inspect_iolist_as_binary(env, argv[2], &text)) {
 	return enif_make_badarg(env);
     }
 
diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml
index 19db6c9..ff939ce 100644
--- a/lib/crypto/doc/src/crypto.xml
+++ b/lib/crypto/doc/src/crypto.xml
@@ -643,16 +643,14 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
 
     <func>
       <name>aes_cfb_128_encrypt(Key, IVec, Text) -> Cipher</name>
-      <name>aes_cbc_128_encrypt(Key, IVec, Text) -> Cipher</name>
-      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Feedback  mode or Cipher Block Chaining mode</fsummary>
+      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Feedback mode</fsummary>
       <type>
         <v>Key = Text = iolist() | binary()</v>
         <v>IVec = Cipher = binary()</v>
       </type>
       <desc>
         <p>Encrypts <c>Text</c> according to AES in Cipher Feedback 
-          mode (CFB) or Cipher Block Chaining mode (CBC). <c>Text</c>
-          must be a multiple of 128 bits (16 bytes). <c>Key</c> is the
+          mode (CFB).<c>Key</c> is the
           AES key, and <c>IVec</c> is an arbitrary initializing vector.
           The lengths of <c>Key</c> and <c>IVec</c> must be 128 bits
           (16 bytes).</p>
@@ -660,15 +658,45 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
     </func>
     <func>
       <name>aes_cfb_128_decrypt(Key, IVec, Cipher) -> Text</name>
+      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Feedback mode</fsummary>
+      <type>
+        <v>Key = Cipher = iolist() | binary()</v>
+        <v>IVec = Text = binary()</v>
+      </type>
+      <desc>
+        <p>Decrypts <c>Cipher</c> according to AES in Cipher Feedback Mode (CFB).
+          <c>Key</c> is the AES key, and <c>IVec</c> is an arbitrary
+          initializing vector. <c>Key</c> and <c>IVec</c> must have
+          the same values as those used when encrypting. The lengths of
+          <c>Key</c> and <c>IVec</c> must be 128 bits (16 bytes).</p>
+      </desc>
+    </func>
+    <func>
+      <name>aes_cbc_128_encrypt(Key, IVec, Text) -> Cipher</name>
+      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Block Chaining mode</fsummary>
+      <type>
+        <v>Key = Text = iolist() | binary()</v>
+        <v>IVec = Cipher = binary()</v>
+      </type>
+      <desc>
+          <p>Encrypts <c>Text</c> according to AES in Cipher Block Chaining
+          mode (CBC). <c>Text</c>
+          must be a multiple of 128 bits (16 bytes). <c>Key</c> is the
+          AES key, and <c>IVec</c> is an arbitrary initializing vector.
+          The lengths of <c>Key</c> and <c>IVec</c> must be 128 bits
+          (16 bytes).</p>
+      </desc>
+    </func>
+    <func>
       <name>aes_cbc_128_decrypt(Key, IVec, Cipher) -> Text</name>
-      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Feedback  mode or Cipher Block Chaining mode</fsummary>
+      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Block Chaining mode</fsummary>
       <type>
         <v>Key = Cipher = iolist() | binary()</v>
         <v>IVec = Text = binary()</v>
       </type>
       <desc>
-        <p>Decrypts <c>Cipher</c> according to Cipher Feedback Mode (CFB)
-          or Cipher Block Chaining mode (CBC).
+          <p>Decrypts <c>Cipher</c> according to AES in Cipher Block
+          Chaining mode (CBC).
           <c>Key</c> is the AES key, and <c>IVec</c> is an arbitrary
           initializing vector. <c>Key</c> and <c>IVec</c> must have
           the same values as those used when encrypting. <c>Cipher</c>
-- 
1.7.9.5



-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From henrik@REDACTED  Wed Jul  4 14:10:28 2012
From: henrik@REDACTED (Henrik Nord)
Date: Wed, 4 Jul 2012 14:10:28 +0200
Subject: [erlang-patches] [erlang-bugs] www.erlang.org not responding
In-Reply-To: <1CD1916B-14C7-403D-ABD4-EAAF11A04573@feuerlabs.com>
References: <1CD1916B-14C7-403D-ABD4-EAAF11A04573@feuerlabs.com>
Message-ID: <4FF432B4.3090807@erlang.org>


Hello

The site was down due to a backup-script that went in to a tailspin.

The issue is now resolved and everything is back up!


-- 
/Henrik Nord Erlang/OTP



From stefan.zegenhagen@REDACTED  Wed Jul  4 16:42:04 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Wed, 04 Jul 2012 16:42:04 +0200
Subject: [erlang-patches] SNMPD: bugfix to snmp_generic:split_index_to_keys/2
Message-ID: <1341412924.26870.2.camel@ax-sze>

Dear all,

this mail contains a patch that solves a problem with the indicated
function. For dynamically-sized table indices it wasn't checking whether
the length of the index value matches all constraints specified in the
MIB.

Kind regards,

--- snip ---

>From 12c80f51879da4d36019cf736c1009e5bc09b908 Mon Sep 17 00:00:00 2001
From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
Date: Wed, 4 Jul 2012 14:10:00 +0200
Subject: [PATCH 1/2] fix snmp_generic:split_index_to_keys/2

The function snmp_generic:split_index_to_keys/2 handles dynamic length
RowIndex members, but fails to check that their length actually falls
within the constraints given by the MIB.

The problem becomes real for table indices having an 'OCTET STRING'
syntax, e.g. OCTET STRING(1..32) where it is possible to pass
zero-length strings or strings longer than 32 chars without any
validation failure.
---
 lib/snmp/src/agent/snmp_generic.erl |   20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/lib/snmp/src/agent/snmp_generic.erl b/lib/snmp/src/agent/snmp_generic.erl
index 06afa68..0dd1f8b 100644
--- a/lib/snmp/src/agent/snmp_generic.erl
+++ b/lib/snmp/src/agent/snmp_generic.erl
@@ -640,11 +640,12 @@ collect_keys([#asn1_type{lo = X, hi = X} | _Indexes], Keys)
 %% Otherwise, its a dynamic-length type => its a list
 %% OBJECT IDENTIFIER, OCTET STRING or BITS (or derivatives)
 %% Check if it is IMPLIED (only last element can be IMPLIED)
-collect_keys([#asn1_type{implied = true}], Keys) ->
-    [Keys];
-collect_keys([_Type | Indexes], [Length | Keys]) when length(Keys) >= Length ->
+%% and also check lo/hi constraints...
+collect_keys([#asn1_type{implied = true} = Type], Keys) ->
+    [collect_check_length(Type, Keys)];
+collect_keys([Type | Indexes], [Length | Keys]) when length(Keys) >= Length ->
     {StrKey, Rest} = collect_length(Length, Keys, []),
-    [StrKey | collect_keys(Indexes, Rest)];
+    [collect_check_length(Type, StrKey) | collect_keys(Indexes, Rest)];
 collect_keys([_Type | _Indexes], [Length | Keys]) ->
     exit({error, {size_mismatch, Length, Keys}});
 collect_keys([], []) -> [];
@@ -657,6 +658,17 @@ collect_length(0, Rest, Rts) ->
 collect_length(N, [El | Rest], Rts) ->
     collect_length(N-1, Rest, [El | Rts]).
 
+collect_check_length(#asn1_type{lo = Lo, hi = Hi}, ListVal) ->
+    Length = length(ListVal),
+    if
+        is_integer(Lo) andalso Length < Lo ->
+            exit({error, {size_mismatch, Lo, ListVal}});
+        is_integer(Hi) andalso Length > Hi ->
+            exit({error, {size_mismatch, Hi, ListVal}});
+        true ->
+            ListVal
+    end.
+
 %%------------------------------------------------------------------
 %% Checks if a certain row exists.
 %% Returns true or false.
-- 
1.7.9.5


-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From stefan.zegenhagen@REDACTED  Wed Jul  4 16:53:30 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Wed, 04 Jul 2012 16:53:30 +0200
Subject: [erlang-patches] SNMP: verify RowIndex in table set requests
Message-ID: <1341413610.26870.10.camel@ax-sze>

Dear all,

during intensive testing it was found that it's possible to create table
rows in snmpTargetAddrTable where the row index (snmpTargetAddrName ::
SnmpAdminString(1..32)) is the empty string. This patch solves the issue
generically by checking for all set-requests that the row index is of
acceptable length before calling the table instrumentation function.

The same is not done for get requests because such invalid row indices
should not be present if it's impossible to create them.

The same is not done for get-next requests because the semantic of the
operation requires it to be able to cope with incomplete/overlong row
indices.

Kind regards,


--- snip ---

>From be617c829a89dd9b73768181810776e50fb36c42 Mon Sep 17 00:00:00 2001
From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
Date: Wed, 4 Jul 2012 14:34:46 +0200
Subject: [PATCH 2/2] verify table row index constraints in SNMP set requests

When a set-request is received for table columns, check that the varbind
OIDs are syntactically correct. This includes checking that Table Entry
OID as well as the Column OID are present, and that the row index
contains an acceptable number of subidentifiers.

This check was not done carefully enough before, giving the following
test result from an erlang-snmp agent:

 Response: Version = SNMPv3, MsgId = 173, MaxSize = 2048
 Flags = Auth/Priv, SecurityModel = USM
 EngineId = [...], EngineBoots = 16, EngineTime = 81244, UserName = ...
 ContextEngineId = [...], ContextName = [...]
 Request Id = 173, Error Status = No Error , Error Index = 0
 Oid1 = snmpTargetAddrRowStatus , Type = Integer, Value = 4
 Oid2 = snmpTargetAddrTAddress , Type = OctetString, Value = ...
 Oid3 = snmpTargetAddrTimeout , Type = Integer, Value = 100
 Oid4 = snmpTargetAddrRetryCount , Type = Integer, Value = 3
 Oid5 = snmpTargetAddrTagList , Type = OctetString, Value = abc
 Oid6 = snmpTargetAddrParams , Type = OctetString, Value = abc
 Oid7 = snmpTargetAddrStorageType , Type = Integer, Value = 2
 Oid8 = snmpTargetAddrTDomain , Type = ObjectID, Value = 1.3.6.1.6.1.1

Please observe that a table row in snmptargetAddrParams is created
without errors, although an empty RowIndex was specified for all
columnar objects in the request. The index to that table is defined as
IMPLIED SnmpAdminString(1..32).
---
 lib/snmp/src/agent/snmpa_set_lib.erl |   65 ++++++++++++++++++++++++++++++++--
 1 file changed, 63 insertions(+), 2 deletions(-)

diff --git a/lib/snmp/src/agent/snmpa_set_lib.erl b/lib/snmp/src/agent/snmpa_set_lib.erl
index f5218d5..ef9e20e 100644
--- a/lib/snmp/src/agent/snmpa_set_lib.erl
+++ b/lib/snmp/src/agent/snmpa_set_lib.erl
@@ -70,8 +70,19 @@
 %% Purpose: Call is_varbind_ok for each varbind
 %% Returns: {noError, 0} | {ErrorStatus, ErrorIndex}
 %%-----------------------------------------------------------------
-is_varbinds_ok([{_TableOid, TableVbs} | IVarbinds]) -> 
-    is_varbinds_ok(lists:append(TableVbs, IVarbinds));
+is_varbinds_ok([{TableOid, TableVbs} | IVarbinds]) ->
+    % TableOid points to the table entry, not the table itself.
+    % for table row index checking, we need the table entry, though.
+    % Hence, strip the last OID.
+    NTableOid = lists:reverse(tl(lists:reverse(TableOid))),
+    case is_table_varbinds_ok(NTableOid, TableVbs) of
+        true ->
+            is_varbinds_ok(lists:append(TableVbs, IVarbinds));
+        {ErrorStatus, Varbind} ->
+            ?vtrace("varbinds erroneous: ~p -> ~p",
+                [Varbind#varbind.org_index,ErrorStatus]),
+            {ErrorStatus, Varbind#varbind.org_index}
+    end;
 is_varbinds_ok([IVarbind | IVarbinds]) -> 
     case catch is_varbind_ok(IVarbind) of
 	true -> is_varbinds_ok(IVarbinds);
@@ -85,6 +96,56 @@ is_varbinds_ok([]) ->
     ?vtrace("varbinds ok",[]),
     {noError, 0}.
 
+
+%%-----------------------------------------------------------------
+%% Func: is_table_varbinds_ok/2
+%% Purpose: Perform basic checks on the RowIndex of varbinds against
+%%          the MIB. Here we don't call any instrumentation functions.
+%% Returns: true |
+%% Fails:   with an <error-atom>.
+%%-----------------------------------------------------------------
+is_table_varbinds_ok(_TableOid, []) ->
+    true;
+is_table_varbinds_ok(TableOid, [Hd | _] = Varbinds) ->
+    ErrResult = {noCreation, Hd#ivarbind.varbind},
+    % resolve table information, it's required for the checks we do
+    case snmpa_symbolic_store:oid_to_aliasname(TableOid) of
+        {value, TableName} ->
+            case snmpa_symbolic_store:table_info(TableName) of
+                {value, #table_info{} = TableInfo} ->
+                    is_table_varbinds_ok(TableOid, TableInfo, Varbinds);
+                false ->
+                    snmpa_error:user_err("Failed in table_info(~w)", [TableName]),
+                    ErrResult
+            end;
+        false ->
+            snmpa_error:user_err("Failed in oid_to_aliasname(~w)", [TableOid]),
+            ErrResult
+    end.
+
+is_table_varbinds_ok(TableOid, TableInfo, [#ivarbind{varbind = Varbind} | Varbinds]) ->
+    Indices = TableInfo#table_info.index_types,
+    Oid = Varbind#varbind.oid,
+    case snmp_misc:diff(Oid, TableOid) of
+        [_TblEntryId, _TblColumnId | RowIndex] ->
+            case (catch snmp_generic:split_index_to_keys(Indices, RowIndex)) of
+                Keys when is_list(Keys) ->
+                    % RowIndex seems to be OK
+                    is_table_varbinds_ok(TableOid, TableInfo, Varbinds);
+                Error ->
+                    % RowIndex is syntactically wrong...
+                    ?vtrace("table varbind RowIndex erroneous: ~p", [Error]),
+                    {noCreation, Varbind}
+            end;
+        _Else ->
+            % OID is syntactically wrong...
+            ?vtrace("table varbind OID erroneous: ~p", [Oid]),
+            {noCreation, Varbind}
+    end;
+is_table_varbinds_ok(_TableOid, _TableInfo, []) ->
+    true.
+
+
 %%-----------------------------------------------------------------
 %% Func: is_varbind_ok/1
 %% Purpose: Check everything we can check about the varbind against
-- 
1.7.9.5


-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From tuncer.ayaz@REDACTED  Thu Jul  5 21:41:56 2012
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Thu, 5 Jul 2012 21:41:56 +0200
Subject: [erlang-patches] erts,
	kernel: make internal fun easier to use correctly
Message-ID: <CAOvwQ4h5Kx8DwiS0V-2W9wCHkORC8qAsVNp=mgZTVvkYOOzEQg@mail.gmail.com>

As discussed with Lukas Larsson, make the internal socket
locking function easier to use correctly.

git fetch git://github.com/tuncer/otp.git inet-lock_socket

https://github.com/tuncer/otp/compare/inet-lock_socket
https://github.com/tuncer/otp/compare/inet-lock_socket.patch


From carlsson.richard@REDACTED  Thu Jul  5 23:17:35 2012
From: carlsson.richard@REDACTED (Richard Carlsson)
Date: Thu, 05 Jul 2012 23:17:35 +0200
Subject: [erlang-patches] break out assert macros from eunit
Message-ID: <4FF6046F.70705@gmail.com>

Several people have requested that the assert macros in eunit should be 
moved out to a separate header file. The following branch puts them in 
stdlib/include/assert.hrl:

   git fetch git://github.com/richcarl/otp.git break-out-asserts

I added a test, but there is no documentation: I don't know quite where 
to put that, since it's just a header file and no module. However, they 
are described in the EUnit documentation.

    /Richard


From tuncer.ayaz@REDACTED  Fri Jul  6 14:37:49 2012
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Fri, 6 Jul 2012 14:37:49 +0200
Subject: [erlang-patches] erts,
	kernel: make internal fun easier to use correctly
In-Reply-To: <CAOvwQ4h5Kx8DwiS0V-2W9wCHkORC8qAsVNp=mgZTVvkYOOzEQg@mail.gmail.com>
References: <CAOvwQ4h5Kx8DwiS0V-2W9wCHkORC8qAsVNp=mgZTVvkYOOzEQg@mail.gmail.com>
Message-ID: <CAOvwQ4jFjOh2T_DZ+vdT+EOQyPAyJoSA4D6L84tgX2X3fC8_sg@mail.gmail.com>

On Thu, Jul 5, 2012 at 9:41 PM, Tuncer Ayaz wrote:
> As discussed with Lukas Larsson, make the internal socket
> locking function easier to use correctly.
>
> git fetch git://github.com/tuncer/otp.git inet-lock_socket
>
> https://github.com/tuncer/otp/compare/inet-lock_socket
> https://github.com/tuncer/otp/compare/inet-lock_socket.patch

Henrik, please ignore this patch.

After some reflection and discussion with Lukas we've decided
to not apply this.

Thanks.


From sverker.eriksson@REDACTED  Fri Jul  6 14:55:45 2012
From: sverker.eriksson@REDACTED (Sverker Eriksson)
Date: Fri, 6 Jul 2012 14:55:45 +0200
Subject: [erlang-patches] error in crypto application
In-Reply-To: <1341309553.3506.67.camel@ax-sze>
References: <1340874032.3506.2.camel@ax-sze>
 <4FEC6825.2060508@erix.ericsson.se> <1341234194.3506.30.camel@ax-sze>
 <1341309553.3506.67.camel@ax-sze>
Message-ID: <4FF6E051.5010205@erix.ericsson.se>

Thanks for your contribution, Stefan.

This latest patch (plus an expanded test case) is now scheduled for R15B02.

/Sverker

Stefan Zegenhagen wrote:
> Dear all,
>
> here, comes a third version of the patch that
>       * fixes the aes_cfb_128_crypt() function to not require the
>         plaintext/cipher data to be a multiple of 16 bytes in length
>       * adjusts the documentation accordingly, and
>       * leaves the aes_cbc_crypt() function untouched because its
>         current API in erlang requires the plaintext/cipher data to be a
>         multiple of 16 bytes.
>
>
> Kind regards,
>
>
> --- snip ---
> >From c9f86ea4a2244deeae4efd2955b329857d91e340 Mon Sep 17 00:00:00 2001
> From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
> Date: Mon, 2 Jul 2012 22:00:09 +0200
> Subject: [PATCH] CRYPTO: aes_cfb_128_crypt can handle arbitrary text length
>
> The OpenSSL implementation of AES_cfb128_encrypt can handle data to
> en-/decrypt with arbitrary length.
>
> The restriction that the data length needs to be a multiple of 16 bytes
> is unnecessary and breaks the SNMP application (causing decryption and
> encryption errors in SNMPv3 AES-encrypted packets).
> ---
>  lib/crypto/c_src/crypto.c     |    3 +--
>  lib/crypto/doc/src/crypto.xml |   42 ++++++++++++++++++++++++++++++++++-------
>  2 files changed, 36 insertions(+), 9 deletions(-)
>
> diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c
> index 4be593e..a6a81d6 100644
> --- a/lib/crypto/c_src/crypto.c
> +++ b/lib/crypto/c_src/crypto.c
> @@ -954,8 +954,7 @@ static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TE
>  
>      if (!enif_inspect_iolist_as_binary(env, argv[0], &key) || key.size != 16
>  	|| !enif_inspect_binary(env, argv[1], &ivec) || ivec.size != 16
> -	|| !enif_inspect_iolist_as_binary(env, argv[2], &text)
> -	|| text.size % 16 != 0) {
> +	|| !enif_inspect_iolist_as_binary(env, argv[2], &text)) {
>  	return enif_make_badarg(env);
>      }
>  
> diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml
> index 19db6c9..ff939ce 100644
> --- a/lib/crypto/doc/src/crypto.xml
> +++ b/lib/crypto/doc/src/crypto.xml
> @@ -643,16 +643,14 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
>  
>      <func>
>        <name>aes_cfb_128_encrypt(Key, IVec, Text) -> Cipher</name>
> -      <name>aes_cbc_128_encrypt(Key, IVec, Text) -> Cipher</name>
> -      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Feedback  mode or Cipher Block Chaining mode</fsummary>
> +      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Feedback mode</fsummary>
>        <type>
>          <v>Key = Text = iolist() | binary()</v>
>          <v>IVec = Cipher = binary()</v>
>        </type>
>        <desc>
>          <p>Encrypts <c>Text</c> according to AES in Cipher Feedback 
> -          mode (CFB) or Cipher Block Chaining mode (CBC). <c>Text</c>
> -          must be a multiple of 128 bits (16 bytes). <c>Key</c> is the
> +          mode (CFB).<c>Key</c> is the
>            AES key, and <c>IVec</c> is an arbitrary initializing vector.
>            The lengths of <c>Key</c> and <c>IVec</c> must be 128 bits
>            (16 bytes).</p>
> @@ -660,15 +658,45 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]>
>      </func>
>      <func>
>        <name>aes_cfb_128_decrypt(Key, IVec, Cipher) -> Text</name>
> +      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Feedback mode</fsummary>
> +      <type>
> +        <v>Key = Cipher = iolist() | binary()</v>
> +        <v>IVec = Text = binary()</v>
> +      </type>
> +      <desc>
> +        <p>Decrypts <c>Cipher</c> according to AES in Cipher Feedback Mode (CFB).
> +          <c>Key</c> is the AES key, and <c>IVec</c> is an arbitrary
> +          initializing vector. <c>Key</c> and <c>IVec</c> must have
> +          the same values as those used when encrypting. The lengths of
> +          <c>Key</c> and <c>IVec</c> must be 128 bits (16 bytes).</p>
> +      </desc>
> +    </func>
> +    <func>
> +      <name>aes_cbc_128_encrypt(Key, IVec, Text) -> Cipher</name>
> +      <fsummary>Encrypt <c>Text</c>according to AES in Cipher Block Chaining mode</fsummary>
> +      <type>
> +        <v>Key = Text = iolist() | binary()</v>
> +        <v>IVec = Cipher = binary()</v>
> +      </type>
> +      <desc>
> +          <p>Encrypts <c>Text</c> according to AES in Cipher Block Chaining
> +          mode (CBC). <c>Text</c>
> +          must be a multiple of 128 bits (16 bytes). <c>Key</c> is the
> +          AES key, and <c>IVec</c> is an arbitrary initializing vector.
> +          The lengths of <c>Key</c> and <c>IVec</c> must be 128 bits
> +          (16 bytes).</p>
> +      </desc>
> +    </func>
> +    <func>
>        <name>aes_cbc_128_decrypt(Key, IVec, Cipher) -> Text</name>
> -      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Feedback  mode or Cipher Block Chaining mode</fsummary>
> +      <fsummary>Decrypt <c>Cipher</c>according to AES in Cipher Block Chaining mode</fsummary>
>        <type>
>          <v>Key = Cipher = iolist() | binary()</v>
>          <v>IVec = Text = binary()</v>
>        </type>
>        <desc>
> -        <p>Decrypts <c>Cipher</c> according to Cipher Feedback Mode (CFB)
> -          or Cipher Block Chaining mode (CBC).
> +          <p>Decrypts <c>Cipher</c> according to AES in Cipher Block
> +          Chaining mode (CBC).
>            <c>Key</c> is the AES key, and <c>IVec</c> is an arbitrary
>            initializing vector. <c>Key</c> and <c>IVec</c> must have
>            the same values as those used when encrypting. <c>Cipher</c>
>   



From henrik@REDACTED  Fri Jul  6 15:12:14 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 6 Jul 2012 15:12:14 +0200
Subject: [erlang-patches] Mnesia index becomes corrupted if deleting an
 nonexisting object
In-Reply-To: <0991fe2b-5d0a-4fd1-98ff-c64e4ea818e8@knuth>
References: <0991fe2b-5d0a-4fd1-98ff-c64e4ea818e8@knuth>
Message-ID: <4FF6E42E.8050001@erlang.org>

Thank you for your contribution, I have added your patch to 'pu'

On 06/28/2012 09:22 PM, Bart?omiej Puzo? wrote:
> Dear OTP team,
>
> ======= Description
>
> Let's have the following record:
> -record(r, {key,index,data}).
>
> test()->                                                                                                                               mnesia:delete_table(r),
>   mnesia:create_table(r, [{type, bag},
>                           {attributes, record_info(fields,r)}]),
>   mnesia:add_table_index(r, index),
>
>   %% Lets insert a random record to our table
>   Record1 = #r{key = k, index = i, data = 1},
>   mnesia:dirty_write(r, Record1),
>
>   %% This should return [{r,k,i,1}] and it does
>   mnesia:dirty_index_read(r, i, #r.index)
>
>   %% Now lets try to delete something that is not in the table, but
>   %% shares the same key and index
>   Record2 = #r{key = k, index = i, data = 2},
>   mnesia:dirty_delete_object(r, Record2),
>
>   %% And now our big finale. This call, which is exactly the same
>   %% as the previous index read, should give the same results
>   mnesia:dirty_index_read(r, i, #r.index).
>   %% BUT it returns [] instead. Somehow the index got changed even thoug
>   %% we never touched the table itself
>
> After a period of witch-hunting the culprit turned out to be hiding in mnesia_index:
>
> 119 del_object_bag(Tab, Key, Obj, Pos, Ixt, undefined) ->
> 120    IxKey = element(Pos, Obj),
> 121    Old = [X || X<-  mnesia_lib:db_get(Tab, Key), element(Pos, X) =:= IxKey],
> 122    del_object_bag(Tab, Key, Obj, Pos, Ixt, Old);
> 123 %% If Tab type is bag we need remove index identifier if Tab
> 124 %% contains less than 2 elements.
> 125 del_object_bag(_Tab, Key, Obj, Pos, Ixt, Old) when length(Old)<  2 ->
> 126     del_ixes(Ixt, [Obj], Pos, Key);
> 127 del_object_bag(_Tab, _Key, _Obj, _Pos, _Ixt, _Old) ->  ok.
>
> Our call to dirty_delete_object results in mnesia_index:del_object_bag being called as follows:
>      mnesia_index:del_object_bag(r,k,{r,k,i,2},3,{ram,14221431},[{r,k,i,1}])
> As you can see, the object we are deleting and the object that is in the table (last argument) are
> different. However, mnesia_index still happily deletes the index.
>
>
> ======= Links
> Github: bpuzon/otp, branch: fix_mnesia_index_drop
> Git command: git fetch git://github.com/bpuzon/otp.git fix_mnesia_index_drop
> https://github.com/bpuzon/otp/compare/fix_mnesia_index_drop
> https://github.com/bpuzon/otp/compare/fix_mnesia_index_drop.patch
>
>
> Bart?omiej Puzo?
> Erlang Solutions
>
>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Fri Jul  6 15:17:23 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 6 Jul 2012 15:17:23 +0200
Subject: [erlang-patches] [PATCH] Zip archives disk numbering
In-Reply-To: <028E625E-3B6E-4941-9764-40C94F69CA1A@kallisys.net>
References: <028E625E-3B6E-4941-9764-40C94F69CA1A@kallisys.net>
Message-ID: <4FF6E563.4040907@erlang.org>

Thank you

I grabbed

git fetch git://github.com/pguyot/otp.git fix-zip-multidisk

;)



On 06/18/2012 03:26 PM, Paul Guyot wrote:
> Hello,
>
> There is a bug in zip.erl: archives are wrongly created with the central directory's disk_num_start set to 1. The specification is not extremely clear about the base number of this value, however zip.erl uses 0 everywhere else for the disk number. Besides, this bug causes some incompatibility with other software that insists in reading 0 as disk_num_start or otherwise consider that the archive is multidisk. This is the case with Microsoft Office applications. For example, the following code tests for 0:
> http://code.google.com/p/xmind-merger/source/browse/OpenXmlManager/Internal/IO/Zip/ZipIOCentralDirectoryFileHeader.cs#204
>
> You will find a patch against 'maint' here:
> git fetch git://github.com/pguyot/otp.git fix-multidisk-zip
> https://github.com/pguyot/otp/commit/da11eb78dac39940edf4dcfbb47b438493345598
>
> This patch simply fixes the single buggy line. It does not include a test. Indeed, I am confused about the best way to test this fix. zip_SUITE tests in maint branch are mostly about extracting files from zip archives, and there is no comparison or binary matching of the output of zip:create/{2,3} function. If a test is required for inclusion, please let me know the best approach for designing such a test, and I will gladly provide an amended patch.
>
> Regards,
>
> Paul

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Fri Jul  6 15:19:51 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 6 Jul 2012 15:19:51 +0200
Subject: [erlang-patches] break out assert macros from eunit
In-Reply-To: <4FF6046F.70705@gmail.com>
References: <4FF6046F.70705@gmail.com>
Message-ID: <4FF6E5F7.3090507@erlang.org>

I have included this in 'pu'

Thank you for the contribution!

On 07/05/2012 11:17 PM, Richard Carlsson wrote:
>   git fetch git://github.com/richcarl/otp.git break-out-asserts 

-- 
/Henrik Nord Erlang/OTP



From cyg.cao@REDACTED  Mon Jul  9 08:47:14 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Mon, 9 Jul 2012 14:47:14 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to scheduler
Message-ID: <CADS4Raz9LWLJOEaNRTGQdJe2rMYpAT_Wt2KFYD5bEBpBa5dzXw@mail.gmail.com>

In our proxy project which has massive network IO, we found
erts_check_io()  become system bottleneck,
a call to erts_check_io() would take 500 microseconds to complete, and
invoked about 3.5k times per second
under heavy preesure, which resulted in the average response time of
user request longer than 2 milliseconds,
so that limited system throughput down, while cpus were idle(in
starvation) at the same time.

This patch allocates one pollset for each scheduler, so that each
scheduler could invoke erts_check_io()
on its own pollset concurrently. To be incorporate well with port
migration, all used fd in each port're recorded,
removed from old scheduler's pollset and added to new scheduler's when
port is migrating between schedulers.

After applying this patch, together with binding process&port with
schedulers, erts_check_io() is invoked about 230k
times per second (dozens of times more than before), and throughtput
increases from 45k to 105k, so it works.

This patch SHOULD be compiled with ./configure
CFLAGS=-DERTS_POLLSET_PER_SCHEDULER

Pollset per scheduler:

git fetch git://github.com/weicao/otp.git pollset_per_scheduler

https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler
https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch


Bind port to scheduler:

git fetch git://github.com/weicao/otp.git bind_port_to_scheduler

https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler
https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch



--

Best,

Wei Cao


From lukas@REDACTED  Mon Jul  9 15:05:02 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Mon, 9 Jul 2012 15:05:02 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Raz9LWLJOEaNRTGQdJe2rMYpAT_Wt2KFYD5bEBpBa5dzXw@mail.gmail.com>
References: <CADS4Raz9LWLJOEaNRTGQdJe2rMYpAT_Wt2KFYD5bEBpBa5dzXw@mail.gmail.com>
Message-ID: <4FFAD6FE.6060300@erlang.org>

Hi,

Thanks a lot for these patches!

Multiple poll-sets is something which we have been thinking about doing 
for a while now, but haven't as yet gotten around to do. So it is great 
that you have taken the time to implement it!

What is the reason for not enabling the multiple poll-set functionality 
by default? Have you seen any scenarios where it is unwanted? Or 
platforms which the approach does not work?

When compiling the multiple poll-set patch on Windows we got an error[1] 
because the compiler we use there does not allow empty structs. Do you 
have a windows environment available to test your patch on? For 
instructions on how to set up a test environment on windows see[2]. We 
really want this patch, but have to make sure that it works properly on 
all platforms as it is a core change to how erts handler fd polling.

When the Windows compile issue is fixed, we will include this patch in 
our daily builds and let you know of any test failures which we find.

Regarding binding a port to a scheduler, we are unsure about building 
more functionality on top of the internal `scheduler` option as it 
effects the soft-realtime properties of the system. Generally we do not 
want to add functionality which exposes the internal scheduler semantics 
to the programmer, but rather make the scheduler more intelligent about 
where they schedule tasks. We'll have to discuss internally what we want 
to do with this.

Thanks!
Lukas

[1]: Windows 7, mingw, vs 10 compiler

cc.sh   -O3 -I/c/jenkins/workspace/compile-master-opu-win32/erts/win32   -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -DERTS_MIXED_MSYS_VC -DSTATIC_ERLANG_DRIVER  -DHAVE_CONFIG_H  -DUSE_THREADS -DWIN32_THREADS  -Iwin32/opt/plain -Ibeam -Isys/win32 -Isys/common -Iwin32 -Izlib  -Ipcre -Ihipe -I../include -I../include/win32 -I../include/internal -I../include/internal/win32 -c sys/common/erl_check_io.c -o obj/win32/opt/plain/erl_check_io.o
sys/common/erl_check_io.c(99) : error C2016: C requires that a struct or union has at least one member

[2]: http://www.erlang.org/doc/installation_guide/INSTALL-WIN32.html

On 09/07/12 08:47, Wei Cao wrote:
> In our proxy project which has massive network IO, we found
> erts_check_io()  become system bottleneck,
> a call to erts_check_io() would take 500 microseconds to complete, and
> invoked about 3.5k times per second
> under heavy preesure, which resulted in the average response time of
> user request longer than 2 milliseconds,
> so that limited system throughput down, while cpus were idle(in
> starvation) at the same time.
>
> This patch allocates one pollset for each scheduler, so that each
> scheduler could invoke erts_check_io()
> on its own pollset concurrently. To be incorporate well with port
> migration, all used fd in each port're recorded,
> removed from old scheduler's pollset and added to new scheduler's when
> port is migrating between schedulers.
>
> After applying this patch, together with binding process&port with
> schedulers, erts_check_io() is invoked about 230k
> times per second (dozens of times more than before), and throughtput
> increases from 45k to 105k, so it works.
>
> This patch SHOULD be compiled with ./configure
> CFLAGS=-DERTS_POLLSET_PER_SCHEDULER
>
> Pollset per scheduler:
>
> git fetch git://github.com/weicao/otp.git pollset_per_scheduler
>
> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler
> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch
>
>
> Bind port to scheduler:
>
> git fetch git://github.com/weicao/otp.git bind_port_to_scheduler
>
> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler
> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch
>
>
>
> --
>
> Best,
>
> Wei Cao
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches
>


From carlsson.richard@REDACTED  Mon Jul  9 20:58:59 2012
From: carlsson.richard@REDACTED (Richard Carlsson)
Date: Mon, 09 Jul 2012 20:58:59 +0200
Subject: [erlang-patches] Fix the erlc -MP flag
Message-ID: <4FFB29F3.1000801@gmail.com>

I posted this to erlang-bugs before, but got no response, so here's a 
new attempt. This time, the patch also has a test case courtesy of Uwe 
Dauernheim.

Because of a copy-and-paste error in erlc.c, the -MP flag had the same 
effect as -MG. As a workaround, you had to pass +makedep_phony to enable 
the MP option. This patch makes -MP work as intended.

     git fetch git://github.com/richcarl/otp.git fix-MP-compiler-flag


   /Richard


From henrik@REDACTED  Tue Jul 10 15:00:25 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 10 Jul 2012 15:00:25 +0200
Subject: [erlang-patches] extended back references in re:replace/3,4
In-Reply-To: <20120404134729.GI14994@aluminum.wavenet.lk>
References: <20120308155816.GI22798@aluminum.wavenet.lk>
 <20120404134729.GI14994@aluminum.wavenet.lk>
Message-ID: <4FFC2769.4060903@erlang.org>

Thank you for you contribution, I have added your patch in 'pu'

On 04/04/2012 03:47 PM, Vance Shipley wrote:
> I created a patch to resolve the use case probem I have with
> the current implementation of re:replace/3,4 with regard to
> using back references on numeric digit strings:
>
> 	git fetch git://github.com/vances/otp.git re_back_reference
>
> 	https://github.com/vances/otp/compare/re_back_reference
> 	https://github.com/vances/otp/compare/re_back_reference
>
> Test cases and updated the documentation are included.
>
> I hope you'll accept it for inclusion in a future release.
>

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Tue Jul 10 15:01:13 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 10 Jul 2012 15:01:13 +0200
Subject: [erlang-patches] Fix the erlc -MP flag
In-Reply-To: <4FFB29F3.1000801@gmail.com>
References: <4FFB29F3.1000801@gmail.com>
Message-ID: <4FFC2799.6090500@erlang.org>

Thank you!

On 07/09/2012 08:58 PM, Richard Carlsson wrote:
> I posted this to erlang-bugs before, but got no response, so here's a 
> new attempt. This time, the patch also has a test case courtesy of Uwe 
> Dauernheim.
>
> Because of a copy-and-paste error in erlc.c, the -MP flag had the same 
> effect as -MG. As a workaround, you had to pass +makedep_phony to 
> enable the MP option. This patch makes -MP work as intended.
>
>     git fetch git://github.com/richcarl/otp.git fix-MP-compiler-flag
>
>
>   /Richard
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Tue Jul 10 15:07:31 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 10 Jul 2012 15:07:31 +0200
Subject: [erlang-patches] Fix rpc:call/5 for local calls with a finite
 Timeout
In-Reply-To: <CAFOzqpFxUzcbqb1jcVUZypzCHyhA8rm9O28NkUtXt0cZXUT3cw@mail.gmail.com>
References: <CAFOzqpEpuAmQkO_omy6viUUMNnCyVECA+yBxykVSaj5dNnQM9w@mail.gmail.com>
 <CAFOzqpFxUzcbqb1jcVUZypzCHyhA8rm9O28NkUtXt0cZXUT3cw@mail.gmail.com>
Message-ID: <4FFC2913.9070807@erlang.org>

This was Added in pu:7/5 -12
Thank you for the contribution!

On 05/04/2012 01:16 PM, Tomer Chachamu wrote:
> The rpc testcases don't include any local calls, unfortunately.
>
> The patch:
>
> git fetch git://github.com/r3m0t/otp.git rpc-call-fix
>
> https://github.com/r3m0t/otp/compare/rpc-call-fix
> https://github.com/r3m0t/otp/compare/rpc-call-fix.patch
>
> [assuming my previous copy of this message was silently dropped as I
> wasn't subscribed to the list at the time]
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From cyg.cao@REDACTED  Wed Jul 11 04:21:02 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 11 Jul 2012 10:21:02 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
Message-ID: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>

I added a macro to conditional compile the patch because I think it
can be more selectable, I can remove the macro, fix the compilation
error and test on mingw platform in later version.

how about provide another BIF named port_flag (like process_flag) to
let user bind port to a given scheduler?


-- 

Best,

Wei Cao

> Message: 1
> Date: Mon, 9 Jul 2012 15:05:02 +0200
> From: Lukas Larsson <lukas@REDACTED>
> To: erlang-patches@REDACTED
> Subject: Re: [erlang-patches] Pollset per scheduler and bind port to
>         scheduler
> Message-ID: <4FFAD6FE.6060300@REDACTED>
> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
>
> Hi,
>
> Thanks a lot for these patches!
>
> Multiple poll-sets is something which we have been thinking about doing
> for a while now, but haven't as yet gotten around to do. So it is great
> that you have taken the time to implement it!
>
> What is the reason for not enabling the multiple poll-set functionality
> by default? Have you seen any scenarios where it is unwanted? Or
> platforms which the approach does not work?
>
> When compiling the multiple poll-set patch on Windows we got an error[1]
> because the compiler we use there does not allow empty structs. Do you
> have a windows environment available to test your patch on? For
> instructions on how to set up a test environment on windows see[2]. We
> really want this patch, but have to make sure that it works properly on
> all platforms as it is a core change to how erts handler fd polling.
>
> When the Windows compile issue is fixed, we will include this patch in
> our daily builds and let you know of any test failures which we find.
>
> Regarding binding a port to a scheduler, we are unsure about building
> more functionality on top of the internal `scheduler` option as it
> effects the soft-realtime properties of the system. Generally we do not
> want to add functionality which exposes the internal scheduler semantics
> to the programmer, but rather make the scheduler more intelligent about
> where they schedule tasks. We'll have to discuss internally what we want
> to do with this.
>
> Thanks!
> Lukas
>
> [1]: Windows 7, mingw, vs 10 compiler
>
> cc.sh   -O3 -I/c/jenkins/workspace/compile-master-opu-win32/erts/win32   -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -DERTS_MIXED_MSYS_VC -DSTATIC_ERLANG_DRIVER  -DHAVE_CONFIG_H  -DUSE_THREADS -DWIN32_THREADS  -Iwin32/opt/plain -Ibeam -Isys/win32 -Isys/common -Iwin32 -Izlib  -Ipcre -Ihipe -I../include -I../include/win32 -I../include/internal -I../include/internal/win32 -c sys/common/erl_check_io.c -o obj/win32/opt/plain/erl_check_io.o
> sys/common/erl_check_io.c(99) : error C2016: C requires that a struct or union has at least one member
>
> [2]: http://www.erlang.org/doc/installation_guide/INSTALL-WIN32.html
>
> On 09/07/12 08:47, Wei Cao wrote:
>> In our proxy project which has massive network IO, we found
>> erts_check_io()  become system bottleneck,
>> a call to erts_check_io() would take 500 microseconds to complete, and
>> invoked about 3.5k times per second
>> under heavy preesure, which resulted in the average response time of
>> user request longer than 2 milliseconds,
>> so that limited system throughput down, while cpus were idle(in
>> starvation) at the same time.
>>
>> This patch allocates one pollset for each scheduler, so that each
>> scheduler could invoke erts_check_io()
>> on its own pollset concurrently. To be incorporate well with port
>> migration, all used fd in each port're recorded,
>> removed from old scheduler's pollset and added to new scheduler's when
>> port is migrating between schedulers.
>>
>> After applying this patch, together with binding process&port with
>> schedulers, erts_check_io() is invoked about 230k
>> times per second (dozens of times more than before), and throughtput
>> increases from 45k to 105k, so it works.
>>
>> This patch SHOULD be compiled with ./configure
>> CFLAGS=-DERTS_POLLSET_PER_SCHEDULER
>>
>> Pollset per scheduler:
>>
>> git fetch git://github.com/weicao/otp.git pollset_per_scheduler
>>
>> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler
>> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch
>>
>>
>> Bind port to scheduler:
>>
>> git fetch git://github.com/weicao/otp.git bind_port_to_scheduler
>>
>> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler
>> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch
>>
>>
>>
>> --
>>
>> Best,
>>
>> Wei Cao
>> _______________________________________________
>> erlang-patches mailing list
>> erlang-patches@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-patches
>>


From lukas@REDACTED  Wed Jul 11 16:03:12 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 11 Jul 2012 16:03:12 +0200
Subject: [erlang-patches] Float endianness in 15B
In-Reply-To: <y9lehsjrano.fsf@deinprogramm.de>
References: <y9lehsjrano.fsf@deinprogramm.de>
Message-ID: <4FFD87A0.6060503@erlang.org>

Hello!

Sorry for the long delay, we just now got our ARM machines working. 
Unfortunately it does not seem to use PDP endian floats. So I hope that 
it is ok with you to do some more work on your end.

Does the compiler you use set __BYTE_ORDER to __PDP_ENDIAN in endian.h? 
If so could you update the ac function to check that first? If we can 
rely on the define it will make the check work better when cross compiling.

Also it would be great to have a testcase for this, would you mind a 
small testcase to erts/emulator/test/float_SUITE.erl ?

Thanks!
Lukas

On 23/03/12 09:20, Michael Sperber wrote:
> On certain ARM machines, there's a problem with float endianness, which
> has been reported many times before.  Symptons:
>
> io_lib:write(1.0).
> "0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005299808824"
>
> The problem is *not* that the ARM is not using IEEE 754, as is inferred
> here:
>
> http://erlang.org/pipermail/erlang-questions/2008-October/039148.html
>
> The ARM uses IEEE 754, but the two 32-bit words in the FP
> representation are swapped wrt. Erlang's assumptions.
>
> The fix to the source code is very simple, but the associated autoconf
> hackery is a bit more extensive.
>
> Such a fix is here:
>
> git fetch git://github.com/bjorng/otp.git double_middle_endian
> https://github.com/mikesperber/otp/compare/double_middle_endian
> https://github.com/mikesperber/otp/compare/double_middle_endian.patch
>
> (This is the first time I'm submitting a fix to Erlang, and I needed our
> resident git ninja to help me set up, so I hope you'll forgive me if I
> did something wrong - let me know.)
>


From lukas@REDACTED  Wed Jul 11 16:44:44 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 11 Jul 2012 16:44:44 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
Message-ID: <4FFD915C.2040804@erlang.org>

Hi,

The reason I'm skeptical about anything which binds processes/ports to 
scheduler is that it feels like a temporary solution and would much 
rather do a proper solution where the scheduler takes care of these 
things for you. But as I said, internally we need to talk this over when 
it is not in the middle of summer vacation.

I did some benchmarking using ab and found basically the same figures as 
you. The below is with keep-alive and the values are requests per second:

                    not-bound        bound

R15B01                 44k        37k

master                  44k        35k

master+mp          48k        49k

master+mp+pb    49k        55k

[mp]: multi-poll patch
[pb]: port bind patch
[bound]: Used {scheduler,I} to spread load

Unfortunately I also found that when doing the non-keep alive benchmark 
the performance is seriously degraded.

R15B01 not-bound                  8255
master+mp+pb not-bound    7668
master+mp+pb bound           5765

I did some gprof runs but could not find anything obvious that is going 
wrong.

Lukas

On 11/07/12 04:21, Wei Cao wrote:
> I added a macro to conditional compile the patch because I think it
> can be more selectable, I can remove the macro, fix the compilation
> error and test on mingw platform in later version.
>
> how about provide another BIF named port_flag (like process_flag) to
> let user bind port to a given scheduler?
>
>


From cyg.cao@REDACTED  Wed Jul 11 17:34:08 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 11 Jul 2012 23:34:08 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <4FFD915C.2040804@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
Message-ID: <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>

In non keep-alive cases, all new connections 're accepted by the
Erlang port which listens on the TCP port, and how frequently/fast the
port be scheduled to run limits the QPS. (requests per second), so
this port can be regarded as bottleneck of non keep-alive
applications.

So I guess performance degradation observed is caused by the listener
port not be scheduled frequent or fast enough, I'll look into this
problem tomorrow, now is at night in China, :-)

BTW, I found this patch should be compiled like this today,
./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3 -fomit-frame-pointer"
otherwise compiler optimization is disabled.

Regarding binding processes/ports to scheduler, I admit it's really a
temporary solution to bind port to the same scheduler as its owner
process like the pb patch did, and I suggest it's better to add a
additional BIF like erlang:process_flag, to allow user explicitly bind
port to a given scheduler, if it benefits.


2012/7/11 Lukas Larsson <lukas@REDACTED>:
> Hi,
>
> The reason I'm skeptical about anything which binds processes/ports to
> scheduler is that it feels like a temporary solution and would much rather
> do a proper solution where the scheduler takes care of these things for you.
> But as I said, internally we need to talk this over when it is not in the
> middle of summer vacation.
>
> I did some benchmarking using ab and found basically the same figures as
> you. The below is with keep-alive and the values are requests per second:
>
>                    not-bound        bound
>
> R15B01                 44k        37k
>
> master                  44k        35k
>
> master+mp          48k        49k
>
> master+mp+pb    49k        55k
>
> [mp]: multi-poll patch
> [pb]: port bind patch
> [bound]: Used {scheduler,I} to spread load
>
> Unfortunately I also found that when doing the non-keep alive benchmark the
> performance is seriously degraded.
>
> R15B01 not-bound                  8255
> master+mp+pb not-bound    7668
> master+mp+pb bound           5765
>
> I did some gprof runs but could not find anything obvious that is going
> wrong.
>
> Lukas
>
>
> On 11/07/12 04:21, Wei Cao wrote:
>>
>> I added a macro to conditional compile the patch because I think it
>> can be more selectable, I can remove the macro, fix the compilation
>> error and test on mingw platform in later version.
>>
>> how about provide another BIF named port_flag (like process_flag) to
>> let user bind port to a given scheduler?
>>
>>
>



-- 

Best,

Wei Cao


From max.lapshin@REDACTED  Wed Jul 11 17:44:07 2012
From: max.lapshin@REDACTED (Max Lapshin)
Date: Wed, 11 Jul 2012 19:44:07 +0400
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
Message-ID: <CAMxVRxB1HnwTQ9YHFPv8Hg5v0NAod4L2eus6PCfQ6=ujU9FAUQ@mail.gmail.com>

I've found that it may be very important to launch erlang with +sbt s
because without this strict option, you may experience veeeery slow
distribution among cores.


From vances@REDACTED  Mon Jul 16 09:14:25 2012
From: vances@REDACTED (Vance Shipley)
Date: Mon, 16 Jul 2012 12:44:25 +0530
Subject: [erlang-patches] extended back references in re:replace/3,4
In-Reply-To: <20120404134729.GI14994@aluminum.wavenet.lk>
References: <20120308155816.GI22798@aluminum.wavenet.lk>
 <20120404134729.GI14994@aluminum.wavenet.lk>
Message-ID: <20120716071424.GA494@aluminum.motivity.ca>

Folks,

I never heard anything further after submitting this patch in April.
I have redone it today using the current maint branch and it is available
here:

	https://github.com/vances/otp/commit/2094d07ca0fae8e2d983a4aeaaf2e4db7896edd5

Please let me know what I have to do to see this included in pu.

-- 
	-Vance

On Wed, Apr 04, 2012 at 07:17:29PM +0530, Vance Shipley wrote:
}  I created a patch to resolve the use case probem I have with
}  the current implementation of re:replace/3,4 with regard to
}  using back references on numeric digit strings:
}  
}  	git fetch git://github.com/vances/otp.git re_back_reference
}  
}  	https://github.com/vances/otp/compare/re_back_reference
}  	https://github.com/vances/otp/compare/re_back_reference
}  
}  Test cases and updated the documentation are included.
}  
}  I hope you'll accept it for inclusion in a future release.
}  
}  -- 
}  	-Vance
}  
}  On Thu, Mar 08, 2012 at 09:28:19PM +0530, Vance Shipley wrote:
}  }  Is there a syntax supported which allows us to insert a back
}  }  reference in a replacement followed directly by digits?  I
}  }  haven't found one yet.
}  }  
}  }     Eshell V5.9  (abort with ^G)
}  }     1> re:replace("012345", "0(...)45", "\\1bcde").
}  }     [[<<"123">>,<<"bcde">>]]
}  }     2> re:replace("012345", "0(...)45", "\\16789").
}  }     [[<<>>]]
}  }     3> re:replace("012345", "0(...)45", "\\g{1}6789").
}  }     [<<"g{1}6789">>]
}  }  
}  }  
}  }  -- 
}  }  	-Vance
}  }  _______________________________________________
}  }  erlang-questions mailing list
}  }  erlang-questions@REDACTED
}  }  http://erlang.org/mailman/listinfo/erlang-questions


From vances@REDACTED  Mon Jul 16 09:21:15 2012
From: vances@REDACTED (Vance Shipley)
Date: Mon, 16 Jul 2012 12:51:15 +0530
Subject: [erlang-patches] extended back references in re:replace/3,4
In-Reply-To: <20120716071424.GA494@aluminum.motivity.ca>
References: <20120308155816.GI22798@aluminum.wavenet.lk>
 <20120404134729.GI14994@aluminum.wavenet.lk>
 <20120716071424.GA494@aluminum.motivity.ca>
Message-ID: <20120716072114.GB494@aluminum.motivity.ca>

Oops ... never mind.  I see that it was added just five days ago!

   https://github.com/erlang/otp/commit/9bb6cd9ea8185c2447e6335b2616babaae7e250b

On Mon, Jul 16, 2012 at 12:44:24PM +0530, Vance Shipley wrote:
}  I never heard anything further after submitting this patch in April.

-- 
	-Vance


From piotr.sikora@REDACTED  Mon Jul 16 23:58:42 2012
From: piotr.sikora@REDACTED (Piotr Sikora)
Date: Mon, 16 Jul 2012 21:59:42 +0001
Subject: [erlang-patches] Fix use of "clever" mktime.
Message-ID: <20120716220007.GA28144@puffy52.local>

Fix use of "clever" mktime.

Commit 1eef765 introduced regression (conditional _always_ evaluates
to true) in which erlang:localtime_to_universaltime/2 stopped working
on systems configured with timezone without DST (i.e. UTC) on *BSD
platforms:

1> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}, true).
** exception error: bad argument


git fetch git://github.com/PiotrSikora/otp.git fix-clever-mktime

https://github.com/PiotrSikora/otp/compare/fix-clever-mktime
https://github.com/PiotrSikora/otp/compare/fix-clever-mktime.patch


Best regards,
Piotr Sikora < piotr.sikora@REDACTED >


From piotr.sikora@REDACTED  Mon Jul 16 23:59:42 2012
From: piotr.sikora@REDACTED (Piotr Sikora)
Date: Mon, 16 Jul 2012 22:00:42 +0001
Subject: [erlang-patches] Fix support for leap seconds-aware timezones.
Message-ID: <20120716220106.GB28144@puffy52.local>

Fix support for leap seconds-aware timezones.

erlang:universaltime_to_localtime is leap seconds-aware (since 2008),
however erlang:localtime_to_universaltime is not, which gives
surprising results on systems configured with leap seconds-aware
timezones:

1> erlang:universaltime_to_localtime({{2012,1,1},{0,0,0}}).
{{2012,1,1},{0,0,0}}
2> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}).
{{2012,1,1},{0,0,24}}

and completely breaks calendar:local_time_to_universal_time_dst:

3> calendar:local_time_to_universal_time_dst({{2012,1,1},{0,0,0}}).
[]


git fetch git://github.com/PiotrSikora/otp.git fix-leap-seconds

https://github.com/PiotrSikora/otp/compare/fix-leap-seconds
https://github.com/PiotrSikora/otp/compare/fix-leap-seconds.patch


Note: this patch was tested on OpenBSD and it fixes the issue, but it should 
be tested on other operating systems with leap seconds-aware timezones. 
Information on how to enable them on Linux and FreeBSD can be found here:
http://thedjbway.b0llix.net/clockspeed/leapsecs.html

Best regards,
Piotr Sikora < piotr.sikora@REDACTED >


From sperber@REDACTED  Tue Jul 17 10:14:38 2012
From: sperber@REDACTED (Michael Sperber)
Date: Tue, 17 Jul 2012 10:14:38 +0200
Subject: [erlang-patches] Float endianness in 15B
In-Reply-To: <4FFD87A0.6060503@erlang.org> (Lukas Larsson's message of "Wed,
 11 Jul 2012 16:03:12 +0200")
References: <y9lehsjrano.fsf@deinprogramm.de> <4FFD87A0.6060503@erlang.org>
Message-ID: <y9l8veikdn5.fsf@deinprogramm.de>


Lukas Larsson <lukas@REDACTED> writes:

> Sorry for the long delay, we just now got our ARM machines
> working. Unfortunately it does not seem to use PDP endian floats. So I
> hope that it is ok with you to do some more work on your end.

Of course.  Thanks for looking into this!

> Does the compiler you use set __BYTE_ORDER to __PDP_ENDIAN in
> endian.h?

No, it's __LITTLE_ENDIAN.  And that's the correct setting for integers.
The problem is that the information from endian.h is not enough to
determine float endianness.

> If so could you update the ac function to check that first? 
> If we can rely on the define it will make the check work better when
> cross compiling.

Good point.  (Especially as we used to cross-compile before switching to
building under qemu.)  I'm afraid I don't know a good solution.  Maybe
we do need to look at the architecture name after all.  I'd be happy to
code up an alternative if we can agree on the direction.

> Also it would be great to have a testcase for this, would you mind a
> small testcase to erts/emulator/test/float_SUITE.erl ?

Good point.  I believe the floating-point test suite didn't pass the
existing tests, but of course that was coincidental.  Here's the test
suite:

https://github.com/mikesperber/otp/commit/cab9edd0a6f1fad2c3ec9661b39627756ad9d2ba

Please let me if there's anything else I can do to move this along.

-- 
Regards,
Mike


From lukas@REDACTED  Tue Jul 17 12:08:08 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Tue, 17 Jul 2012 12:08:08 +0200
Subject: [erlang-patches] Float endianness in 15B
In-Reply-To: <y9l8veikdn5.fsf@deinprogramm.de>
References: <y9lehsjrano.fsf@deinprogramm.de> <4FFD87A0.6060503@erlang.org>
 <y9l8veikdn5.fsf@deinprogramm.de>
Message-ID: <50053988.9030005@erlang.org>

Aha, so that is the way it works. Then I think we'll have to accept the 
current solution until someones google skills are better than mine and 
find a compile time way to do this.

Thanks for the testcase! I'll let it run a night in our testbench and 
then I'll merge it to be released with R15B02!

Lukas


On 17/07/12 10:14, Michael Sperber wrote:
> Lukas Larsson <lukas@REDACTED> writes:
>
>> Sorry for the long delay, we just now got our ARM machines
>> working. Unfortunately it does not seem to use PDP endian floats. So I
>> hope that it is ok with you to do some more work on your end.
> Of course.  Thanks for looking into this!
>
>> Does the compiler you use set __BYTE_ORDER to __PDP_ENDIAN in
>> endian.h?
> No, it's __LITTLE_ENDIAN.  And that's the correct setting for integers.
> The problem is that the information from endian.h is not enough to
> determine float endianness.
>
>> If so could you update the ac function to check that first?
>> If we can rely on the define it will make the check work better when
>> cross compiling.
> Good point.  (Especially as we used to cross-compile before switching to
> building under qemu.)  I'm afraid I don't know a good solution.  Maybe
> we do need to look at the architecture name after all.  I'd be happy to
> code up an alternative if we can agree on the direction.
>
>> Also it would be great to have a testcase for this, would you mind a
>> small testcase to erts/emulator/test/float_SUITE.erl ?
> Good point.  I believe the floating-point test suite didn't pass the
> existing tests, but of course that was coincidental.  Here's the test
> suite:
>
> https://github.com/mikesperber/otp/commit/cab9edd0a6f1fad2c3ec9661b39627756ad9d2ba
>
> Please let me if there's anything else I can do to move this along.
>




From lukas@REDACTED  Tue Jul 17 12:29:45 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Tue, 17 Jul 2012 12:29:45 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
Message-ID: <50053E99.7090909@erlang.org>

Hi!

After fixing the patch to compile on windows[1] I put it in our daily 
builds and a couple of issues came up. I did not include 'move 
erts_deliver_time out of erl_check_io 
<https://github.com/weicao/otp/commit/7577507ae849aa94a5cff147d8d13e7c1d327a2f>' 
commit as it deadlocks the non-smp emulator.

When running a debug test build on Linux we got the following assertion:
Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in 
sys/common/erl_check_io.c, line 1909

On OS X Lion we got the following when compiling gs:
erl -pa ../ebin -s gs_make -s erlang halt -noshell
../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock(): 
Invalid argument (22)
make[3]: *** [gstk_generic.hrl] Abort trap: 6
make[2]: *** [opt] Error 2
make[1]: *** [opt] Error 2
make: *** [libs] Error 2

I'll remove the branch and see if the same problems appear again 
tomorrow (unfortunately we do not have enough machines to let your 
branch run alone, so it might be some other branch causing this). Let me 
know if you need any help tracking down these issues.

Lukas

[1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler

On 11/07/12 17:34, Wei Cao wrote:
> In non keep-alive cases, all new connections 're accepted by the
> Erlang port which listens on the TCP port, and how frequently/fast the
> port be scheduled to run limits the QPS. (requests per second), so
> this port can be regarded as bottleneck of non keep-alive
> applications.
>
> So I guess performance degradation observed is caused by the listener
> port not be scheduled frequent or fast enough, I'll look into this
> problem tomorrow, now is at night in China, :-)
>
> BTW, I found this patch should be compiled like this today,
> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3 -fomit-frame-pointer"
> otherwise compiler optimization is disabled.
>
> Regarding binding processes/ports to scheduler, I admit it's really a
> temporary solution to bind port to the same scheduler as its owner
> process like the pb patch did, and I suggest it's better to add a
> additional BIF like erlang:process_flag, to allow user explicitly bind
> port to a given scheduler, if it benefits.
>
>
> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>> Hi,
>>
>> The reason I'm skeptical about anything which binds processes/ports to
>> scheduler is that it feels like a temporary solution and would much rather
>> do a proper solution where the scheduler takes care of these things for you.
>> But as I said, internally we need to talk this over when it is not in the
>> middle of summer vacation.
>>
>> I did some benchmarking using ab and found basically the same figures as
>> you. The below is with keep-alive and the values are requests per second:
>>
>>                     not-bound        bound
>>
>> R15B01                 44k        37k
>>
>> master                  44k        35k
>>
>> master+mp          48k        49k
>>
>> master+mp+pb    49k        55k
>>
>> [mp]: multi-poll patch
>> [pb]: port bind patch
>> [bound]: Used {scheduler,I} to spread load
>>
>> Unfortunately I also found that when doing the non-keep alive benchmark the
>> performance is seriously degraded.
>>
>> R15B01 not-bound                  8255
>> master+mp+pb not-bound    7668
>> master+mp+pb bound           5765
>>
>> I did some gprof runs but could not find anything obvious that is going
>> wrong.
>>
>> Lukas
>>
>>
>> On 11/07/12 04:21, Wei Cao wrote:
>>> I added a macro to conditional compile the patch because I think it
>>> can be more selectable, I can remove the macro, fix the compilation
>>> error and test on mingw platform in later version.
>>>
>>> how about provide another BIF named port_flag (like process_flag) to
>>> let user bind port to a given scheduler?
>>>
>>>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120717/366446dd/attachment.htm>

From cyg.cao@REDACTED  Wed Jul 18 06:15:46 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 18 Jul 2012 12:15:46 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <50053E99.7090909@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
Message-ID: <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>

2012/7/17 Lukas Larsson <lukas@REDACTED>:
> Hi!
>
> After fixing the patch to compile on windows[1] I put it in our daily builds

Glad to hear it can compile on windows now, thanks a lot~ I tried to set up
mingw etc on a windows virtual machine yesterday, but it runned so slow, and
met a lot of mistakes during configuration.

> and a couple of issues came up. I did not include 'move erts_deliver_time
> out of erl_check_io' commit as it deadlocks the non-smp emulator.

Yeah, I can confirm this problem, there is only one OS thread
innon-smp emulator,
if check_io starts waiting, other threads will have no opportunity to
run, including the
thread I added to call erts_deliver_time() periodically, so check_io
will have no timer
to wake it up, and would wait endlessly.

I'll consider to replace a better solution.

>
> When running a debug test build on Linux we got the following assertion:
> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
> sys/common/erl_check_io.c, line 1909

hm... this is a very silly mistake, the variable erts_no_pollsets is
set after the assertion,
after fixing it(post to pollset_per_scheduler branch), I met another
assertion error,

erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241

>
> On OS X Lion we got the following when compiling gs:
> erl -pa ../ebin -s gs_make -s erlang halt -noshell
> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
> Invalid argument (22)
> make[3]: *** [gstk_generic.hrl] Abort trap: 6
> make[2]: *** [opt] Error 2
> make[1]: *** [opt] Error 2
> make: *** [libs] Error 2

I didn't have a mac machine to test it.

>
> I'll remove the branch and see if the same problems appear again tomorrow
> (unfortunately we do not have enough machines to let your branch run alone,
> so it might be some other branch causing this). Let me know if you need any
> help tracking down these issues.
>
> Lukas
>
> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>
>
> On 11/07/12 17:34, Wei Cao wrote:
>
> In non keep-alive cases, all new connections 're accepted by the
> Erlang port which listens on the TCP port, and how frequently/fast the
> port be scheduled to run limits the QPS. (requests per second), so
> this port can be regarded as bottleneck of non keep-alive
> applications.
>
> So I guess performance degradation observed is caused by the listener
> port not be scheduled frequent or fast enough, I'll look into this
> problem tomorrow, now is at night in China, :-)
>
> BTW, I found this patch should be compiled like this today,
> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
> -fomit-frame-pointer"
> otherwise compiler optimization is disabled.
>
> Regarding binding processes/ports to scheduler, I admit it's really a
> temporary solution to bind port to the same scheduler as its owner
> process like the pb patch did, and I suggest it's better to add a
> additional BIF like erlang:process_flag, to allow user explicitly bind
> port to a given scheduler, if it benefits.
>
>
> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>
> Hi,
>
> The reason I'm skeptical about anything which binds processes/ports to
> scheduler is that it feels like a temporary solution and would much rather
> do a proper solution where the scheduler takes care of these things for you.
> But as I said, internally we need to talk this over when it is not in the
> middle of summer vacation.
>
> I did some benchmarking using ab and found basically the same figures as
> you. The below is with keep-alive and the values are requests per second:
>
>                    not-bound        bound
>
> R15B01                 44k        37k
>
> master                  44k        35k
>
> master+mp          48k        49k
>
> master+mp+pb    49k        55k
>
> [mp]: multi-poll patch
> [pb]: port bind patch
> [bound]: Used {scheduler,I} to spread load
>
> Unfortunately I also found that when doing the non-keep alive benchmark the
> performance is seriously degraded.
>
> R15B01 not-bound                  8255
> master+mp+pb not-bound    7668
> master+mp+pb bound           5765
>
> I did some gprof runs but could not find anything obvious that is going
> wrong.
>
> Lukas
>
>
> On 11/07/12 04:21, Wei Cao wrote:
>
> I added a macro to conditional compile the patch because I think it
> can be more selectable, I can remove the macro, fix the compilation
> error and test on mingw platform in later version.
>
> how about provide another BIF named port_flag (like process_flag) to
> let user bind port to a given scheduler?
>
>
>
>
>
>



-- 

Best,

Wei Cao


From cyg.cao@REDACTED  Wed Jul 18 08:23:44 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 18 Jul 2012 14:23:44 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>
Message-ID: <CADS4RawtJ_xW_BaD2vM3KcrEvFzybvhZ8HVpY2DLTRu04R36kg@mail.gmail.com>

2012/7/18 Wei Cao <cyg.cao@REDACTED>:
> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>> Hi!
>>
>> After fixing the patch to compile on windows[1] I put it in our daily builds
>
> Glad to hear it can compile on windows now, thanks a lot~ I tried to set up
> mingw etc on a windows virtual machine yesterday, but it runned so slow, and
> met a lot of mistakes during configuration.
>
>> and a couple of issues came up. I did not include 'move erts_deliver_time
>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>
> Yeah, I can confirm this problem, there is only one OS thread
> innon-smp emulator,
> if check_io starts waiting, other threads will have no opportunity to
> run, including the
> thread I added to call erts_deliver_time() periodically, so check_io
> will have no timer
> to wake it up, and would wait endlessly.
>
> I'll consider to replace a better solution.
>
>>
>> When running a debug test build on Linux we got the following assertion:
>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>> sys/common/erl_check_io.c, line 1909
>
> hm... this is a very silly mistake, the variable erts_no_pollsets is
> set after the assertion,
> after fixing it(post to pollset_per_scheduler branch), I met another
> assertion error,
>
> erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241
>

I tried the latest master branch on erlang/otp(without my patch), configure with
./configure CFLAGS="-DDEBUG -g -O3 -fomit-frame-pointer"
there is the same assertion failure:
Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241

However, if instead I configure with
./configure CFLAGS="-DDEBUG", above assertion failures disappear,
instead reporting warnings like this:

bit length overflow
code 12 bits 7->6

It seems CFLAGS influence these assertions.

>>
>> On OS X Lion we got the following when compiling gs:
>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>> Invalid argument (22)
>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>> make[2]: *** [opt] Error 2
>> make[1]: *** [opt] Error 2
>> make: *** [libs] Error 2
>
> I didn't have a mac machine to test it.
>
>>
>> I'll remove the branch and see if the same problems appear again tomorrow
>> (unfortunately we do not have enough machines to let your branch run alone,
>> so it might be some other branch causing this). Let me know if you need any
>> help tracking down these issues.
>>
>> Lukas
>>
>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>
>>
>> On 11/07/12 17:34, Wei Cao wrote:
>>
>> In non keep-alive cases, all new connections 're accepted by the
>> Erlang port which listens on the TCP port, and how frequently/fast the
>> port be scheduled to run limits the QPS. (requests per second), so
>> this port can be regarded as bottleneck of non keep-alive
>> applications.
>>
>> So I guess performance degradation observed is caused by the listener
>> port not be scheduled frequent or fast enough, I'll look into this
>> problem tomorrow, now is at night in China, :-)
>>
>> BTW, I found this patch should be compiled like this today,
>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>> -fomit-frame-pointer"
>> otherwise compiler optimization is disabled.
>>
>> Regarding binding processes/ports to scheduler, I admit it's really a
>> temporary solution to bind port to the same scheduler as its owner
>> process like the pb patch did, and I suggest it's better to add a
>> additional BIF like erlang:process_flag, to allow user explicitly bind
>> port to a given scheduler, if it benefits.
>>
>>
>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>
>> Hi,
>>
>> The reason I'm skeptical about anything which binds processes/ports to
>> scheduler is that it feels like a temporary solution and would much rather
>> do a proper solution where the scheduler takes care of these things for you.
>> But as I said, internally we need to talk this over when it is not in the
>> middle of summer vacation.
>>
>> I did some benchmarking using ab and found basically the same figures as
>> you. The below is with keep-alive and the values are requests per second:
>>
>>                    not-bound        bound
>>
>> R15B01                 44k        37k
>>
>> master                  44k        35k
>>
>> master+mp          48k        49k
>>
>> master+mp+pb    49k        55k
>>
>> [mp]: multi-poll patch
>> [pb]: port bind patch
>> [bound]: Used {scheduler,I} to spread load
>>
>> Unfortunately I also found that when doing the non-keep alive benchmark the
>> performance is seriously degraded.
>>
>> R15B01 not-bound                  8255
>> master+mp+pb not-bound    7668
>> master+mp+pb bound           5765
>>
>> I did some gprof runs but could not find anything obvious that is going
>> wrong.
>>
>> Lukas
>>
>>
>> On 11/07/12 04:21, Wei Cao wrote:
>>
>> I added a macro to conditional compile the patch because I think it
>> can be more selectable, I can remove the macro, fix the compilation
>> error and test on mingw platform in later version.
>>
>> how about provide another BIF named port_flag (like process_flag) to
>> let user bind port to a given scheduler?
>>
>>
>>
>>
>>
>>
>
>
>
> --
>
> Best,
>
> Wei Cao



-- 

Best,

Wei Cao


From lukas@REDACTED  Wed Jul 18 09:39:53 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 18 Jul 2012 09:39:53 +0200
Subject: [erlang-patches] Float endianness in 15B
In-Reply-To: <y9l8veikdn5.fsf@deinprogramm.de>
References: <y9lehsjrano.fsf@deinprogramm.de> <4FFD87A0.6060503@erlang.org>
 <y9l8veikdn5.fsf@deinprogramm.de>
Message-ID: <50066849.5060303@erlang.org>

Hello,

The patch just ran through our nightly build and a couple of issues 
popped up.

It seems that the ac script does not correct detect double endianess on 
big endian platforms. The configure on all of our sparc and ppc machines 
detected double word ordering as middle-endian.

Also don't think an error should be thrown in configure if we cannot 
detect the double endianess. Instead a warning should be issued and 
configure should continue with the assumption that the doubles are not 
middle-endian. There are a number of options that already do this and I 
think it makes sense here as we can never detect it in a cross environment.

Lukas

On 17/07/12 10:14, Michael Sperber wrote:
> Lukas Larsson <lukas@REDACTED> writes:
>
>> Sorry for the long delay, we just now got our ARM machines
>> working. Unfortunately it does not seem to use PDP endian floats. So I
>> hope that it is ok with you to do some more work on your end.
> Of course.  Thanks for looking into this!
>
>> Does the compiler you use set __BYTE_ORDER to __PDP_ENDIAN in
>> endian.h?
> No, it's __LITTLE_ENDIAN.  And that's the correct setting for integers.
> The problem is that the information from endian.h is not enough to
> determine float endianness.
>
>> If so could you update the ac function to check that first?
>> If we can rely on the define it will make the check work better when
>> cross compiling.
> Good point.  (Especially as we used to cross-compile before switching to
> building under qemu.)  I'm afraid I don't know a good solution.  Maybe
> we do need to look at the architecture name after all.  I'd be happy to
> code up an alternative if we can agree on the direction.
>
>> Also it would be great to have a testcase for this, would you mind a
>> small testcase to erts/emulator/test/float_SUITE.erl ?
> Good point.  I believe the floating-point test suite didn't pass the
> existing tests, but of course that was coincidental.  Here's the test
> suite:
>
> https://github.com/mikesperber/otp/commit/cab9edd0a6f1fad2c3ec9661b39627756ad9d2ba
>
> Please let me if there's anything else I can do to move this along.
>




From lukas@REDACTED  Wed Jul 18 09:50:18 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 18 Jul 2012 09:50:18 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4RawtJ_xW_BaD2vM3KcrEvFzybvhZ8HVpY2DLTRu04R36kg@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>
 <CADS4RawtJ_xW_BaD2vM3KcrEvFzybvhZ8HVpY2DLTRu04R36kg@mail.gmail.com>
Message-ID: <50066ABA.1080806@erlang.org>

Hi!

You should disable hipe before trying to compile the debug build. Also 
I'm unsure if just adding -DDEBUG to CFLAGS is enough. To build a debug 
emulator the way we do it is:

(cd $ERL_TOP && ./otp_build autoconf && ./otp_build configure 
--disable-hipe && ./otp_build boot)
(cd $ERL_TOP/erts/emulator && make FLAVOR=smp debug)

and then start it using

$ERL_TOP/bin/cerl -debug

Lukas

On 18/07/12 08:23, Wei Cao wrote:
> 2012/7/18 Wei Cao <cyg.cao@REDACTED>:
>> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>>> Hi!
>>>
>>> After fixing the patch to compile on windows[1] I put it in our daily builds
>> Glad to hear it can compile on windows now, thanks a lot~ I tried to set up
>> mingw etc on a windows virtual machine yesterday, but it runned so slow, and
>> met a lot of mistakes during configuration.
>>
>>> and a couple of issues came up. I did not include 'move erts_deliver_time
>>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>> Yeah, I can confirm this problem, there is only one OS thread
>> innon-smp emulator,
>> if check_io starts waiting, other threads will have no opportunity to
>> run, including the
>> thread I added to call erts_deliver_time() periodically, so check_io
>> will have no timer
>> to wake it up, and would wait endlessly.
>>
>> I'll consider to replace a better solution.
>>
>>> When running a debug test build on Linux we got the following assertion:
>>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>>> sys/common/erl_check_io.c, line 1909
>> hm... this is a very silly mistake, the variable erts_no_pollsets is
>> set after the assertion,
>> after fixing it(post to pollset_per_scheduler branch), I met another
>> assertion error,
>>
>> erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
>> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241
>>
> I tried the latest master branch on erlang/otp(without my patch), configure with
> ./configure CFLAGS="-DDEBUG -g -O3 -fomit-frame-pointer"
> there is the same assertion failure:
> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241
>
> However, if instead I configure with
> ./configure CFLAGS="-DDEBUG", above assertion failures disappear,
> instead reporting warnings like this:
>
> bit length overflow
> code 12 bits 7->6
>
> It seems CFLAGS influence these assertions.
>
>>> On OS X Lion we got the following when compiling gs:
>>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>>> Invalid argument (22)
>>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>>> make[2]: *** [opt] Error 2
>>> make[1]: *** [opt] Error 2
>>> make: *** [libs] Error 2
>> I didn't have a mac machine to test it.
>>
>>> I'll remove the branch and see if the same problems appear again tomorrow
>>> (unfortunately we do not have enough machines to let your branch run alone,
>>> so it might be some other branch causing this). Let me know if you need any
>>> help tracking down these issues.
>>>
>>> Lukas
>>>
>>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>>
>>>
>>> On 11/07/12 17:34, Wei Cao wrote:
>>>
>>> In non keep-alive cases, all new connections 're accepted by the
>>> Erlang port which listens on the TCP port, and how frequently/fast the
>>> port be scheduled to run limits the QPS. (requests per second), so
>>> this port can be regarded as bottleneck of non keep-alive
>>> applications.
>>>
>>> So I guess performance degradation observed is caused by the listener
>>> port not be scheduled frequent or fast enough, I'll look into this
>>> problem tomorrow, now is at night in China, :-)
>>>
>>> BTW, I found this patch should be compiled like this today,
>>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>>> -fomit-frame-pointer"
>>> otherwise compiler optimization is disabled.
>>>
>>> Regarding binding processes/ports to scheduler, I admit it's really a
>>> temporary solution to bind port to the same scheduler as its owner
>>> process like the pb patch did, and I suggest it's better to add a
>>> additional BIF like erlang:process_flag, to allow user explicitly bind
>>> port to a given scheduler, if it benefits.
>>>
>>>
>>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>>
>>> Hi,
>>>
>>> The reason I'm skeptical about anything which binds processes/ports to
>>> scheduler is that it feels like a temporary solution and would much rather
>>> do a proper solution where the scheduler takes care of these things for you.
>>> But as I said, internally we need to talk this over when it is not in the
>>> middle of summer vacation.
>>>
>>> I did some benchmarking using ab and found basically the same figures as
>>> you. The below is with keep-alive and the values are requests per second:
>>>
>>>                     not-bound        bound
>>>
>>> R15B01                 44k        37k
>>>
>>> master                  44k        35k
>>>
>>> master+mp          48k        49k
>>>
>>> master+mp+pb    49k        55k
>>>
>>> [mp]: multi-poll patch
>>> [pb]: port bind patch
>>> [bound]: Used {scheduler,I} to spread load
>>>
>>> Unfortunately I also found that when doing the non-keep alive benchmark the
>>> performance is seriously degraded.
>>>
>>> R15B01 not-bound                  8255
>>> master+mp+pb not-bound    7668
>>> master+mp+pb bound           5765
>>>
>>> I did some gprof runs but could not find anything obvious that is going
>>> wrong.
>>>
>>> Lukas
>>>
>>>
>>> On 11/07/12 04:21, Wei Cao wrote:
>>>
>>> I added a macro to conditional compile the patch because I think it
>>> can be more selectable, I can remove the macro, fix the compilation
>>> error and test on mingw platform in later version.
>>>
>>> how about provide another BIF named port_flag (like process_flag) to
>>> let user bind port to a given scheduler?
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>>
>> Best,
>>
>> Wei Cao
>
>




From cyg.cao@REDACTED  Wed Jul 18 11:09:52 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 18 Jul 2012 17:09:52 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <50066ABA.1080806@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>
 <CADS4RawtJ_xW_BaD2vM3KcrEvFzybvhZ8HVpY2DLTRu04R36kg@mail.gmail.com>
 <50066ABA.1080806@erlang.org>
Message-ID: <CADS4Razjkp3D5_QsdgL1_i6EO_GJh0sXwRh8KKtXgcvDk1wbpg@mail.gmail.com>

Thanks, warnings disappear after disable hipe, I then runned serveral
tests with debug build ,fixed several assertion failures found and it
works all right now.

Fixes're pushed to pollset_per_scheduler branch at
git://github.com/weicao/otp.git


2012/7/18 Lukas Larsson <lukas@REDACTED>:
> Hi!
>
> You should disable hipe before trying to compile the debug build. Also I'm
> unsure if just adding -DDEBUG to CFLAGS is enough. To build a debug emulator
> the way we do it is:
>
> (cd $ERL_TOP && ./otp_build autoconf && ./otp_build configure --disable-hipe
> && ./otp_build boot)
> (cd $ERL_TOP/erts/emulator && make FLAVOR=smp debug)
>
> and then start it using
>
> $ERL_TOP/bin/cerl -debug
>
> Lukas
>
>
> On 18/07/12 08:23, Wei Cao wrote:
>>
>> 2012/7/18 Wei Cao <cyg.cao@REDACTED>:
>>>
>>> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>>>>
>>>> Hi!
>>>>
>>>> After fixing the patch to compile on windows[1] I put it in our daily
>>>> builds
>>>
>>> Glad to hear it can compile on windows now, thanks a lot~ I tried to set
>>> up
>>> mingw etc on a windows virtual machine yesterday, but it runned so slow,
>>> and
>>> met a lot of mistakes during configuration.
>>>
>>>> and a couple of issues came up. I did not include 'move
>>>> erts_deliver_time
>>>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>>>
>>> Yeah, I can confirm this problem, there is only one OS thread
>>> innon-smp emulator,
>>> if check_io starts waiting, other threads will have no opportunity to
>>> run, including the
>>> thread I added to call erts_deliver_time() periodically, so check_io
>>> will have no timer
>>> to wake it up, and would wait endlessly.
>>>
>>> I'll consider to replace a better solution.
>>>
>>>> When running a debug test build on Linux we got the following assertion:
>>>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>>>> sys/common/erl_check_io.c, line 1909
>>>
>>> hm... this is a very silly mistake, the variable erts_no_pollsets is
>>> set after the assertion,
>>> after fixing it(post to pollset_per_scheduler branch), I met another
>>> assertion error,
>>>
>>> erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
>>> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c,
>>> line 1241
>>>
>> I tried the latest master branch on erlang/otp(without my patch),
>> configure with
>> ./configure CFLAGS="-DDEBUG -g -O3 -fomit-frame-pointer"
>> there is the same assertion failure:
>> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c,
>> line 1241
>>
>> However, if instead I configure with
>> ./configure CFLAGS="-DDEBUG", above assertion failures disappear,
>> instead reporting warnings like this:
>>
>> bit length overflow
>> code 12 bits 7->6
>>
>> It seems CFLAGS influence these assertions.
>>
>>>> On OS X Lion we got the following when compiling gs:
>>>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>>>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>>>> Invalid argument (22)
>>>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>>>> make[2]: *** [opt] Error 2
>>>> make[1]: *** [opt] Error 2
>>>> make: *** [libs] Error 2
>>>
>>> I didn't have a mac machine to test it.
>>>
>>>> I'll remove the branch and see if the same problems appear again
>>>> tomorrow
>>>> (unfortunately we do not have enough machines to let your branch run
>>>> alone,
>>>> so it might be some other branch causing this). Let me know if you need
>>>> any
>>>> help tracking down these issues.
>>>>
>>>> Lukas
>>>>
>>>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>>>
>>>>
>>>> On 11/07/12 17:34, Wei Cao wrote:
>>>>
>>>> In non keep-alive cases, all new connections 're accepted by the
>>>> Erlang port which listens on the TCP port, and how frequently/fast the
>>>> port be scheduled to run limits the QPS. (requests per second), so
>>>> this port can be regarded as bottleneck of non keep-alive
>>>> applications.
>>>>
>>>> So I guess performance degradation observed is caused by the listener
>>>> port not be scheduled frequent or fast enough, I'll look into this
>>>> problem tomorrow, now is at night in China, :-)
>>>>
>>>> BTW, I found this patch should be compiled like this today,
>>>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>>>> -fomit-frame-pointer"
>>>> otherwise compiler optimization is disabled.
>>>>
>>>> Regarding binding processes/ports to scheduler, I admit it's really a
>>>> temporary solution to bind port to the same scheduler as its owner
>>>> process like the pb patch did, and I suggest it's better to add a
>>>> additional BIF like erlang:process_flag, to allow user explicitly bind
>>>> port to a given scheduler, if it benefits.
>>>>
>>>>
>>>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>>>
>>>> Hi,
>>>>
>>>> The reason I'm skeptical about anything which binds processes/ports to
>>>> scheduler is that it feels like a temporary solution and would much
>>>> rather
>>>> do a proper solution where the scheduler takes care of these things for
>>>> you.
>>>> But as I said, internally we need to talk this over when it is not in
>>>> the
>>>> middle of summer vacation.
>>>>
>>>> I did some benchmarking using ab and found basically the same figures as
>>>> you. The below is with keep-alive and the values are requests per
>>>> second:
>>>>
>>>>                     not-bound        bound
>>>>
>>>> R15B01                 44k        37k
>>>>
>>>> master                  44k        35k
>>>>
>>>> master+mp          48k        49k
>>>>
>>>> master+mp+pb    49k        55k
>>>>
>>>> [mp]: multi-poll patch
>>>> [pb]: port bind patch
>>>> [bound]: Used {scheduler,I} to spread load
>>>>
>>>> Unfortunately I also found that when doing the non-keep alive benchmark
>>>> the
>>>> performance is seriously degraded.
>>>>
>>>> R15B01 not-bound                  8255
>>>> master+mp+pb not-bound    7668
>>>> master+mp+pb bound           5765
>>>>
>>>> I did some gprof runs but could not find anything obvious that is going
>>>> wrong.
>>>>
>>>> Lukas
>>>>
>>>>
>>>> On 11/07/12 04:21, Wei Cao wrote:
>>>>
>>>> I added a macro to conditional compile the patch because I think it
>>>> can be more selectable, I can remove the macro, fix the compilation
>>>> error and test on mingw platform in later version.
>>>>
>>>> how about provide another BIF named port_flag (like process_flag) to
>>>> let user bind port to a given scheduler?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Best,
>>>
>>> Wei Cao
>>
>>
>>
>
>



-- 

Best,

Wei Cao


From lukas@REDACTED  Wed Jul 18 11:18:01 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 18 Jul 2012 11:18:01 +0200
Subject: [erlang-patches] [patch] new float_to_list/2
In-Reply-To: <4FBE3F78.8050405@aleynikov.org>
References: <4D2D3FA8.1000009@aleynikov.org>
 <AANLkTim8vXNuy1ySbst4Seim7G6m+mGc+1pZ_i=mRc2x@mail.gmail.com>
 <C06991E5-03B1-4802-BAFF-9041B7A6430E@gmail.com>
 <4D2DB65C.1090106@aleynikov.org>
 <AANLkTindOp++U1PeaXVyCRRLgRH9GwC8HZdroES8kLRY@mail.gmail.com>
 <4D2DE13E.7040404@aleynikov.org>
 <AANLkTim5t-MgBFuvbUyoejQN3HawW2SNC4Fa=GMOP+nN@mail.gmail.com>
 <4D33B762.3080202@aleynikov.org> <4D90ABBD.8050100@erix.ericsson.se>
 <4F58F46A.5050701@aleynikov.org> <4F5F52FE.7060003@erlang.org>
 <4F5F8097.5060003@erix.ericsson.se> <4F8EE138.60402@aleynikov.org>
 <4F9129B5.6020304@erlang.org> <4F924348.6070400@aleynikov.org>
 <4FBDFCB2.2050402@erlang.org> <4FBE3F78.8050405@aleynikov.org>
Message-ID: <50067F49.4020206@erlang.org>

Hi Serge!

I think the added functionality is a good idea. I will however have to 
discuss it with the floating point experts here before accepting the 
patch (they are currently enjoying their summer vacations).

However, I would also like the fast functionality to be used by 
float_to_list_1 as well, is possible to do this and stay backwards 
compatible? Hopefully you just have to shift the comma and add e+XX at 
the end of the optimized case and call sys_double_to_chars for the 
unoptimized.

Also float_to_list(1.0) should return the same thing as 
float_to_list(1.0,[]), otherwise the API will be inconsistent with how 
other such APIs work.

Lukas

On 24/05/12 16:02, Serge Aleynikov wrote:
> Henrik,
>
> Fetch:	git fetch https://github.com/saleyn/otp/tree/float_to_list_2
> Diff:	https://github.com/saleyn/otp/compare/float_to_list_2
>
> I added the definition for the new BIF to make the type checker happy:
> https://github.com/saleyn/otp/commit/f9ddbeda5426ca83cda03c06a9860220ea4a22c7
>
> Once you do the "otp_build tests", how do you execute all tests suites
> in $ERL_TOP or if possible only tests in a given SUITE?  I tried the
> following but all tests fail:
>
> [otp/erts/emulator/test]$ ../../../bin/erl -noshell -s test_server_ctrl
> run_test DIR "." -s erlang halt
>
> I did however run individual tests in bif_SUIT:types to make sure my
> patch didn't break anything.
>
>
>
> Serge
>
> On 5/24/2012 5:17 AM, Henrik Nord wrote:
>> Hi again.
>>
>>
>> This test is not passing: emulator/bif_SUIT:types
>>
>> No type information:
>>
>> [{erlang,float_to_list,2}]
>>
>>
>>
>> On 04/21/2012 07:19 AM, Serge Aleynikov wrote:
>>> git fetch https://github.com/saleyn/otp/tree/float_to_list_2
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches
>




From lukas@REDACTED  Wed Jul 18 11:19:24 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 18 Jul 2012 11:19:24 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Razjkp3D5_QsdgL1_i6EO_GJh0sXwRh8KKtXgcvDk1wbpg@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4Raw-H=M1vuR5M1SXXTDahL0HzmkW2=N1tBVy0ouApqtvQg@mail.gmail.com>
 <CADS4RawtJ_xW_BaD2vM3KcrEvFzybvhZ8HVpY2DLTRu04R36kg@mail.gmail.com>
 <50066ABA.1080806@erlang.org>
 <CADS4Razjkp3D5_QsdgL1_i6EO_GJh0sXwRh8KKtXgcvDk1wbpg@mail.gmail.com>
Message-ID: <50067F9C.1010708@erlang.org>

Great, I'll include it in our nightly builds again and let you know of 
the results.

Lukas
On 18/07/12 11:09, Wei Cao wrote:
> Thanks, warnings disappear after disable hipe, I then runned serveral
> tests with debug build ,fixed several assertion failures found and it
> works all right now.
>
> Fixes're pushed to pollset_per_scheduler branch at
> git://github.com/weicao/otp.git
>
>
> 2012/7/18 Lukas Larsson <lukas@REDACTED>:
>> Hi!
>>
>> You should disable hipe before trying to compile the debug build. Also I'm
>> unsure if just adding -DDEBUG to CFLAGS is enough. To build a debug emulator
>> the way we do it is:
>>
>> (cd $ERL_TOP && ./otp_build autoconf && ./otp_build configure --disable-hipe
>> && ./otp_build boot)
>> (cd $ERL_TOP/erts/emulator && make FLAVOR=smp debug)
>>
>> and then start it using
>>
>> $ERL_TOP/bin/cerl -debug
>>
>> Lukas
>>
>>
>> On 18/07/12 08:23, Wei Cao wrote:
>>> 2012/7/18 Wei Cao <cyg.cao@REDACTED>:
>>>> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>>>>> Hi!
>>>>>
>>>>> After fixing the patch to compile on windows[1] I put it in our daily
>>>>> builds
>>>> Glad to hear it can compile on windows now, thanks a lot~ I tried to set
>>>> up
>>>> mingw etc on a windows virtual machine yesterday, but it runned so slow,
>>>> and
>>>> met a lot of mistakes during configuration.
>>>>
>>>>> and a couple of issues came up. I did not include 'move
>>>>> erts_deliver_time
>>>>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>>>> Yeah, I can confirm this problem, there is only one OS thread
>>>> innon-smp emulator,
>>>> if check_io starts waiting, other threads will have no opportunity to
>>>> run, including the
>>>> thread I added to call erts_deliver_time() periodically, so check_io
>>>> will have no timer
>>>> to wake it up, and would wait endlessly.
>>>>
>>>> I'll consider to replace a better solution.
>>>>
>>>>> When running a debug test build on Linux we got the following assertion:
>>>>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>>>>> sys/common/erl_check_io.c, line 1909
>>>> hm... this is a very silly mistake, the variable erts_no_pollsets is
>>>> set after the assertion,
>>>> after fixing it(post to pollset_per_scheduler branch), I met another
>>>> assertion error,
>>>>
>>>> erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
>>>> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c,
>>>> line 1241
>>>>
>>> I tried the latest master branch on erlang/otp(without my patch),
>>> configure with
>>> ./configure CFLAGS="-DDEBUG -g -O3 -fomit-frame-pointer"
>>> there is the same assertion failure:
>>> Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c,
>>> line 1241
>>>
>>> However, if instead I configure with
>>> ./configure CFLAGS="-DDEBUG", above assertion failures disappear,
>>> instead reporting warnings like this:
>>>
>>> bit length overflow
>>> code 12 bits 7->6
>>>
>>> It seems CFLAGS influence these assertions.
>>>
>>>>> On OS X Lion we got the following when compiling gs:
>>>>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>>>>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>>>>> Invalid argument (22)
>>>>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>>>>> make[2]: *** [opt] Error 2
>>>>> make[1]: *** [opt] Error 2
>>>>> make: *** [libs] Error 2
>>>> I didn't have a mac machine to test it.
>>>>
>>>>> I'll remove the branch and see if the same problems appear again
>>>>> tomorrow
>>>>> (unfortunately we do not have enough machines to let your branch run
>>>>> alone,
>>>>> so it might be some other branch causing this). Let me know if you need
>>>>> any
>>>>> help tracking down these issues.
>>>>>
>>>>> Lukas
>>>>>
>>>>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>>>>
>>>>>
>>>>> On 11/07/12 17:34, Wei Cao wrote:
>>>>>
>>>>> In non keep-alive cases, all new connections 're accepted by the
>>>>> Erlang port which listens on the TCP port, and how frequently/fast the
>>>>> port be scheduled to run limits the QPS. (requests per second), so
>>>>> this port can be regarded as bottleneck of non keep-alive
>>>>> applications.
>>>>>
>>>>> So I guess performance degradation observed is caused by the listener
>>>>> port not be scheduled frequent or fast enough, I'll look into this
>>>>> problem tomorrow, now is at night in China, :-)
>>>>>
>>>>> BTW, I found this patch should be compiled like this today,
>>>>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>>>>> -fomit-frame-pointer"
>>>>> otherwise compiler optimization is disabled.
>>>>>
>>>>> Regarding binding processes/ports to scheduler, I admit it's really a
>>>>> temporary solution to bind port to the same scheduler as its owner
>>>>> process like the pb patch did, and I suggest it's better to add a
>>>>> additional BIF like erlang:process_flag, to allow user explicitly bind
>>>>> port to a given scheduler, if it benefits.
>>>>>
>>>>>
>>>>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> The reason I'm skeptical about anything which binds processes/ports to
>>>>> scheduler is that it feels like a temporary solution and would much
>>>>> rather
>>>>> do a proper solution where the scheduler takes care of these things for
>>>>> you.
>>>>> But as I said, internally we need to talk this over when it is not in
>>>>> the
>>>>> middle of summer vacation.
>>>>>
>>>>> I did some benchmarking using ab and found basically the same figures as
>>>>> you. The below is with keep-alive and the values are requests per
>>>>> second:
>>>>>
>>>>>                      not-bound        bound
>>>>>
>>>>> R15B01                 44k        37k
>>>>>
>>>>> master                  44k        35k
>>>>>
>>>>> master+mp          48k        49k
>>>>>
>>>>> master+mp+pb    49k        55k
>>>>>
>>>>> [mp]: multi-poll patch
>>>>> [pb]: port bind patch
>>>>> [bound]: Used {scheduler,I} to spread load
>>>>>
>>>>> Unfortunately I also found that when doing the non-keep alive benchmark
>>>>> the
>>>>> performance is seriously degraded.
>>>>>
>>>>> R15B01 not-bound                  8255
>>>>> master+mp+pb not-bound    7668
>>>>> master+mp+pb bound           5765
>>>>>
>>>>> I did some gprof runs but could not find anything obvious that is going
>>>>> wrong.
>>>>>
>>>>> Lukas
>>>>>
>>>>>
>>>>> On 11/07/12 04:21, Wei Cao wrote:
>>>>>
>>>>> I added a macro to conditional compile the patch because I think it
>>>>> can be more selectable, I can remove the macro, fix the compilation
>>>>> error and test on mingw platform in later version.
>>>>>
>>>>> how about provide another BIF named port_flag (like process_flag) to
>>>>> let user bind port to a given scheduler?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> Best,
>>>>
>>>> Wei Cao
>>>
>>>
>>
>
>




From cyg.cao@REDACTED  Wed Jul 18 11:28:50 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Wed, 18 Jul 2012 17:28:50 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <50053E99.7090909@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
Message-ID: <CADS4RazCb-N71h79Uk70vgfhCJtwwCitDsgR4dwWYikA4j3L3g@mail.gmail.com>

2012/7/17 Lukas Larsson <lukas@REDACTED>:
> Hi!
>
> After fixing the patch to compile on windows[1] I put it in our daily builds
> and a couple of issues came up. I did not include 'move erts_deliver_time
> out of erl_check_io' commit as it deadlocks the non-smp emulator.

This problem is also fixed by testing macro ERTS_SMP to tell whether
it's a non-smp emulator, now both smp and non-smp emulators work.

>
> When running a debug test build on Linux we got the following assertion:
> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
> sys/common/erl_check_io.c, line 1909
>
> On OS X Lion we got the following when compiling gs:
> erl -pa ../ebin -s gs_make -s erlang halt -noshell
> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
> Invalid argument (22)
> make[3]: *** [gstk_generic.hrl] Abort trap: 6
> make[2]: *** [opt] Error 2
> make[1]: *** [opt] Error 2
> make: *** [libs] Error 2
>
> I'll remove the branch and see if the same problems appear again tomorrow
> (unfortunately we do not have enough machines to let your branch run alone,
> so it might be some other branch causing this). Let me know if you need any
> help tracking down these issues.
>
> Lukas
>
> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>
>
> On 11/07/12 17:34, Wei Cao wrote:
>
> In non keep-alive cases, all new connections 're accepted by the
> Erlang port which listens on the TCP port, and how frequently/fast the
> port be scheduled to run limits the QPS. (requests per second), so
> this port can be regarded as bottleneck of non keep-alive
> applications.
>
> So I guess performance degradation observed is caused by the listener
> port not be scheduled frequent or fast enough, I'll look into this
> problem tomorrow, now is at night in China, :-)
>
> BTW, I found this patch should be compiled like this today,
> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
> -fomit-frame-pointer"
> otherwise compiler optimization is disabled.
>
> Regarding binding processes/ports to scheduler, I admit it's really a
> temporary solution to bind port to the same scheduler as its owner
> process like the pb patch did, and I suggest it's better to add a
> additional BIF like erlang:process_flag, to allow user explicitly bind
> port to a given scheduler, if it benefits.
>
>
> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>
> Hi,
>
> The reason I'm skeptical about anything which binds processes/ports to
> scheduler is that it feels like a temporary solution and would much rather
> do a proper solution where the scheduler takes care of these things for you.
> But as I said, internally we need to talk this over when it is not in the
> middle of summer vacation.
>
> I did some benchmarking using ab and found basically the same figures as
> you. The below is with keep-alive and the values are requests per second:
>
>                    not-bound        bound
>
> R15B01                 44k        37k
>
> master                  44k        35k
>
> master+mp          48k        49k
>
> master+mp+pb    49k        55k
>
> [mp]: multi-poll patch
> [pb]: port bind patch
> [bound]: Used {scheduler,I} to spread load
>
> Unfortunately I also found that when doing the non-keep alive benchmark the
> performance is seriously degraded.
>
> R15B01 not-bound                  8255
> master+mp+pb not-bound    7668
> master+mp+pb bound           5765
>
> I did some gprof runs but could not find anything obvious that is going
> wrong.
>
> Lukas
>
>
> On 11/07/12 04:21, Wei Cao wrote:
>
> I added a macro to conditional compile the patch because I think it
> can be more selectable, I can remove the macro, fix the compilation
> error and test on mingw platform in later version.
>
> how about provide another BIF named port_flag (like process_flag) to
> let user bind port to a given scheduler?
>
>
>
>
>
>



-- 

Best,

Wei Cao


From lukas@REDACTED  Wed Jul 18 17:36:37 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Wed, 18 Jul 2012 17:36:37 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4RazCb-N71h79Uk70vgfhCJtwwCitDsgR4dwWYikA4j3L3g@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4RazCb-N71h79Uk70vgfhCJtwwCitDsgR4dwWYikA4j3L3g@mail.gmail.com>
Message-ID: <5006D805.9070607@erlang.org>

fyi the patch has introduced three new gcc warnings:

erl_port_task.c:230, gcc4,

control reaches end of non-void function [-Wreturn-type]

erl_check_io.c:992, gcc4,

?inps? may be used uninitialized in this function [-Wuninitialized]

erl_check_io.c:998, gcc4,

?outps? may be used uninitialized in this function [-Wuninitialized]

Lukas

On 18/07/12 11:28, Wei Cao wrote:
> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>> Hi!
>>
>> After fixing the patch to compile on windows[1] I put it in our daily builds
>> and a couple of issues came up. I did not include 'move erts_deliver_time
>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
> This problem is also fixed by testing macro ERTS_SMP to tell whether
> it's a non-smp emulator, now both smp and non-smp emulators work.
>
>> When running a debug test build on Linux we got the following assertion:
>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>> sys/common/erl_check_io.c, line 1909
>>
>> On OS X Lion we got the following when compiling gs:
>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>> Invalid argument (22)
>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>> make[2]: *** [opt] Error 2
>> make[1]: *** [opt] Error 2
>> make: *** [libs] Error 2
>>
>> I'll remove the branch and see if the same problems appear again tomorrow
>> (unfortunately we do not have enough machines to let your branch run alone,
>> so it might be some other branch causing this). Let me know if you need any
>> help tracking down these issues.
>>
>> Lukas
>>
>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>
>>
>> On 11/07/12 17:34, Wei Cao wrote:
>>
>> In non keep-alive cases, all new connections 're accepted by the
>> Erlang port which listens on the TCP port, and how frequently/fast the
>> port be scheduled to run limits the QPS. (requests per second), so
>> this port can be regarded as bottleneck of non keep-alive
>> applications.
>>
>> So I guess performance degradation observed is caused by the listener
>> port not be scheduled frequent or fast enough, I'll look into this
>> problem tomorrow, now is at night in China, :-)
>>
>> BTW, I found this patch should be compiled like this today,
>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>> -fomit-frame-pointer"
>> otherwise compiler optimization is disabled.
>>
>> Regarding binding processes/ports to scheduler, I admit it's really a
>> temporary solution to bind port to the same scheduler as its owner
>> process like the pb patch did, and I suggest it's better to add a
>> additional BIF like erlang:process_flag, to allow user explicitly bind
>> port to a given scheduler, if it benefits.
>>
>>
>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>
>> Hi,
>>
>> The reason I'm skeptical about anything which binds processes/ports to
>> scheduler is that it feels like a temporary solution and would much rather
>> do a proper solution where the scheduler takes care of these things for you.
>> But as I said, internally we need to talk this over when it is not in the
>> middle of summer vacation.
>>
>> I did some benchmarking using ab and found basically the same figures as
>> you. The below is with keep-alive and the values are requests per second:
>>
>>                     not-bound        bound
>>
>> R15B01                 44k        37k
>>
>> master                  44k        35k
>>
>> master+mp          48k        49k
>>
>> master+mp+pb    49k        55k
>>
>> [mp]: multi-poll patch
>> [pb]: port bind patch
>> [bound]: Used {scheduler,I} to spread load
>>
>> Unfortunately I also found that when doing the non-keep alive benchmark the
>> performance is seriously degraded.
>>
>> R15B01 not-bound                  8255
>> master+mp+pb not-bound    7668
>> master+mp+pb bound           5765
>>
>> I did some gprof runs but could not find anything obvious that is going
>> wrong.
>>
>> Lukas
>>
>>
>> On 11/07/12 04:21, Wei Cao wrote:
>>
>> I added a macro to conditional compile the patch because I think it
>> can be more selectable, I can remove the macro, fix the compilation
>> error and test on mingw platform in later version.
>>
>> how about provide another BIF named port_flag (like process_flag) to
>> let user bind port to a given scheduler?
>>
>>
>>
>>
>>
>>
>
>




From serge@REDACTED  Wed Jul 18 20:47:26 2012
From: serge@REDACTED (Serge Aleynikov)
Date: Wed, 18 Jul 2012 14:47:26 -0400
Subject: [erlang-patches] [patch] new float_to_list/2
In-Reply-To: <50067F49.4020206@erlang.org>
References: <4D2D3FA8.1000009@aleynikov.org>
 <AANLkTim8vXNuy1ySbst4Seim7G6m+mGc+1pZ_i=mRc2x@mail.gmail.com>
 <C06991E5-03B1-4802-BAFF-9041B7A6430E@gmail.com>
 <4D2DB65C.1090106@aleynikov.org>
 <AANLkTindOp++U1PeaXVyCRRLgRH9GwC8HZdroES8kLRY@mail.gmail.com>
 <4D2DE13E.7040404@aleynikov.org>
 <AANLkTim5t-MgBFuvbUyoejQN3HawW2SNC4Fa=GMOP+nN@mail.gmail.com>
 <4D33B762.3080202@aleynikov.org> <4D90ABBD.8050100@erix.ericsson.se>
 <4F58F46A.5050701@aleynikov.org> <4F5F52FE.7060003@erlang.org>
 <4F5F8097.5060003@erix.ericsson.se> <4F8EE138.60402@aleynikov.org>
 <4F9129B5.6020304@erlang.org> <4F924348.6070400@aleynikov.org>
 <4FBDFCB2.2050402@erlang.org> <4FBE3F78.8050405@aleynikov.org>
 <50067F49.4020206@erlang.org>
Message-ID: <500704BE.1090709@aleynikov.org>

On 7/18/2012 5:18 AM, Lukas Larsson wrote:
> However, I would also like the fast functionality to be used by
> float_to_list_1 as well, is possible to do this and stay backwards
> compatible? Hopefully you just have to shift the comma and add e+XX at
> the end of the optimized case and call sys_double_to_chars for the
> unoptimized.

See my comments below regarding 'scientific' option.  Since
float_to_list_2 is a new function I would think that you are questioning
the issue of backward compatibility only in terms of converting
float_to_list_1 to use float_to_list_2 implementation.  However, I think
that this will have adverse performance tax on float_to_list_2, which
will diminish the benefit.  It's been a while however, since I wrote
that patch, perhaps there's a way to retrofit scientific notation
without a performance penalty.

It's more than simple shifting of the comma, since there's also rounding
involved.

This case is easy:

4> float_to_list(1.01234).
"1.01234000000000001762e+00"
5> float_to_list(1.01234, [{decimals, 20}]).
"1.01234000000000001762"

This case is a bit more complex (illustration of rounding impact):

7> float_to_list(10123412345.0123451234).
"1.01234123450123443604e+10"
8> float_to_list(10123412345.0123451234, [{decimals, 20}]).
"10123412345.01234436035156250000"

> Also float_to_list(1.0) should return the same thing as
> float_to_list(1.0,[]), otherwise the API will be inconsistent with how
> other such APIs work.

Actually if you trace this subject back there was another request that
the default number of decimals is chosen to be consistent with what
printf() does, so I changed the implementation to accommodate that request:

Eshell V5.9  (abort with ^G)
1> float_to_list(1.0, []).
"1.000000"

Which is different from the default of float_to_list/1:

2> float_to_list(1.0).
"1.00000000000000000000e+00"

Maybe one could introduce a 'scientific' option to float_to_list/2, to
use the float_to_list/1 implementation?

Serge

> On 24/05/12 16:02, Serge Aleynikov wrote:
>> Henrik,
>>
>> Fetch:    git fetch https://github.com/saleyn/otp/tree/float_to_list_2
>> Diff:    https://github.com/saleyn/otp/compare/float_to_list_2
>>
>> I added the definition for the new BIF to make the type checker happy:
>> https://github.com/saleyn/otp/commit/f9ddbeda5426ca83cda03c06a9860220ea4a22c7
>>
>>
>> Once you do the "otp_build tests", how do you execute all tests suites
>> in $ERL_TOP or if possible only tests in a given SUITE?  I tried the
>> following but all tests fail:
>>
>> [otp/erts/emulator/test]$ ../../../bin/erl -noshell -s test_server_ctrl
>> run_test DIR "." -s erlang halt
>>
>> I did however run individual tests in bif_SUIT:types to make sure my
>> patch didn't break anything.
>>
>>
>>
>> Serge
>>
>> On 5/24/2012 5:17 AM, Henrik Nord wrote:
>>> Hi again.
>>>
>>>
>>> This test is not passing: emulator/bif_SUIT:types
>>>
>>> No type information:
>>>
>>> [{erlang,float_to_list,2}]
>>>
>>>
>>>
>>> On 04/21/2012 07:19 AM, Serge Aleynikov wrote:
>>>> git fetch https://github.com/saleyn/otp/tree/float_to_list_2
>> _______________________________________________
>> erlang-patches mailing list
>> erlang-patches@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-patches
>>
> 
> 


From cyg.cao@REDACTED  Thu Jul 19 04:48:10 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Thu, 19 Jul 2012 10:48:10 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <5006D805.9070607@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4RazCb-N71h79Uk70vgfhCJtwwCitDsgR4dwWYikA4j3L3g@mail.gmail.com>
 <5006D805.9070607@erlang.org>
Message-ID: <CADS4Rawh+wk0d=gFvMmxN0NjbmOK5oSaRbdt-XVLzyNtuvoAXg@mail.gmail.com>

Thanks for pointing it out, fixed and pushed to pollset_per_scheduler
branch at git://github.com/weicao/otp.git

2012/7/18 Lukas Larsson <lukas@REDACTED>:
> fyi the patch has introduced three new gcc warnings:
>
> erl_port_task.c:230, gcc4,
>
> control reaches end of non-void function [-Wreturn-type]
>
> erl_check_io.c:992, gcc4,
>
> ?inps? may be used uninitialized in this function [-Wuninitialized]
>
> erl_check_io.c:998, gcc4,
>
> ?outps? may be used uninitialized in this function [-Wuninitialized]
>
> Lukas
>
>
> On 18/07/12 11:28, Wei Cao wrote:
>>
>> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>>>
>>> Hi!
>>>
>>> After fixing the patch to compile on windows[1] I put it in our daily
>>> builds
>>> and a couple of issues came up. I did not include 'move erts_deliver_time
>>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>>
>> This problem is also fixed by testing macro ERTS_SMP to tell whether
>> it's a non-smp emulator, now both smp and non-smp emulators work.
>>
>>> When running a debug test build on Linux we got the following assertion:
>>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>>> sys/common/erl_check_io.c, line 1909
>>>
>>> On OS X Lion we got the following when compiling gs:
>>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>>> Invalid argument (22)
>>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>>> make[2]: *** [opt] Error 2
>>> make[1]: *** [opt] Error 2
>>> make: *** [libs] Error 2
>>>
>>> I'll remove the branch and see if the same problems appear again tomorrow
>>> (unfortunately we do not have enough machines to let your branch run
>>> alone,
>>> so it might be some other branch causing this). Let me know if you need
>>> any
>>> help tracking down these issues.
>>>
>>> Lukas
>>>
>>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>>
>>>
>>> On 11/07/12 17:34, Wei Cao wrote:
>>>
>>> In non keep-alive cases, all new connections 're accepted by the
>>> Erlang port which listens on the TCP port, and how frequently/fast the
>>> port be scheduled to run limits the QPS. (requests per second), so
>>> this port can be regarded as bottleneck of non keep-alive
>>> applications.
>>>
>>> So I guess performance degradation observed is caused by the listener
>>> port not be scheduled frequent or fast enough, I'll look into this
>>> problem tomorrow, now is at night in China, :-)
>>>
>>> BTW, I found this patch should be compiled like this today,
>>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>>> -fomit-frame-pointer"
>>> otherwise compiler optimization is disabled.
>>>
>>> Regarding binding processes/ports to scheduler, I admit it's really a
>>> temporary solution to bind port to the same scheduler as its owner
>>> process like the pb patch did, and I suggest it's better to add a
>>> additional BIF like erlang:process_flag, to allow user explicitly bind
>>> port to a given scheduler, if it benefits.
>>>
>>>
>>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>>
>>> Hi,
>>>
>>> The reason I'm skeptical about anything which binds processes/ports to
>>> scheduler is that it feels like a temporary solution and would much
>>> rather
>>> do a proper solution where the scheduler takes care of these things for
>>> you.
>>> But as I said, internally we need to talk this over when it is not in the
>>> middle of summer vacation.
>>>
>>> I did some benchmarking using ab and found basically the same figures as
>>> you. The below is with keep-alive and the values are requests per second:
>>>
>>>                     not-bound        bound
>>>
>>> R15B01                 44k        37k
>>>
>>> master                  44k        35k
>>>
>>> master+mp          48k        49k
>>>
>>> master+mp+pb    49k        55k
>>>
>>> [mp]: multi-poll patch
>>> [pb]: port bind patch
>>> [bound]: Used {scheduler,I} to spread load
>>>
>>> Unfortunately I also found that when doing the non-keep alive benchmark
>>> the
>>> performance is seriously degraded.
>>>
>>> R15B01 not-bound                  8255
>>> master+mp+pb not-bound    7668
>>> master+mp+pb bound           5765
>>>
>>> I did some gprof runs but could not find anything obvious that is going
>>> wrong.
>>>
>>> Lukas
>>>
>>>
>>> On 11/07/12 04:21, Wei Cao wrote:
>>>
>>> I added a macro to conditional compile the patch because I think it
>>> can be more selectable, I can remove the macro, fix the compilation
>>> error and test on mingw platform in later version.
>>>
>>> how about provide another BIF named port_flag (like process_flag) to
>>> let user bind port to a given scheduler?
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
>



-- 

Best,

Wei Cao


From jonas.falkevik@REDACTED  Fri Jul 20 00:46:55 2012
From: jonas.falkevik@REDACTED (Jonas Falkevik)
Date: Fri, 20 Jul 2012 00:46:55 +0200
Subject: [erlang-patches] SCTP Peeloff socket can block erlang VM if set to
	{active, true}
Message-ID: <C0B7BBD6-F5BE-4E55-9D81-391246478C19@mobilearts.com>

When using SCTP sockets and peeloff there is a problem if trying to set the socket to {active,true}.

The new socket returned by the sctp_peeloff call is not set to non-blocking, which the reused packet_inet_input() is relying on.
There is a default packet poll for 5 packet and if there are less then five packets, the receive will block the thread unless the socket is non-blocking.

git fetch git@REDACTED:falkevik/otp.git fix_sctp_peeloff_active_true

https://github.com/falkevik/otp/compare/falkevik:master...falkevik:fix_sctp_peeloff_active_true
https://github.com/falkevik/otp/compare/falkevik:master...falkevik:fix_sctp_peeloff_active_true.patch
https://github.com/falkevik/otp/commit/7adbaf8b29f1411c4b83a5911869adabc5e6038b.patch


Test case added so that both {active, true} and {active, once} are tested after sctp_peeloff().

git fetch git@REDACTED:falkevik/otp.git sctp_peeloff_active_true_test

https://github.com/falkevik/otp/compare/falkevik:master...falkevik:sctp_peeloff_active_true_test
https://github.com/falkevik/otp/compare/falkevik:master...falkevik:sctp_peeloff_active_true_test.patch
https://github.com/falkevik/otp/commit/044604a655db0d3329e589638d40f8fb075b6057.patch


Jonas

From stefan.zegenhagen@REDACTED  Fri Jul 20 10:24:38 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 10:24:38 +0200
Subject: [erlang-patches] SSH: allow specifying 'auth_methods' in server
 opts
In-Reply-To: <1340874470.3506.5.camel@ax-sze>
References: <1340874470.3506.5.camel@ax-sze>
Message-ID: <1342772678.3472.19.camel@ax-sze>

Dear all,

I've not heard any comment on the patch below. Anyone interested???

Kind regards,

Stefan.

> the 'auth_methods' option to the SSH server is still handled by the
> code, but is not allowed to be specified in Opts. The patch below
> corrects the issue.
> 
> --- snip ---
> 
> From f71d45af6f3565cede4de444a0f8f691abbb5945 Mon Sep 17 00:00:00 2001
> From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
> Date: Thu, 28 Jun 2012 11:02:45 +0200
> Subject: [PATCH] SSH: Make "auth_methods" server option re-usable
> 
> The 'auth_methods' option is used by the server side of the SSH code to
> tell a connecting SSH client about the authentication methods that are
> supported by the server. The code still extracts and handles the
> 'auth_methods' option from Opts in appropriate places, but the Opts
> checking code in ssh.erl didn't allow that option to be specified.
> ---
>  lib/ssh/src/ssh.erl |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lib/ssh/src/ssh.erl b/lib/ssh/src/ssh.erl
> index f4a40c8..85f5f68 100644
> --- a/lib/ssh/src/ssh.erl
> +++ b/lib/ssh/src/ssh.erl
> @@ -369,6 +369,8 @@ handle_option([{shell, _} = Opt | Rest], SocketOptions, SshOptions) ->
>      handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]);
>  handle_option([{exec, _} = Opt | Rest], SocketOptions, SshOptions) ->
>      handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]);
> +handle_option([{auth_methods, _} = Opt | Rest], SocketOptions, SshOptions) ->
> +    handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]);
>  handle_option([Opt | Rest], SocketOptions, SshOptions) ->
>      handle_option(Rest, [handle_inet_option(Opt) | SocketOptions], SshOptions).
> 
> @@ -408,6 +410,8 @@ handle_ssh_option({exec, {Module, Function, _}} = Opt) when is_atom(Module),
>                                 is_atom(Function) ->
> 
>      Opt;
> +handle_ssh_option({auth_methods, Value} = Opt)  when is_list(Value) ->
> +    Opt;
>  handle_ssh_option({infofun, Value} = Opt)  when is_function(Value) ->
>      Opt;
>  handle_ssh_option({connectfun, Value} = Opt) when is_function(Value) ->
> -- 
> 1.7.9.5
> 

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From stefan.zegenhagen@REDACTED  Fri Jul 20 10:25:22 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 10:25:22 +0200
Subject: [erlang-patches] SNMPD: bugfix to
 snmp_generic:split_index_to_keys/2
In-Reply-To: <1341412924.26870.2.camel@ax-sze>
References: <1341412924.26870.2.camel@ax-sze>
Message-ID: <1342772722.3472.20.camel@ax-sze>

Dear all,

I've not heard any comment on the patch below. Anyone interested???

Kind regards,

Stefan.

Am Mittwoch, den 04.07.2012, 16:42 +0200 schrieb Stefan Zegenhagen:
> Dear all,
> 
> this mail contains a patch that solves a problem with the indicated
> function. For dynamically-sized table indices it wasn't checking whether
> the length of the index value matches all constraints specified in the
> MIB.
> 
> Kind regards,
> 
> --- snip ---
> 
> From 12c80f51879da4d36019cf736c1009e5bc09b908 Mon Sep 17 00:00:00 2001
> From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
> Date: Wed, 4 Jul 2012 14:10:00 +0200
> Subject: [PATCH 1/2] fix snmp_generic:split_index_to_keys/2
> 
> The function snmp_generic:split_index_to_keys/2 handles dynamic length
> RowIndex members, but fails to check that their length actually falls
> within the constraints given by the MIB.
> 
> The problem becomes real for table indices having an 'OCTET STRING'
> syntax, e.g. OCTET STRING(1..32) where it is possible to pass
> zero-length strings or strings longer than 32 chars without any
> validation failure.
> ---
>  lib/snmp/src/agent/snmp_generic.erl |   20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/snmp/src/agent/snmp_generic.erl b/lib/snmp/src/agent/snmp_generic.erl
> index 06afa68..0dd1f8b 100644
> --- a/lib/snmp/src/agent/snmp_generic.erl
> +++ b/lib/snmp/src/agent/snmp_generic.erl
> @@ -640,11 +640,12 @@ collect_keys([#asn1_type{lo = X, hi = X} | _Indexes], Keys)
>  %% Otherwise, its a dynamic-length type => its a list
>  %% OBJECT IDENTIFIER, OCTET STRING or BITS (or derivatives)
>  %% Check if it is IMPLIED (only last element can be IMPLIED)
> -collect_keys([#asn1_type{implied = true}], Keys) ->
> -    [Keys];
> -collect_keys([_Type | Indexes], [Length | Keys]) when length(Keys) >= Length ->
> +%% and also check lo/hi constraints...
> +collect_keys([#asn1_type{implied = true} = Type], Keys) ->
> +    [collect_check_length(Type, Keys)];
> +collect_keys([Type | Indexes], [Length | Keys]) when length(Keys) >= Length ->
>      {StrKey, Rest} = collect_length(Length, Keys, []),
> -    [StrKey | collect_keys(Indexes, Rest)];
> +    [collect_check_length(Type, StrKey) | collect_keys(Indexes, Rest)];
>  collect_keys([_Type | _Indexes], [Length | Keys]) ->
>      exit({error, {size_mismatch, Length, Keys}});
>  collect_keys([], []) -> [];
> @@ -657,6 +658,17 @@ collect_length(0, Rest, Rts) ->
>  collect_length(N, [El | Rest], Rts) ->
>      collect_length(N-1, Rest, [El | Rts]).
>  
> +collect_check_length(#asn1_type{lo = Lo, hi = Hi}, ListVal) ->
> +    Length = length(ListVal),
> +    if
> +        is_integer(Lo) andalso Length < Lo ->
> +            exit({error, {size_mismatch, Lo, ListVal}});
> +        is_integer(Hi) andalso Length > Hi ->
> +            exit({error, {size_mismatch, Hi, ListVal}});
> +        true ->
> +            ListVal
> +    end.
> +
>  %%------------------------------------------------------------------
>  %% Checks if a certain row exists.
>  %% Returns true or false.
> -- 
> 1.7.9.5
> 
> 

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From stefan.zegenhagen@REDACTED  Fri Jul 20 10:25:41 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 10:25:41 +0200
Subject: [erlang-patches] SNMP: verify RowIndex in table set requests
In-Reply-To: <1341413610.26870.10.camel@ax-sze>
References: <1341413610.26870.10.camel@ax-sze>
Message-ID: <1342772741.3472.21.camel@ax-sze>

Dear all,

I've not heard any comment on the patch below. Anyone interested???

Kind regards,

Stefan.

> Dear all,
> 
> during intensive testing it was found that it's possible to create table
> rows in snmpTargetAddrTable where the row index (snmpTargetAddrName ::
> SnmpAdminString(1..32)) is the empty string. This patch solves the issue
> generically by checking for all set-requests that the row index is of
> acceptable length before calling the table instrumentation function.
> 
> The same is not done for get requests because such invalid row indices
> should not be present if it's impossible to create them.
> 
> The same is not done for get-next requests because the semantic of the
> operation requires it to be able to cope with incomplete/overlong row
> indices.
> 
> Kind regards,
> 
> 
> --- snip ---
> 
> From be617c829a89dd9b73768181810776e50fb36c42 Mon Sep 17 00:00:00 2001
> From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
> Date: Wed, 4 Jul 2012 14:34:46 +0200
> Subject: [PATCH 2/2] verify table row index constraints in SNMP set requests
> 
> When a set-request is received for table columns, check that the varbind
> OIDs are syntactically correct. This includes checking that Table Entry
> OID as well as the Column OID are present, and that the row index
> contains an acceptable number of subidentifiers.
> 
> This check was not done carefully enough before, giving the following
> test result from an erlang-snmp agent:
> 
>  Response: Version = SNMPv3, MsgId = 173, MaxSize = 2048
>  Flags = Auth/Priv, SecurityModel = USM
>  EngineId = [...], EngineBoots = 16, EngineTime = 81244, UserName = ...
>  ContextEngineId = [...], ContextName = [...]
>  Request Id = 173, Error Status = No Error , Error Index = 0
>  Oid1 = snmpTargetAddrRowStatus , Type = Integer, Value = 4
>  Oid2 = snmpTargetAddrTAddress , Type = OctetString, Value = ...
>  Oid3 = snmpTargetAddrTimeout , Type = Integer, Value = 100
>  Oid4 = snmpTargetAddrRetryCount , Type = Integer, Value = 3
>  Oid5 = snmpTargetAddrTagList , Type = OctetString, Value = abc
>  Oid6 = snmpTargetAddrParams , Type = OctetString, Value = abc
>  Oid7 = snmpTargetAddrStorageType , Type = Integer, Value = 2
>  Oid8 = snmpTargetAddrTDomain , Type = ObjectID, Value = 1.3.6.1.6.1.1
> 
> Please observe that a table row in snmptargetAddrParams is created
> without errors, although an empty RowIndex was specified for all
> columnar objects in the request. The index to that table is defined as
> IMPLIED SnmpAdminString(1..32).
> ---
>  lib/snmp/src/agent/snmpa_set_lib.erl |   65 ++++++++++++++++++++++++++++++++--
>  1 file changed, 63 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/snmp/src/agent/snmpa_set_lib.erl b/lib/snmp/src/agent/snmpa_set_lib.erl
> index f5218d5..ef9e20e 100644
> --- a/lib/snmp/src/agent/snmpa_set_lib.erl
> +++ b/lib/snmp/src/agent/snmpa_set_lib.erl
> @@ -70,8 +70,19 @@
>  %% Purpose: Call is_varbind_ok for each varbind
>  %% Returns: {noError, 0} | {ErrorStatus, ErrorIndex}
>  %%-----------------------------------------------------------------
> -is_varbinds_ok([{_TableOid, TableVbs} | IVarbinds]) -> 
> -    is_varbinds_ok(lists:append(TableVbs, IVarbinds));
> +is_varbinds_ok([{TableOid, TableVbs} | IVarbinds]) ->
> +    % TableOid points to the table entry, not the table itself.
> +    % for table row index checking, we need the table entry, though.
> +    % Hence, strip the last OID.
> +    NTableOid = lists:reverse(tl(lists:reverse(TableOid))),
> +    case is_table_varbinds_ok(NTableOid, TableVbs) of
> +        true ->
> +            is_varbinds_ok(lists:append(TableVbs, IVarbinds));
> +        {ErrorStatus, Varbind} ->
> +            ?vtrace("varbinds erroneous: ~p -> ~p",
> +                [Varbind#varbind.org_index,ErrorStatus]),
> +            {ErrorStatus, Varbind#varbind.org_index}
> +    end;
>  is_varbinds_ok([IVarbind | IVarbinds]) -> 
>      case catch is_varbind_ok(IVarbind) of
>  	true -> is_varbinds_ok(IVarbinds);
> @@ -85,6 +96,56 @@ is_varbinds_ok([]) ->
>      ?vtrace("varbinds ok",[]),
>      {noError, 0}.
>  
> +
> +%%-----------------------------------------------------------------
> +%% Func: is_table_varbinds_ok/2
> +%% Purpose: Perform basic checks on the RowIndex of varbinds against
> +%%          the MIB. Here we don't call any instrumentation functions.
> +%% Returns: true |
> +%% Fails:   with an <error-atom>.
> +%%-----------------------------------------------------------------
> +is_table_varbinds_ok(_TableOid, []) ->
> +    true;
> +is_table_varbinds_ok(TableOid, [Hd | _] = Varbinds) ->
> +    ErrResult = {noCreation, Hd#ivarbind.varbind},
> +    % resolve table information, it's required for the checks we do
> +    case snmpa_symbolic_store:oid_to_aliasname(TableOid) of
> +        {value, TableName} ->
> +            case snmpa_symbolic_store:table_info(TableName) of
> +                {value, #table_info{} = TableInfo} ->
> +                    is_table_varbinds_ok(TableOid, TableInfo, Varbinds);
> +                false ->
> +                    snmpa_error:user_err("Failed in table_info(~w)", [TableName]),
> +                    ErrResult
> +            end;
> +        false ->
> +            snmpa_error:user_err("Failed in oid_to_aliasname(~w)", [TableOid]),
> +            ErrResult
> +    end.
> +
> +is_table_varbinds_ok(TableOid, TableInfo, [#ivarbind{varbind = Varbind} | Varbinds]) ->
> +    Indices = TableInfo#table_info.index_types,
> +    Oid = Varbind#varbind.oid,
> +    case snmp_misc:diff(Oid, TableOid) of
> +        [_TblEntryId, _TblColumnId | RowIndex] ->
> +            case (catch snmp_generic:split_index_to_keys(Indices, RowIndex)) of
> +                Keys when is_list(Keys) ->
> +                    % RowIndex seems to be OK
> +                    is_table_varbinds_ok(TableOid, TableInfo, Varbinds);
> +                Error ->
> +                    % RowIndex is syntactically wrong...
> +                    ?vtrace("table varbind RowIndex erroneous: ~p", [Error]),
> +                    {noCreation, Varbind}
> +            end;
> +        _Else ->
> +            % OID is syntactically wrong...
> +            ?vtrace("table varbind OID erroneous: ~p", [Oid]),
> +            {noCreation, Varbind}
> +    end;
> +is_table_varbinds_ok(_TableOid, _TableInfo, []) ->
> +    true.
> +
> +
>  %%-----------------------------------------------------------------
>  %% Func: is_varbind_ok/1
>  %% Purpose: Check everything we can check about the varbind against
> -- 
> 1.7.9.5
> 
> 

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From sperber@REDACTED  Fri Jul 20 11:30:51 2012
From: sperber@REDACTED (Michael Sperber)
Date: Fri, 20 Jul 2012 11:30:51 +0200
Subject: [erlang-patches] Float endianness in 15B
In-Reply-To: <50066849.5060303@erlang.org> (Lukas Larsson's message of "Wed,
 18 Jul 2012 09:39:53 +0200")
References: <y9lehsjrano.fsf@deinprogramm.de> <4FFD87A0.6060503@erlang.org>
 <y9l8veikdn5.fsf@deinprogramm.de> <50066849.5060303@erlang.org>
Message-ID: <y9lpq7qeq44.fsf@deinprogramm.de>


Lukas Larsson <lukas@REDACTED> writes:

> The patch just ran through our nightly build and a couple of issues
> popped up.
>
> It seems that the ac script does not correct detect double endianess
> on big endian platforms. The configure on all of our sparc and ppc
> machines detected double word ordering as middle-endian.
>
> Also don't think an error should be thrown in configure if we cannot
> detect the double endianess. Instead a warning should be issued and
> configure should continue with the assumption that the doubles are not
> middle-endian. There are a number of options that already do this and
> I think it makes sense here as we can never detect it in a cross
> environment.

Thanks for the feedback, and sorry for the hassle.  I re-wrote the test
and found a Solaris SPARC to test on - it should work better.  I also
turned the error message into a warning.  The result is here:

https://github.com/mikesperber/otp/commit/eba4d247740189c3b8045edeaf8224651e77a3e0

-- 
Regards,
Mike


From stefan.zegenhagen@REDACTED  Fri Jul 20 11:41:48 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 11:41:48 +0200
Subject: [erlang-patches] Problems with erlang SSH server
Message-ID: <1342777308.3472.26.camel@ax-sze>

Dear all,

The patch below fixes a few problems with the erlang SSH server that
prevents numerous SSH clients from connecting/operating correctly with
an erlang-SSH server.

Kind regards,


--- snip ---

>From acb7a76a5cd46d03f20e74db8d7b0a7723284814 Mon Sep 17 00:00:00 2001
From: Stefan Zegenhagen <stefan.zegenhagen@REDACTED>
Date: Fri, 20 Jul 2012 11:38:57 +0200
Subject: [PATCH] Improve interoperability with SSH clients

This patch fixes a few problems of the SSH shell server that affect the
interoperability with SSH clients in widespread use.

First problem is that, whenever a channel_request message is
received with want_reply=true, the reply ends up being sent to the
servers channel id, not the clients channel id. This causes the client
to terminate the connection. The easiest solution to the problem appears
to be a new function in ssh_connection_manager.erl that translates the
servers channel id before sending the reply (in the same manner as
other functions do it).

Second problem is in ssh_cli.erl. When an SSH client sends a
window_change request between PTY allocation and starting the shell
(which appears to happen with some clients), ssh_cli.erl crashes
because #state.buf is yet 'undefined'. Allocating an empty buffer at PTY
allocation time solves the problem.

Affected SSH clients:
 - all clients based on SSH-2.0-TrileadSSH2Java_213 (problem #1)
 - SSH Term Pro (problem #2)
---
 lib/ssh/src/ssh_cli.erl                |    3 ++-
 lib/ssh/src/ssh_connection.erl         |    2 +-
 lib/ssh/src/ssh_connection_manager.erl |   15 ++++++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl
index 781e01b..c8c610f 100644
--- a/lib/ssh/src/ssh_cli.erl
+++ b/lib/ssh/src/ssh_cli.erl
@@ -81,7 +81,8 @@ handle_ssh_msg({ssh_cm, ConnectionManager,
 				  height = not_zero(Height, 24),
 				  pixel_width = PixWidth,
 				  pixel_height = PixHeight,
-				  modes = Modes}},
+                  modes = Modes},
+             buf = empty_buf()},
     set_echo(State),
     ssh_connection:reply_request(ConnectionManager, WantReply, 
 				 success, ChannelId),
diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
index c46f799..913f505 100644
--- a/lib/ssh/src/ssh_connection.erl
+++ b/lib/ssh/src/ssh_connection.erl
@@ -177,7 +177,7 @@ close(ConnectionManager, ChannelId) ->
 %% Description: Send status replies to requests that want such replies.
 %%--------------------------------------------------------------------
 reply_request(ConnectionManager, true, Status, ChannelId) ->
-    ConnectionManager ! {ssh_cm, self(), {Status, ChannelId}},
+    ssh_connection_manager:reply_request(ConnectionManager, Status, ChannelId),
     ok;
 reply_request(_,false, _, _) ->
     ok.
diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl
index e53cd4f..e350f96 100644
--- a/lib/ssh/src/ssh_connection_manager.erl
+++ b/lib/ssh/src/ssh_connection_manager.erl
@@ -40,7 +40,7 @@
 	 close/2, stop/1, send/5,
 	 send_eof/2]).
 
--export([open_channel/6, request/6, request/7, global_request/4, event/2,
+-export([open_channel/6, reply_request/3, request/6, request/7, global_request/4, event/2,
 	 cast/2]).
 
 %% Internal application API and spawn
@@ -95,6 +95,9 @@ request(ConnectionManager, ChannelId, Type, true, Data, Timeout) ->
 request(ConnectionManager, ChannelId, Type, false, Data, _) ->
     cast(ConnectionManager, {request, ChannelId, Type, Data}).
 
+reply_request(ConnectionManager, Status, ChannelId) ->
+    cast(ConnectionManager, {reply_request, Status, ChannelId}).
+
 global_request(ConnectionManager, Type, true = Reply, Data) ->
     case call(ConnectionManager, 
 	      {global_request, self(), Type, Reply, Data}) of
@@ -431,6 +434,16 @@ handle_cast({request, ChannelId, Type, Data}, State0) ->
     lists:foreach(fun send_msg/1, Replies),
     {noreply, State};
 
+handle_cast({reply_request, Status, ChannelId}, #state{connection_state =
+        #connection{channel_cache = Cache}} = State0) ->
+    State = case ssh_channel:cache_lookup(Cache, ChannelId) of
+        #channel{remote_id = RemoteId} ->
+            cm_message({Status, RemoteId}, State0);
+        undefined ->
+            State0
+    end,
+    {noreply, State};
+
 handle_cast({global_request, _, _, _, _} = Request, State0) ->
     State = handle_global_request(Request, State0),
     {noreply, State};
-- 
1.7.9.5



-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From henrik@REDACTED  Fri Jul 20 14:17:51 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 20 Jul 2012 14:17:51 +0200
Subject: [erlang-patches] Problems with erlang SSH server
In-Reply-To: <1342777308.3472.26.camel@ax-sze>
References: <1342777308.3472.26.camel@ax-sze>
Message-ID: <50094C6F.1060400@erlang.org>

Hi

I can not get your mailpatches to apply using git -am.

Could you resend them as git fetch links instead?

Thank you!


On 07/20/2012 11:41 AM, Stefan Zegenhagen wrote:
> Dear all,
>
> The patch below fixes a few problems with the erlang SSH server that
> prevents numerous SSH clients from connecting/operating correctly with
> an erlang-SSH server.
>
> Kind regards,
>
>
> --- snip ---
>
> > From acb7a76a5cd46d03f20e74db8d7b0a7723284814 Mon Sep 17 00:00:00 2001
> From: Stefan Zegenhagen<stefan.zegenhagen@REDACTED>
> Date: Fri, 20 Jul 2012 11:38:57 +0200
> Subject: [PATCH] Improve interoperability with SSH clients
>
> This patch fixes a few problems of the SSH shell server that affect the
> interoperability with SSH clients in widespread use.
>
> First problem is that, whenever a channel_request message is
> received with want_reply=true, the reply ends up being sent to the
> servers channel id, not the clients channel id. This causes the client
> to terminate the connection. The easiest solution to the problem appears
> to be a new function in ssh_connection_manager.erl that translates the
> servers channel id before sending the reply (in the same manner as
> other functions do it).
>
> Second problem is in ssh_cli.erl. When an SSH client sends a
> window_change request between PTY allocation and starting the shell
> (which appears to happen with some clients), ssh_cli.erl crashes
> because #state.buf is yet 'undefined'. Allocating an empty buffer at PTY
> allocation time solves the problem.
>
> Affected SSH clients:
>   - all clients based on SSH-2.0-TrileadSSH2Java_213 (problem #1)
>   - SSH Term Pro (problem #2)
> ---
>   lib/ssh/src/ssh_cli.erl                |    3 ++-
>   lib/ssh/src/ssh_connection.erl         |    2 +-
>   lib/ssh/src/ssh_connection_manager.erl |   15 ++++++++++++++-
>   3 files changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl
> index 781e01b..c8c610f 100644
> --- a/lib/ssh/src/ssh_cli.erl
> +++ b/lib/ssh/src/ssh_cli.erl
> @@ -81,7 +81,8 @@ handle_ssh_msg({ssh_cm, ConnectionManager,
>   				  height = not_zero(Height, 24),
>   				  pixel_width = PixWidth,
>   				  pixel_height = PixHeight,
> -				  modes = Modes}},
> +                  modes = Modes},
> +             buf = empty_buf()},
>       set_echo(State),
>       ssh_connection:reply_request(ConnectionManager, WantReply,
>   				 success, ChannelId),
> diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
> index c46f799..913f505 100644
> --- a/lib/ssh/src/ssh_connection.erl
> +++ b/lib/ssh/src/ssh_connection.erl
> @@ -177,7 +177,7 @@ close(ConnectionManager, ChannelId) ->
>   %% Description: Send status replies to requests that want such replies.
>   %%--------------------------------------------------------------------
>   reply_request(ConnectionManager, true, Status, ChannelId) ->
> -    ConnectionManager ! {ssh_cm, self(), {Status, ChannelId}},
> +    ssh_connection_manager:reply_request(ConnectionManager, Status, ChannelId),
>       ok;
>   reply_request(_,false, _, _) ->
>       ok.
> diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl
> index e53cd4f..e350f96 100644
> --- a/lib/ssh/src/ssh_connection_manager.erl
> +++ b/lib/ssh/src/ssh_connection_manager.erl
> @@ -40,7 +40,7 @@
>   	 close/2, stop/1, send/5,
>   	 send_eof/2]).
>
> --export([open_channel/6, request/6, request/7, global_request/4, event/2,
> +-export([open_channel/6, reply_request/3, request/6, request/7, global_request/4, event/2,
>   	 cast/2]).
>
>   %% Internal application API and spawn
> @@ -95,6 +95,9 @@ request(ConnectionManager, ChannelId, Type, true, Data, Timeout) ->
>   request(ConnectionManager, ChannelId, Type, false, Data, _) ->
>       cast(ConnectionManager, {request, ChannelId, Type, Data}).
>
> +reply_request(ConnectionManager, Status, ChannelId) ->
> +    cast(ConnectionManager, {reply_request, Status, ChannelId}).
> +
>   global_request(ConnectionManager, Type, true = Reply, Data) ->
>       case call(ConnectionManager,
>   	      {global_request, self(), Type, Reply, Data}) of
> @@ -431,6 +434,16 @@ handle_cast({request, ChannelId, Type, Data}, State0) ->
>       lists:foreach(fun send_msg/1, Replies),
>       {noreply, State};
>
> +handle_cast({reply_request, Status, ChannelId}, #state{connection_state =
> +        #connection{channel_cache = Cache}} = State0) ->
> +    State = case ssh_channel:cache_lookup(Cache, ChannelId) of
> +        #channel{remote_id = RemoteId} ->
> +            cm_message({Status, RemoteId}, State0);
> +        undefined ->
> +            State0
> +    end,
> +    {noreply, State};
> +
>   handle_cast({global_request, _, _, _, _} = Request, State0) ->
>       State = handle_global_request(Request, State0),
>       {noreply, State};

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Fri Jul 20 14:25:41 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 20 Jul 2012 14:25:41 +0200
Subject: [erlang-patches] Fix use of "clever" mktime.
In-Reply-To: <20120716220007.GA28144@puffy52.local>
References: <20120716220007.GA28144@puffy52.local>
Message-ID: <50094E45.6060403@erlang.org>

Thank you, I Have included this patch in 'pu'

On 07/16/2012 11:58 PM, Piotr Sikora wrote:
> Fix use of "clever" mktime.
>
> Commit 1eef765 introduced regression (conditional _always_ evaluates
> to true) in which erlang:localtime_to_universaltime/2 stopped working
> on systems configured with timezone without DST (i.e. UTC) on *BSD
> platforms:
>
> 1>  erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}, true).
> ** exception error: bad argument
>
>
> git fetch git://github.com/PiotrSikora/otp.git fix-clever-mktime
>
> https://github.com/PiotrSikora/otp/compare/fix-clever-mktime
> https://github.com/PiotrSikora/otp/compare/fix-clever-mktime.patch
>
>
> Best regards,
> Piotr Sikora<  piotr.sikora@REDACTED>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From henrik@REDACTED  Fri Jul 20 14:27:22 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 20 Jul 2012 14:27:22 +0200
Subject: [erlang-patches] Fix support for leap seconds-aware timezones.
In-Reply-To: <20120716220106.GB28144@puffy52.local>
References: <20120716220106.GB28144@puffy52.local>
Message-ID: <50094EAA.3080709@erlang.org>

Thank you, I have included this branch in 'pu'

On 07/16/2012 11:59 PM, Piotr Sikora wrote:
> Fix support for leap seconds-aware timezones.
>
> erlang:universaltime_to_localtime is leap seconds-aware (since 2008),
> however erlang:localtime_to_universaltime is not, which gives
> surprising results on systems configured with leap seconds-aware
> timezones:
>
> 1>  erlang:universaltime_to_localtime({{2012,1,1},{0,0,0}}).
> {{2012,1,1},{0,0,0}}
> 2>  erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}).
> {{2012,1,1},{0,0,24}}
>
> and completely breaks calendar:local_time_to_universal_time_dst:
>
> 3>  calendar:local_time_to_universal_time_dst({{2012,1,1},{0,0,0}}).
> []
>
>
> git fetch git://github.com/PiotrSikora/otp.git fix-leap-seconds
>
> https://github.com/PiotrSikora/otp/compare/fix-leap-seconds
> https://github.com/PiotrSikora/otp/compare/fix-leap-seconds.patch
>
>
> Note: this patch was tested on OpenBSD and it fixes the issue, but it should
> be tested on other operating systems with leap seconds-aware timezones.
> Information on how to enable them on Linux and FreeBSD can be found here:
> http://thedjbway.b0llix.net/clockspeed/leapsecs.html
>
> Best regards,
> Piotr Sikora<  piotr.sikora@REDACTED>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From stefan.zegenhagen@REDACTED  Fri Jul 20 14:28:10 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 14:28:10 +0200
Subject: [erlang-patches] Problems with erlang SSH server
In-Reply-To: <50094C6F.1060400@erlang.org>
References: <1342777308.3472.26.camel@ax-sze> <50094C6F.1060400@erlang.org>
Message-ID: <1342787290.3472.31.camel@ax-sze>

Dear Henrik,

> I can not get your mailpatches to apply using git -am.
> 
> Could you resend them as git fetch links instead?

Unfortunately, I'm not registered on github.

The patch submission guidelines say that patches can be sent inline in
mails to erlang-patches and I would like to stick to that possibility.

Maybe I just mis-interpreted that statement. Would it help to send the
patches
 - as attachments, or
 - posted directly using git-send-email
instead?


Kind regards,

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.



From henrik@REDACTED  Fri Jul 20 14:34:22 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 20 Jul 2012 14:34:22 +0200
Subject: [erlang-patches] Problems with erlang SSH server
In-Reply-To: <1342787290.3472.31.camel@ax-sze>
References: <1342777308.3472.26.camel@ax-sze> <50094C6F.1060400@erlang.org>
 <1342787290.3472.31.camel@ax-sze>
Message-ID: <5009504E.8010102@erlang.org>



On 07/20/2012 02:28 PM, Stefan Zegenhagen wrote:
> Dear Henrik,
>
>> I can not get your mailpatches to apply using git -am.
>>
>> Could you resend them as git fetch links instead?
> Unfortunately, I'm not registered on github.
>
> The patch submission guidelines say that patches can be sent inline in
> mails to erlang-patches and I would like to stick to that possibility.
>
> Maybe I just mis-interpreted that statement. Would it help to send the
> patches
>   - as attachments, or
>   - posted directly using git-send-email
> instead?
Well if you really dont want to register at github I cant force you.
But please make sure that you can apply the patches.

you can try it by saving your email and

$ cat ~/ldisk/mail-patches/\[erlang-patches\]\ SSH\:\ allow\ specifying\ 
\'auth_methods\'\ in\ server\ opts.eml |git am


There might be some help in here regarding git format-patch
https://github.com/erlang/otp/wiki/Submitting-patches


>
> Kind regards,
>

-- 
/Henrik Nord Erlang/OTP



From jonas.falkevik@REDACTED  Fri Jul 20 14:36:50 2012
From: jonas.falkevik@REDACTED (Jonas Falkevik)
Date: Fri, 20 Jul 2012 14:36:50 +0200
Subject: [erlang-patches] SCTP Peeloff socket can block erlang VM if set
	to {active, true}
In-Reply-To: <C0B7BBD6-F5BE-4E55-9D81-391246478C19@mobilearts.com>
References: <C0B7BBD6-F5BE-4E55-9D81-391246478C19@mobilearts.com>
Message-ID: <5D51F801-4627-401F-A342-CE5537A6133A@mobilearts.com>

The previous git fetch command seem to faulty, sorry for that.
Let me try again.

On Jul 20, 2012, at 0:46 , Jonas Falkevik wrote:
> 
> git fetch git@REDACTED:falkevik/otp.git fix_sctp_peeloff_active_true
git fetch git://github.com/falkevik/otp.git fix_sctp_peeloff_active_true

> 
> git fetch git@REDACTED:falkevik/otp.git sctp_peeloff_active_true_test
git fetch git://github.com/falkevik/otp.git sctp_peeloff_active_true_test


Jonas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120720/26c56fc1/attachment.htm>

From lukas@REDACTED  Fri Jul 20 14:38:58 2012
From: lukas@REDACTED (Lukas Larsson)
Date: Fri, 20 Jul 2012 14:38:58 +0200
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <CADS4Rawh+wk0d=gFvMmxN0NjbmOK5oSaRbdt-XVLzyNtuvoAXg@mail.gmail.com>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
 <CADS4Rawy-hVQ4BfEMbbEbyF-uC3eeyKZm2=gzNTJmZJApJWd3Q@mail.gmail.com>
 <50053E99.7090909@erlang.org>
 <CADS4RazCb-N71h79Uk70vgfhCJtwwCitDsgR4dwWYikA4j3L3g@mail.gmail.com>
 <5006D805.9070607@erlang.org>
 <CADS4Rawh+wk0d=gFvMmxN0NjbmOK5oSaRbdt-XVLzyNtuvoAXg@mail.gmail.com>
Message-ID: <50095162.7060904@erlang.org>

Hello!

The multiple-pollset branch seems to be running without any testcase 
failures in our nightly builds now. We will have to review the code in 
detail and then get back to you, unfortunately that could take a while 
as it is in the middle of vacations times and we are also closing in on 
the next maintenance release.

Regarding binding ports to schedulers:

We do not want to include the functionality the way it is now. It builds 
on a mechanism which really should only be used for testing and can 
create unforeseeable problems with process/port scheduling and scheduler 
load balancing.

If the functionality was changed so that a process_flag could be set on 
a process so that all ports created by that process are bound to the 
process (and migrate with the process), you could still achieve the same 
results but through a functionality which is more general. Depending on 
what changes have to be made to the erlang vm, we might accept such a 
patch. If it works we might even document and support the functionality.

Lukas

On 19/07/12 04:48, Wei Cao wrote:
> Thanks for pointing it out, fixed and pushed to pollset_per_scheduler
> branch at git://github.com/weicao/otp.git
>
> 2012/7/18 Lukas Larsson <lukas@REDACTED>:
>> fyi the patch has introduced three new gcc warnings:
>>
>> erl_port_task.c:230, gcc4,
>>
>> control reaches end of non-void function [-Wreturn-type]
>>
>> erl_check_io.c:992, gcc4,
>>
>> ?inps? may be used uninitialized in this function [-Wuninitialized]
>>
>> erl_check_io.c:998, gcc4,
>>
>> ?outps? may be used uninitialized in this function [-Wuninitialized]
>>
>> Lukas
>>
>>
>> On 18/07/12 11:28, Wei Cao wrote:
>>> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>>>> Hi!
>>>>
>>>> After fixing the patch to compile on windows[1] I put it in our daily
>>>> builds
>>>> and a couple of issues came up. I did not include 'move erts_deliver_time
>>>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
>>> This problem is also fixed by testing macro ERTS_SMP to tell whether
>>> it's a non-smp emulator, now both smp and non-smp emulators work.
>>>
>>>> When running a debug test build on Linux we got the following assertion:
>>>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>>>> sys/common/erl_check_io.c, line 1909
>>>>
>>>> On OS X Lion we got the following when compiling gs:
>>>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>>>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>>>> Invalid argument (22)
>>>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>>>> make[2]: *** [opt] Error 2
>>>> make[1]: *** [opt] Error 2
>>>> make: *** [libs] Error 2
>>>>
>>>> I'll remove the branch and see if the same problems appear again tomorrow
>>>> (unfortunately we do not have enough machines to let your branch run
>>>> alone,
>>>> so it might be some other branch causing this). Let me know if you need
>>>> any
>>>> help tracking down these issues.
>>>>
>>>> Lukas
>>>>
>>>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>>>
>>>>
>>>> On 11/07/12 17:34, Wei Cao wrote:
>>>>
>>>> In non keep-alive cases, all new connections 're accepted by the
>>>> Erlang port which listens on the TCP port, and how frequently/fast the
>>>> port be scheduled to run limits the QPS. (requests per second), so
>>>> this port can be regarded as bottleneck of non keep-alive
>>>> applications.
>>>>
>>>> So I guess performance degradation observed is caused by the listener
>>>> port not be scheduled frequent or fast enough, I'll look into this
>>>> problem tomorrow, now is at night in China, :-)
>>>>
>>>> BTW, I found this patch should be compiled like this today,
>>>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>>>> -fomit-frame-pointer"
>>>> otherwise compiler optimization is disabled.
>>>>
>>>> Regarding binding processes/ports to scheduler, I admit it's really a
>>>> temporary solution to bind port to the same scheduler as its owner
>>>> process like the pb patch did, and I suggest it's better to add a
>>>> additional BIF like erlang:process_flag, to allow user explicitly bind
>>>> port to a given scheduler, if it benefits.
>>>>
>>>>
>>>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>>>
>>>> Hi,
>>>>
>>>> The reason I'm skeptical about anything which binds processes/ports to
>>>> scheduler is that it feels like a temporary solution and would much
>>>> rather
>>>> do a proper solution where the scheduler takes care of these things for
>>>> you.
>>>> But as I said, internally we need to talk this over when it is not in the
>>>> middle of summer vacation.
>>>>
>>>> I did some benchmarking using ab and found basically the same figures as
>>>> you. The below is with keep-alive and the values are requests per second:
>>>>
>>>>                      not-bound        bound
>>>>
>>>> R15B01                 44k        37k
>>>>
>>>> master                  44k        35k
>>>>
>>>> master+mp          48k        49k
>>>>
>>>> master+mp+pb    49k        55k
>>>>
>>>> [mp]: multi-poll patch
>>>> [pb]: port bind patch
>>>> [bound]: Used {scheduler,I} to spread load
>>>>
>>>> Unfortunately I also found that when doing the non-keep alive benchmark
>>>> the
>>>> performance is seriously degraded.
>>>>
>>>> R15B01 not-bound                  8255
>>>> master+mp+pb not-bound    7668
>>>> master+mp+pb bound           5765
>>>>
>>>> I did some gprof runs but could not find anything obvious that is going
>>>> wrong.
>>>>
>>>> Lukas
>>>>
>>>>
>>>> On 11/07/12 04:21, Wei Cao wrote:
>>>>
>>>> I added a macro to conditional compile the patch because I think it
>>>> can be more selectable, I can remove the macro, fix the compilation
>>>> error and test on mingw platform in later version.
>>>>
>>>> how about provide another BIF named port_flag (like process_flag) to
>>>> let user bind port to a given scheduler?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
>




From stefan.zegenhagen@REDACTED  Fri Jul 20 14:51:27 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 14:51:27 +0200
Subject: [erlang-patches] [PATCH] Improve interoperability with SSH clients
Message-ID: <1342788687-25149-1-git-send-email-stefan.zegenhagen@arcutronix.com>

This patch fixes a few problems of the SSH shell server that affect the
interoperability with SSH clients in widespread use.

First problem is that, whenever a channel_request message is
received with want_reply=true, the reply ends up being sent to the
servers channel id, not the clients channel id. This causes the client
to terminate the connection. The easiest solution to the problem appears
to be a new function in ssh_connection_manager.erl that translates the
servers channel id before sending the reply (in the same manner as
other functions do it).

Second problem is in ssh_cli.erl. When an SSH client sends a
window_change request between PTY allocation and starting the shell
(which appears to happen with some clients), ssh_cli.erl crashes
because #state.buf is yet 'undefined'. Allocating an empty buffer at PTY
allocation time solves the problem.

Affected SSH clients:
 - all clients based on SSH-2.0-TrileadSSH2Java_213 (problem #1)
 - SSH Term Pro (problem #2)
---
 lib/ssh/src/ssh_cli.erl                |    3 ++-
 lib/ssh/src/ssh_connection.erl         |    2 +-
 lib/ssh/src/ssh_connection_manager.erl |   15 ++++++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl
index 781e01b..c8c610f 100644
--- a/lib/ssh/src/ssh_cli.erl
+++ b/lib/ssh/src/ssh_cli.erl
@@ -81,7 +81,8 @@ handle_ssh_msg({ssh_cm, ConnectionManager,
 				  height = not_zero(Height, 24),
 				  pixel_width = PixWidth,
 				  pixel_height = PixHeight,
-				  modes = Modes}},
+                  modes = Modes},
+             buf = empty_buf()},
     set_echo(State),
     ssh_connection:reply_request(ConnectionManager, WantReply, 
 				 success, ChannelId),
diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
index c46f799..913f505 100644
--- a/lib/ssh/src/ssh_connection.erl
+++ b/lib/ssh/src/ssh_connection.erl
@@ -177,7 +177,7 @@ close(ConnectionManager, ChannelId) ->
 %% Description: Send status replies to requests that want such replies.
 %%--------------------------------------------------------------------
 reply_request(ConnectionManager, true, Status, ChannelId) ->
-    ConnectionManager ! {ssh_cm, self(), {Status, ChannelId}},
+    ssh_connection_manager:reply_request(ConnectionManager, Status, ChannelId),
     ok;
 reply_request(_,false, _, _) ->
     ok.
diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl
index e53cd4f..e350f96 100644
--- a/lib/ssh/src/ssh_connection_manager.erl
+++ b/lib/ssh/src/ssh_connection_manager.erl
@@ -40,7 +40,7 @@
 	 close/2, stop/1, send/5,
 	 send_eof/2]).
 
--export([open_channel/6, request/6, request/7, global_request/4, event/2,
+-export([open_channel/6, reply_request/3, request/6, request/7, global_request/4, event/2,
 	 cast/2]).
 
 %% Internal application API and spawn
@@ -95,6 +95,9 @@ request(ConnectionManager, ChannelId, Type, true, Data, Timeout) ->
 request(ConnectionManager, ChannelId, Type, false, Data, _) ->
     cast(ConnectionManager, {request, ChannelId, Type, Data}).
 
+reply_request(ConnectionManager, Status, ChannelId) ->
+    cast(ConnectionManager, {reply_request, Status, ChannelId}).
+
 global_request(ConnectionManager, Type, true = Reply, Data) ->
     case call(ConnectionManager, 
 	      {global_request, self(), Type, Reply, Data}) of
@@ -431,6 +434,16 @@ handle_cast({request, ChannelId, Type, Data}, State0) ->
     lists:foreach(fun send_msg/1, Replies),
     {noreply, State};
 
+handle_cast({reply_request, Status, ChannelId}, #state{connection_state =
+        #connection{channel_cache = Cache}} = State0) ->
+    State = case ssh_channel:cache_lookup(Cache, ChannelId) of
+        #channel{remote_id = RemoteId} ->
+            cm_message({Status, RemoteId}, State0);
+        undefined ->
+            State0
+    end,
+    {noreply, State};
+
 handle_cast({global_request, _, _, _, _} = Request, State0) ->
     State = handle_global_request(Request, State0),
     {noreply, State};
-- 
1.7.9.5



From stefan.zegenhagen@REDACTED  Fri Jul 20 14:55:22 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 14:55:22 +0200
Subject: [erlang-patches] [PATCH] Improve interoperability with SSH clients
Message-ID: <1342788687-25149-1-git-send-email-stefan.zegenhagen@arcutronix.com>

This patch fixes a few problems of the SSH shell server that affect the
interoperability with SSH clients in widespread use.

First problem is that, whenever a channel_request message is
received with want_reply=true, the reply ends up being sent to the
servers channel id, not the clients channel id. This causes the client
to terminate the connection. The easiest solution to the problem appears
to be a new function in ssh_connection_manager.erl that translates the
servers channel id before sending the reply (in the same manner as
other functions do it).

Second problem is in ssh_cli.erl. When an SSH client sends a
window_change request between PTY allocation and starting the shell
(which appears to happen with some clients), ssh_cli.erl crashes
because #state.buf is yet 'undefined'. Allocating an empty buffer at PTY
allocation time solves the problem.

Affected SSH clients:
 - all clients based on SSH-2.0-TrileadSSH2Java_213 (problem #1)
 - SSH Term Pro (problem #2)
---
 lib/ssh/src/ssh_cli.erl                |    3 ++-
 lib/ssh/src/ssh_connection.erl         |    2 +-
 lib/ssh/src/ssh_connection_manager.erl |   15 ++++++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl
index 781e01b..c8c610f 100644
--- a/lib/ssh/src/ssh_cli.erl
+++ b/lib/ssh/src/ssh_cli.erl
@@ -81,7 +81,8 @@ handle_ssh_msg({ssh_cm, ConnectionManager,
 				  height = not_zero(Height, 24),
 				  pixel_width = PixWidth,
 				  pixel_height = PixHeight,
-				  modes = Modes}},
+                  modes = Modes},
+             buf = empty_buf()},
     set_echo(State),
     ssh_connection:reply_request(ConnectionManager, WantReply, 
 				 success, ChannelId),
diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
index c46f799..913f505 100644
--- a/lib/ssh/src/ssh_connection.erl
+++ b/lib/ssh/src/ssh_connection.erl
@@ -177,7 +177,7 @@ close(ConnectionManager, ChannelId) ->
 %% Description: Send status replies to requests that want such replies.
 %%--------------------------------------------------------------------
 reply_request(ConnectionManager, true, Status, ChannelId) ->
-    ConnectionManager ! {ssh_cm, self(), {Status, ChannelId}},
+    ssh_connection_manager:reply_request(ConnectionManager, Status, ChannelId),
     ok;
 reply_request(_,false, _, _) ->
     ok.
diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl
index e53cd4f..e350f96 100644
--- a/lib/ssh/src/ssh_connection_manager.erl
+++ b/lib/ssh/src/ssh_connection_manager.erl
@@ -40,7 +40,7 @@
 	 close/2, stop/1, send/5,
 	 send_eof/2]).
 
--export([open_channel/6, request/6, request/7, global_request/4, event/2,
+-export([open_channel/6, reply_request/3, request/6, request/7, global_request/4, event/2,
 	 cast/2]).
 
 %% Internal application API and spawn
@@ -95,6 +95,9 @@ request(ConnectionManager, ChannelId, Type, true, Data, Timeout) ->
 request(ConnectionManager, ChannelId, Type, false, Data, _) ->
     cast(ConnectionManager, {request, ChannelId, Type, Data}).
 
+reply_request(ConnectionManager, Status, ChannelId) ->
+    cast(ConnectionManager, {reply_request, Status, ChannelId}).
+
 global_request(ConnectionManager, Type, true = Reply, Data) ->
     case call(ConnectionManager, 
 	      {global_request, self(), Type, Reply, Data}) of
@@ -431,6 +434,16 @@ handle_cast({request, ChannelId, Type, Data}, State0) ->
     lists:foreach(fun send_msg/1, Replies),
     {noreply, State};
 
+handle_cast({reply_request, Status, ChannelId}, #state{connection_state =
+        #connection{channel_cache = Cache}} = State0) ->
+    State = case ssh_channel:cache_lookup(Cache, ChannelId) of
+        #channel{remote_id = RemoteId} ->
+            cm_message({Status, RemoteId}, State0);
+        undefined ->
+            State0
+    end,
+    {noreply, State};
+
 handle_cast({global_request, _, _, _, _} = Request, State0) ->
     State = handle_global_request(Request, State0),
     {noreply, State};
-- 
1.7.9.5




From stefan.zegenhagen@REDACTED  Fri Jul 20 14:59:16 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 14:59:16 +0200
Subject: [erlang-patches] Improve interoperability with SSH clients
Message-ID: <1342788687-25149-1-git-send-email-stefan.zegenhagen@arcutronix.com>

This patch fixes a few problems of the SSH shell server that affect the
interoperability with SSH clients in widespread use.

First problem is that, whenever a channel_request message is
received with want_reply=true, the reply ends up being sent to the
servers channel id, not the clients channel id. This causes the client
to terminate the connection. The easiest solution to the problem appears
to be a new function in ssh_connection_manager.erl that translates the
servers channel id before sending the reply (in the same manner as
other functions do it).

Second problem is in ssh_cli.erl. When an SSH client sends a
window_change request between PTY allocation and starting the shell
(which appears to happen with some clients), ssh_cli.erl crashes
because #state.buf is yet 'undefined'. Allocating an empty buffer at PTY
allocation time solves the problem.

Affected SSH clients:
 - all clients based on SSH-2.0-TrileadSSH2Java_213 (problem #1)
 - SSH Term Pro (problem #2)
---
 lib/ssh/src/ssh_cli.erl                |    3 ++-
 lib/ssh/src/ssh_connection.erl         |    2 +-
 lib/ssh/src/ssh_connection_manager.erl |   15 ++++++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl
index 781e01b..c8c610f 100644
--- a/lib/ssh/src/ssh_cli.erl
+++ b/lib/ssh/src/ssh_cli.erl
@@ -81,7 +81,8 @@ handle_ssh_msg({ssh_cm, ConnectionManager,
 				  height = not_zero(Height, 24),
 				  pixel_width = PixWidth,
 				  pixel_height = PixHeight,
-				  modes = Modes}},
+                  modes = Modes},
+             buf = empty_buf()},
     set_echo(State),
     ssh_connection:reply_request(ConnectionManager, WantReply, 
 				 success, ChannelId),
diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
index c46f799..913f505 100644
--- a/lib/ssh/src/ssh_connection.erl
+++ b/lib/ssh/src/ssh_connection.erl
@@ -177,7 +177,7 @@ close(ConnectionManager, ChannelId) ->
 %% Description: Send status replies to requests that want such replies.
 %%--------------------------------------------------------------------
 reply_request(ConnectionManager, true, Status, ChannelId) ->
-    ConnectionManager ! {ssh_cm, self(), {Status, ChannelId}},
+    ssh_connection_manager:reply_request(ConnectionManager, Status, ChannelId),
     ok;
 reply_request(_,false, _, _) ->
     ok.
diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl
index e53cd4f..e350f96 100644
--- a/lib/ssh/src/ssh_connection_manager.erl
+++ b/lib/ssh/src/ssh_connection_manager.erl
@@ -40,7 +40,7 @@
 	 close/2, stop/1, send/5,
 	 send_eof/2]).
 
--export([open_channel/6, request/6, request/7, global_request/4, event/2,
+-export([open_channel/6, reply_request/3, request/6, request/7, global_request/4, event/2,
 	 cast/2]).
 
 %% Internal application API and spawn
@@ -95,6 +95,9 @@ request(ConnectionManager, ChannelId, Type, true, Data, Timeout) ->
 request(ConnectionManager, ChannelId, Type, false, Data, _) ->
     cast(ConnectionManager, {request, ChannelId, Type, Data}).
 
+reply_request(ConnectionManager, Status, ChannelId) ->
+    cast(ConnectionManager, {reply_request, Status, ChannelId}).
+
 global_request(ConnectionManager, Type, true = Reply, Data) ->
     case call(ConnectionManager, 
 	      {global_request, self(), Type, Reply, Data}) of
@@ -431,6 +434,16 @@ handle_cast({request, ChannelId, Type, Data}, State0) ->
     lists:foreach(fun send_msg/1, Replies),
     {noreply, State};
 
+handle_cast({reply_request, Status, ChannelId}, #state{connection_state =
+        #connection{channel_cache = Cache}} = State0) ->
+    State = case ssh_channel:cache_lookup(Cache, ChannelId) of
+        #channel{remote_id = RemoteId} ->
+            cm_message({Status, RemoteId}, State0);
+        undefined ->
+            State0
+    end,
+    {noreply, State};
+
 handle_cast({global_request, _, _, _, _} = Request, State0) ->
     State = handle_global_request(Request, State0),
     {noreply, State};
-- 
1.7.9.5




From stefan.zegenhagen@REDACTED  Fri Jul 20 15:11:43 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Fri, 20 Jul 2012 15:11:43 +0200
Subject: [erlang-patches] Problems with erlang SSH server
In-Reply-To: <1342777308.3472.26.camel@ax-sze>
References: <1342777308.3472.26.camel@ax-sze>
Message-ID: <1342789903.3472.34.camel@ax-sze>

Dear Henrik,

I tried using git-send-email now to post the patch, but unfortunately,
my mails seem to have get filtered at the erlang.org server. Next try is
to send the patch as attachment. The patch should then apply well.

Are there any other patches you want me to resend?


Kind regards,

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Improve-interoperability-with-SSH-clients.patch
Type: text/x-patch
Size: 4261 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120720/1a8dafc4/attachment.bin>

From cyg.cao@REDACTED  Tue Jul 24 11:57:40 2012
From: cyg.cao@REDACTED (Wei Cao)
Date: Tue, 24 Jul 2012 17:57:40 +0800
Subject: [erlang-patches] Pollset per scheduler and bind port to
	scheduler
In-Reply-To: <4FFD915C.2040804@erlang.org>
References: <CADS4RaxsKCt+6cL+7cXB7Q=KA9yKu6Ym2HjZQA1PTu-QndEELA@mail.gmail.com>
 <4FFD915C.2040804@erlang.org>
Message-ID: <CADS4RaxhmirT=fdiUJrUqovcqO-WKso=E9qdMuPtdnT_PQ4iqQ@mail.gmail.com>

2012/7/11 Lukas Larsson <lukas@REDACTED>:
> Hi,
>
> The reason I'm skeptical about anything which binds processes/ports to
> scheduler is that it feels like a temporary solution and would much rather
> do a proper solution where the scheduler takes care of these things for you.
> But as I said, internally we need to talk this over when it is not in the
> middle of summer vacation.
>
> I did some benchmarking using ab and found basically the same figures as
> you. The below is with keep-alive and the values are requests per second:
>
>                    not-bound        bound
>
> R15B01                 44k        37k
>
> master                  44k        35k
>
> master+mp          48k        49k
>
> master+mp+pb    49k        55k
>
> [mp]: multi-poll patch
> [pb]: port bind patch
> [bound]: Used {scheduler,I} to spread load
>
> Unfortunately I also found that when doing the non-keep alive benchmark the
> performance is seriously degraded.
>
> R15B01 not-bound                  8255
> master+mp+pb not-bound    7668
> master+mp+pb bound           5765
>

I found why performance degrates in non-keep alive benchmark. It's
caused by waking schedulers up to do aux works too frequently.

After applying pollset per scheduler patch, each scheduler tends to
wait in poll operation if there is no heavy load. So if an aux work
arrives at this time, erts_check_io_interrupt() will be called and
write a byte to a pipe fd to wake up the scheduler from poll
operation.

Unfortunately, there're quite a lot of aux works in Erlang VM, such as
delayed dealloc, so schedulers're frequently woken up from poll to
process these tasks, say, deallocate a memory block, and go into poll
again.

In a non-keep alive benchmark with 15k QPS, schedulers're woken up
about 300k times per second (I hacked the code adding a atomic counter
to record it), it's really time cosuming.

After commenting out the code which wakes up scheduler to process aux
work, the QPS increases to 20k, which proves above suspect.

I think it's really CPU time wasting to wake up scheduler from polling
to deallocate some memory blocks etc, may be it's more suitable to
wake schedulers up periodically (say, 1 millisecond) to process them,
or doing aux tasks in the aux_thread only. Any ideas to fix this?


> I did some gprof runs but could not find anything obvious that is going
> wrong.
>
> Lukas
>
>
> On 11/07/12 04:21, Wei Cao wrote:
>>
>> I added a macro to conditional compile the patch because I think it
>> can be more selectable, I can remove the macro, fix the compilation
>> error and test on mingw platform in later version.
>>
>> how about provide another BIF named port_flag (like process_flag) to
>> let user bind port to a given scheduler?
>>
>>
>



-- 

Best,

Wei Cao


From henrik@REDACTED  Tue Jul 24 15:11:06 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 24 Jul 2012 15:11:06 +0200
Subject: [erlang-patches] SCTP Peeloff socket can block erlang VM if set
 to {active, true}
In-Reply-To: <5D51F801-4627-401F-A342-CE5537A6133A@mobilearts.com>
References: <C0B7BBD6-F5BE-4E55-9D81-391246478C19@mobilearts.com>
 <5D51F801-4627-401F-A342-CE5537A6133A@mobilearts.com>
Message-ID: <500E9EEA.6050602@erlang.org>

Thank you, I have included your patch(es) in master-pu.

You can have the tests in the same branch as the fix, in a separate 
commit instead of 2 branches.


On 07/20/2012 02:36 PM, Jonas Falkevik wrote:
> The previous git fetch command seem to faulty, sorry for that.
> Let me try again.
>
> On Jul 20, 2012, at 0:46 , Jonas Falkevik wrote:
>>
>> git fetch git@REDACTED:falkevik/otp.git fix_sctp_peeloff_active_true
> git fetch git://github.com/falkevik/otp.git fix_sctp_peeloff_active_true
>
>>
>> git fetch git@REDACTED:falkevik/otp.git sctp_peeloff_active_true_test
> git fetch git://github.com/falkevik/otp.git sctp_peeloff_active_true_test
>
>
> Jonas
>
>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120724/d96c1728/attachment.htm>

From henrik@REDACTED  Tue Jul 24 15:15:34 2012
From: henrik@REDACTED (Henrik Nord)
Date: Tue, 24 Jul 2012 15:15:34 +0200
Subject: [erlang-patches] Problems with erlang SSH server
In-Reply-To: <1342789903.3472.34.camel@ax-sze>
References: <1342777308.3472.26.camel@ax-sze> <1342789903.3472.34.camel@ax-sze>
Message-ID: <500E9FF6.1060007@erlang.org>

Hello

This attached patch worked like a charm.

Feel free to send the other patches the same way.

Thank you for your contributions!



On 07/20/2012 03:11 PM, Stefan Zegenhagen wrote:
> Dear Henrik,
>
> I tried using git-send-email now to post the patch, but unfortunately,
> my mails seem to have get filtered at the erlang.org server. Next try is
> to send the patch as attachment. The patch should then apply well.
>
> Are there any other patches you want me to resend?
>
>
> Kind regards,
>
>
>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120724/b74eca5d/attachment.htm>

From jeremy.heater@REDACTED  Tue Jul 24 17:15:55 2012
From: jeremy.heater@REDACTED (Jeremy Heater)
Date: Tue, 24 Jul 2012 17:15:55 +0200
Subject: [erlang-patches] Add specific return for incomplete asn.1 packets
Message-ID: <CADLoKj0AgzgOarmy__DBBZ97Pvd0uHmANzMJMd1wkvQSvYCMKg@mail.gmail.com>

Hello all,

We had already broached the subject a few years back (cf.
http://erlang.org/pipermail/erlang-patches/2009-December/000627.html) and
we modified our approach to take the comments made at the time into
account. Here follows a description of the changes:

This patch's goal is to allow the user to determine if an incoming
asn.1 packet is incomplete.
To that end, the decode/2 functions of generated asn.1 files were
modified so as to return {error, incomplete} when the packet to be
decoded is incomplete instead of the generic
{error, {asn1, {badmatch, _}}} (or badarg) which is not distinguishable
from other errors.

This would seem to us to be a necessary feature so as to be able to
determine if a packet received from a socket is complete (since overly
large messages are split).
It is also complementary to the undec_rest option where when decode/2
is called on a packet containing more than one message, the function
returns the first message decoded and the rest of the packet
undecoded.

The solution was implemented for all encoding rules, even though BER
encoding allows for simpler checking by manually parsing the first
bytes of the packet, for the sake of having a consistent API whatever
the encoding.

The proposed patch can be found at:
git fetch git://github.com/Kwisatx/otp.git incomplete_packet_asn1

https://github.com/Kwisatx/otp/compare/master...incomplete_packet_asn1
https://github.com/Kwisatx/otp/compare/master...incomplete_packet_asn1.patch


Since the added tests inevitably add some overhead, we also tried to
link the changes to the undec_rest option
(incomplete_packet_asn1_duplicate_files branch), but since most of the
changes are in the runtime, it made duplicating the code necessary (as
the name of the branch suggests), which complicates the code.

What are your thoughts on this change? Any ideas for how to improve it?

Best regards,

Jeremy Heater.


From bernie@REDACTED  Wed Jul 25 09:44:55 2012
From: bernie@REDACTED (Bernie Duggan)
Date: Wed, 25 Jul 2012 17:44:55 +1000
Subject: [erlang-patches] ODBC extended error information
Message-ID: <CA+rqyemvr9mVp4bp93VOhq5p29fWbVCGBgik5-iDJ1SdT7MxRQ@mail.gmail.com>

Hi all,
This patch adds an option to ODBC to allow for extended error reports.
The standard ODBC error reasons are, in some circumstances, not
sufficient for sophisticated error handling. Enabling the
'extended_errors' option on an ODBC connection alters the error return
type from {error, Reason} to {error, {ODBCErrorCode, NativeErrorCode,
Reason}}. The NativeErrorCode is that returned by the underlying
database and can be useful when that database type is known and
specialised error handling is desired.

I did my best to figure out how to add some tests for this change, but
from talking to a couple of other people I'm not alone in having
difficulty even getting the ODBC suite running :) Following the
instructions on the github wiki page just gives me a
port_program_executable_not_found error. I'm happy to try harder to
add some tests if someone can give me a hand with this.

You can grab the changes with

git fetch git://github.com/bernardd/otp.git odbc_extended_errors

Cheers,

Bernard


From magnus.henoch@REDACTED  Wed Jul 25 13:51:39 2012
From: magnus.henoch@REDACTED (Magnus Henoch)
Date: Wed, 25 Jul 2012 12:51:39 +0100 (BST)
Subject: [erlang-patches] Link from erlang:now/0 to os:timestamp/0
Message-ID: <719bc237-fe67-4a83-aafa-3500cd8ec895@knuth>

A small documentation improvement:

git fetch git://github.com/legoscia/otp.git doc-now-vs-timestamp

https://github.com/legoscia/otp/compare/doc-now-vs-timestamp
https://github.com/legoscia/otp/compare/doc-now-vs-timestamp.patch

Regards,
-- 
Magnus Henoch
Erlang Solutions Ltd
http://www.erlang-solutions.com/



From magnus.henoch@REDACTED  Wed Jul 25 19:08:34 2012
From: magnus.henoch@REDACTED (Magnus Henoch)
Date: Wed, 25 Jul 2012 18:08:34 +0100 (BST)
Subject: [erlang-patches] Edoc to detect behaviours using -callback
	attributes
In-Reply-To: <cb1be344-416f-4d7d-9615-d7c47a225b8d@knuth>
Message-ID: <ebed538d-56e1-4ed3-8be8-3a33883388cd@knuth>

Make edoc treat behaviours defined with -callback the same as
behaviours defined with behaviour_info:

git fetch git://github.com/legoscia/otp.git edoc-callback-attribute

https://github.com/legoscia/otp/compare/edoc-callback-attribute
https://github.com/legoscia/otp/compare/edoc-callback-attribute.patch

Regards,
-- 
Magnus Henoch
Erlang Solutions Ltd
http://www.erlang-solutions.com/



From carlsson.richard@REDACTED  Wed Jul 25 22:26:51 2012
From: carlsson.richard@REDACTED (Richard Carlsson)
Date: Wed, 25 Jul 2012 22:26:51 +0200
Subject: [erlang-patches] Edoc to detect behaviours using -callback
	attributes
In-Reply-To: <ebed538d-56e1-4ed3-8be8-3a33883388cd@knuth>
References: <ebed538d-56e1-4ed3-8be8-3a33883388cd@knuth>
Message-ID: <5010568B.7040802@gmail.com>

On 2012-07-25 19:08, Magnus Henoch wrote:
> Make edoc treat behaviours defined with -callback the same as
> behaviours defined with behaviour_info:
>
> git fetch git://github.com/legoscia/otp.git edoc-callback-attribute
>
> https://github.com/legoscia/otp/compare/edoc-callback-attribute
> https://github.com/legoscia/otp/compare/edoc-callback-attribute.patch
>
> Regards,
>

Thanks! I'll look at them and merge them to my edoc dev repo 
(github.com/richcarl/edoc), and try to get it into OTP for the next release.

    /Richard



From bernie@REDACTED  Thu Jul 26 08:59:47 2012
From: bernie@REDACTED (Bernie Duggan)
Date: Thu, 26 Jul 2012 16:59:47 +1000
Subject: [erlang-patches] ODBC extended error information
In-Reply-To: <CA+rqyemvr9mVp4bp93VOhq5p29fWbVCGBgik5-iDJ1SdT7MxRQ@mail.gmail.com>
References: <CA+rqyemvr9mVp4bp93VOhq5p29fWbVCGBgik5-iDJ1SdT7MxRQ@mail.gmail.com>
Message-ID: <CA+rqyenkBS7tUuq-F0otWWoRxZEGJzzRnQ5V947YamXVdJ6ahQ@mail.gmail.com>

I neglected to mention that credit (or blame ;)) for the original code
should go to my colleague Franko Franicevich.

Cheers,

B

On 25 July 2012 17:44, Bernie Duggan <bernie@REDACTED> wrote:
> Hi all,
> This patch adds an option to ODBC to allow for extended error reports.
> The standard ODBC error reasons are, in some circumstances, not
> sufficient for sophisticated error handling. Enabling the
> 'extended_errors' option on an ODBC connection alters the error return
> type from {error, Reason} to {error, {ODBCErrorCode, NativeErrorCode,
> Reason}}. The NativeErrorCode is that returned by the underlying
> database and can be useful when that database type is known and
> specialised error handling is desired.
>
> I did my best to figure out how to add some tests for this change, but
> from talking to a couple of other people I'm not alone in having
> difficulty even getting the ODBC suite running :) Following the
> instructions on the github wiki page just gives me a
> port_program_executable_not_found error. I'm happy to try harder to
> add some tests if someone can give me a hand with this.
>
> You can grab the changes with
>
> git fetch git://github.com/bernardd/otp.git odbc_extended_errors
>
> Cheers,
>
> Bernard


From henrik@REDACTED  Fri Jul 27 08:19:42 2012
From: henrik@REDACTED (Henrik Nord)
Date: Fri, 27 Jul 2012 08:19:42 +0200
Subject: [erlang-patches] Edoc to detect behaviours using -callback
	attributes
In-Reply-To: <5010568B.7040802@gmail.com>
References: <ebed538d-56e1-4ed3-8be8-3a33883388cd@knuth>
 <5010568B.7040802@gmail.com>
Message-ID: <501232FE.6080600@erlang.org>


On 07/25/2012 10:26 PM, Richard Carlsson wrote:
> On 2012-07-25 19:08, Magnus Henoch wrote:
>> Make edoc treat behaviours defined with -callback the same as
>> behaviours defined with behaviour_info:
>>
>> git fetch git://github.com/legoscia/otp.git edoc-callback-attribute
>>
>> https://github.com/legoscia/otp/compare/edoc-callback-attribute
>> https://github.com/legoscia/otp/compare/edoc-callback-attribute.patch
>>
>> Regards,
>>
>
> Thanks! I'll look at them and merge them to my edoc dev repo 
> (github.com/richcarl/edoc), and try to get it into OTP for the next 
> release.
>
>    /Richard
Ok sounds good.

/Henrik

>
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches

-- 
/Henrik Nord Erlang/OTP



From ahf@REDACTED  Fri Jul 27 23:06:18 2012
From: ahf@REDACTED (=?ISO-8859-1?Q?Alexander_F=E6r=F8y?=)
Date: Fri, 27 Jul 2012 23:06:18 +0200
Subject: [erlang-patches] Add support for RIPEMD-160 to the crypto module
Message-ID: <CAK8g1zE-0pjTSx1SvppK74Bes7V5=TvDqP6s7jtk6H9JnjzoQw@mail.gmail.com>

Hello,

This patch adds support for the RIPEMD-160 hash function to the crypto module.
The patch includes both tests and documentation for the newly created
functions.

git fetch git://github.com/ahf/otp.git feature-crypto-ripemd160

https://github.com/ahf/otp/compare/feature-crypto-ripemd160
https://github.com/ahf/otp/compare/feature-crypto-ripemd160.patch

Cheers,
Alex.

-- 
Alexander F?r?y


From stefan.zegenhagen@REDACTED  Tue Jul 31 11:42:01 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Tue, 31 Jul 2012 11:42:01 +0200
Subject: [erlang-patches] SNMP bugfix to vacmAccessTable
Message-ID: <1343727721.29035.6.camel@ax-sze>

Dear all,

some SNMP test suites complain about errors when checking the
implementation of the SNMP table vacmAccessTable. When trying to modify
this table, it can happen that a "genErr" response is returned where a
different error is expected.

genErr is usually an indication of bad programming as it signals
internal agent errors. The patch attached to this e-mail solves the
genErr issue and adjusts the semantic of set operations to match the
expected behaviour.


Kind regards,

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Errors-in-vacmAccessTable-RowStatus-handling.patch
Type: text/x-patch
Size: 6703 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120731/10d4b269/attachment.bin>

From stefan.zegenhagen@REDACTED  Tue Jul 31 11:53:12 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Tue, 31 Jul 2012 11:53:12 +0200
Subject: [erlang-patches] SNMP bugfix to usmUserTable
Message-ID: <1343728392.29035.10.camel@ax-sze>

Dear all,

usmUserTable suffers from the problem that it doesn't support creating
new users (by cloning them) and changing their passwords in the same
operation.

This is allowed by the MIB definition, and some SNMP test suites do
exactly that numerous times when they check the correctness of SNMPv3
support. SNMP managers might also make use of that feature.

The patch attached to this e-mail solves the issue and allows SNMPv3
user creation to succeed if all required information is passed to the
agent in a single SNMP set request.

Kind regards,

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-SNMP-USM-semantic-fixes-to-SNMP-USER-BASED-SM_MIB.patch
Type: text/x-patch
Size: 11480 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120731/b05b2138/attachment.bin>

From stefan.zegenhagen@REDACTED  Tue Jul 31 15:49:59 2012
From: stefan.zegenhagen@REDACTED (Stefan Zegenhagen)
Date: Tue, 31 Jul 2012 15:49:59 +0200
Subject: [erlang-patches] SNMP: fix to vacmAccessTable walk
Message-ID: <1343742599.29035.12.camel@ax-sze>

Dear all,

a get_next sweep over vacmAccessTable does not return all stored table
data. Instead, only the first column of all rows is returned, plus all
columns from the last row.

The patch attached to this e-mail solves the issue and makes the
get_next operation iterate over all table data.


Kind regards,

-- 
Dr. Stefan Zegenhagen

arcutronix GmbH
Garbsener Landstr. 10
30419 Hannover
Germany

Tel:   +49 511 277-2734
Fax:   +49 511 277-2709
Email: stefan.zegenhagen@REDACTED
Web:   www.arcutronix.com

*Synchronize the Ethernet*

General Managers: Dipl. Ing. Juergen Schroeder, Dr. Josef Gfrerer -
Legal Form: GmbH, Registered office: Hannover, HRB 202442, Amtsgericht
Hannover; Ust-Id: DE257551767.

Please consider the environment before printing this message.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-SNMP-VACM-fix-walk-over-vacmAccessTable.patch
Type: text/x-patch
Size: 1830 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120731/4ca6ccee/attachment.bin>