Patch for configurable ERTS_DE_BUSY_LIMIT

[Scott Lystig Fritchie]

Hi, all.  I sent a longer post to the erlang-questions about this.

In a busy system sending large messages in quick succession, I see
'busy_dist_port' messages coming to my monitoring proc, sometimes 5 or
even 15 times per second for the same process.  And then there are the
times where multiple processes are blocked due all sending to procs on
the same destination node.

Don't get me wrong, this kind of scheduler-controlled congestion
mechanism is a good thing.  But the 128KB limit feels cramped.  When
using 4MB instead, in a system with modern 64-bit multi-core CPUs and
gigabit Ethernet, the latency jitter appears to drop significantly for
us.

Patch relative to R13B03.

-Scott

--- snip --- snip --- snip --- snip --- snip --- snip ---

diff -ur otp_src_R13B03.orig/erts/emulator/beam/dist.c otp_src_R13B03/erts/emulator/beam/dist.c
--- otp_src_R13B03.orig/erts/emulator/beam/dist.c	2009-11-20 05:29:24.000000000 -0800
+++ otp_src_R13B03/erts/emulator/beam/dist.c	2010-02-14 15:06:32.000000000 -0800
@@ -97,6 +97,9 @@
 #define PASS_THROUGH 'p'        /* This code should go */
 
 int erts_is_alive; /* System must be blocked on change */
+#define ERTS_DE_BUSY_LIMIT (128*1024)
+int erts_de_busy_limit = ERTS_DE_BUSY_LIMIT;
+
 
 /* distribution trap functions */
 Export* dsend2_trap = NULL;
@@ -1407,8 +1410,6 @@
     return -1;
 }
 
-#define ERTS_DE_BUSY_LIMIT (128*1024)
-
 static int
 dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy)
 {
diff -ur otp_src_R13B03.orig/erts/emulator/beam/erl_init.c otp_src_R13B03/erts/emulator/beam/erl_init.c
--- otp_src_R13B03.orig/erts/emulator/beam/erl_init.c	2009-09-18 07:07:46.000000000 -0700
+++ otp_src_R13B03/erts/emulator/beam/erl_init.c	2010-02-14 16:08:33.000000000 -0800
@@ -788,6 +788,7 @@
     char envbuf[21]; /* enough for any 64-bit integer */
     size_t envbufsz;
     int async_max_threads = erts_async_max_threads;
+    extern int erts_de_busy_limit;
 
     early_init(&argc, argv);
 
@@ -807,6 +808,9 @@
     if (erts_sys_getenv("ERL_THREAD_POOL_SIZE", envbuf, &envbufsz) == 0) {
 	async_max_threads = atoi(envbuf);
     }
+    if (erts_sys_getenv("ERTS_DE_BUSY_LIMIT", envbuf, &envbufsz) == 0) {
+	erts_de_busy_limit = atoi(envbuf);
+    } 
     
 
 #ifdef DEBUG