[erlang-questions] R11B-1 on Solaris 10

Rob erlq@REDACTED
Thu Oct 26 03:33:08 CEST 2006


Peter-Henry Mander wrote:
> Hi Mikael,
> 
> This is the output of isalist, optisa and psrinfo:
> 
> [510] isalist
> amd64 pentium_pro+mmx pentium_pro pentium+mmx pentium i486 i386 i86
> [511] optisa isalist
> (no output)
> [512] psrinfo
> 0       on-line   since 10/04/2006 11:59:37
> 1       on-line   since 10/04/2006 11:59:40
> 2       on-line   since 10/04/2006 11:59:42
> 3       on-line   since 10/04/2006 11:59:44
> 4       on-line   since 10/04/2006 11:59:46
> 5       on-line   since 10/04/2006 11:59:48
> 6       on-line   since 10/04/2006 11:59:50
> 7       on-line   since 10/04/2006 11:59:52
> [513] uname -a
> SunOS elastigirl 5.10 Generic_118844-26 i86pc i386 i86pc
> [514] uname -m
> i86pc
> 
> It does look as if Sun broke uname.
> 
> The error reported is:
> configure: error: cannot build smp enabled emulator since hardware platform (x86 wordsize=8) is not supported
> 
> Maybe a fix would be to asume that wordsize=8 on x86 indicates AMD x86_94 or Intel EMT-64 CPU if the OS is Solaris?
> 
> Pete.
> 
> 
> on Wed Oct 25 09:39:52 CEST 2006, Mikael Pettersson wrote:
> 
> I see that Solaris' `man uname` describes -p as preferred over -m, but
> they fail to mention that uname -p is functionally inferior to -m.
> 
> In any case, what does `uname -a`, `isalist`, and `optisa `isalist``
> report on Solaris 10 on AMD64?
> 
> erts/configure.in uses `uname -m` not `uname -p`, so unless Sun has
> broken uname, it should report 64-bit capability.
> 
> As a last resort (if Sun indeed broke uname), we can add a test in
> erts/configure.in to check for 64-bit capability via the C compiler.

For comparison purposes, here is the output from the same commands
on my Solaris Express box (SuperMicro P4 3.0 GHz HyperThreaded, 2 GB RAM):
# uname -a
SunOS sm1 5.11 snv_46 i86pc i386 i86pc
# uname -p
i386
# uname -m
i86pc
# psrinfo -v
Status of virtual processor 0 as of: 10/25/2006 12:56:30
   on-line since 10/09/2006 20:13:37.
   The i386 processor operates at 3000 MHz,
         and has an i387 compatible floating point processor.
Status of virtual processor 1 as of: 10/25/2006 12:56:30
   on-line since 10/09/2006 20:13:45.
   The i386 processor operates at 3000 MHz,
         and has an i387 compatible floating point processor.

* Pete what is the output for your machine with -v ?  *
* Note the i386 in my listing... *

# isalist
amd64 pentium_pro+mmx pentium_pro pentium+mmx pentium i486 i386 i86
# optisa isalist
# optisa `isalist`
amd64
# dmesg | more
...
Oct  9 20:13:35 sm1 unix: [ID 126719 kern.info] features: 10fffdf<cpuid,cx16,sse3,nx,asysc,htt,sse2,sse,sep,pat,cx8,pae,mca,mmx,cmov,pge,mtrr,msr,tsc,lgpg>
Oct  9 20:13:35 sm1 unix: [ID 168242 kern.info] mem = 2087420K (0x7f67f000)
Oct  9 20:13:35 sm1 rootnex: [ID 466748 kern.info] root nexus = i86pc
...
Oct  9 20:13:43 sm1 unix: [ID 950921 kern.info] cpu0: x86 (chipid 0x0 GenuineIntel family 15 model 4 step 3 clock 3000 MHz)
Oct  9 20:13:43 sm1 unix: [ID 950921 kern.info] cpu0: Intel(r) Pentium(r) 4 CPU 3.00GHz
Oct  9 20:13:43 sm1 unix: [ID 950921 kern.info] cpu1: x86 (chipid 0x0 GenuineIntel family 15 model 4 step 3 clock 3000 MHz)
Oct  9 20:13:43 sm1 unix: [ID 950921 kern.info] cpu1: Intel(r) Pentium(r) 4 CPU 3.00GHz
Oct  9 20:13:45 sm1 unix: [ID 557827 kern.info] cpu1 initialization complete - online

# ./configure --enable-smp-support --disable-odbc
At 15:55
# gmake
At 16:19
# gmake install

My times for running the smp version of the bigbang benchmark were worse that the non-smp runs,
probably because my box isn't really smp.
Some ideas for people with true (not hyperthreaded) SMP Solaris boxes to try.
With pbind and priocntl you can bind your processes to certain processors or processor groups.
In the past, with Solaris 2.51, I was able to assign 3 of 4 processors to my sybase engine processes
and give them extremely long time quantums, so that they were never context switched or migrated.

It would be interesting to see the results of benchmarks varying
--enable-threads, smp, hipe, processor binding and process time quantum changes.

If a system is dedicated solely for the purpose of running a single app, I never understood the point of the normal variable scheduler.
I don't want my dedicated app to have its time quantum _and_ priority reduced just because it was hogging CPU,
I want it to hog all the CPU it wants to hog.
With the new Solaris FX scheduler priority you can set a process (or group of processes) to have a fixed priority and more importantly
a fixed and long time quantum. In an 8 processor box, you could set processors 1-7 to a group then bind your app to it.
I like to leave processor 0 free with no bound processes to handle interrupts and since I had heard rumors that
the OS in some cases treats processor 0 specially (I am not sure that Solaris does this).

Below are some example runs with mpstat output, but again this doesn't show what would happen with true smp.

Rob

PS If anyone know the commands to do the same things in linux 2.6.x, please let me know - TIA.

# start smp erlang shell
bash-3.00# erl -smp
Erlang (BEAM) emulator version 5.5.1 [source] [smp:2] [async-threads:0] [hipe]

Eshell V5.5.1  (abort with ^G)

1> big:bang(1000).

http://www.princeton.edu/~psg/unix/solaris/troubleshoot/mpstat.html
bash-3.00# mpstat 8
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0  312   0    0   327  224   88    9   13    0    0  7921   32   1   0  67
   1  571   0    0    51    5   80   10   14    0    0 11398   44   2   0  54
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0  274   0    0   329  226   85   11   14    0    0  8907   45   1   0  54
   1  316   0    0    50    5   80   12   14    0    0  9731   53   1   0  46
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0  215   0    0   325  223   61   11   12    0    0  9864   61   1   0  37
   1  126   0    0    58    5  101   10   13    0    0  5911   37   1   0  61
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0   73   0    0   328  225   60   14   13    0    0  4496   66   1   0  34
   1   23   0    0    60    5  105    9   13    0    0  1382   34   1   0  65
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0   37   0    0   326  224   53   12   12    0    0  2043   69   0   0  31
   1   16   0    0    61    5  105    8   12    0    0   906   31   0   0  68
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0   36   0    0   327  224   30   12    8    0    0  2107   86   0   0  14
   1    6   0    0    70    5  130    6    9    0    0   399   14   0   0  85
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0   17   0    0   326  224   50   12   12    0    0  1736   72   0   0  27
   1    7   0    0    64    5  112    9   11    0    0   597   28   1   0  72

# start up erlang shell (no smp)
bash-3.00# erl
Erlang (BEAM) emulator version 5.5.1 [source] [async-threads:0] [hipe]

Eshell V5.5.1  (abort with ^G)

# get the OS process id
bash-3.00# ps -elf | grep erl
  0 S     root 15720  5242   0  40 20        ?  55282        ? 16:45:07 pts/5       2:36 /usr/local/lib/erlang/erts-5.5.1/bi
# bind that process to processor #1
bash-3.00# pbind -b 1 15720
process id 15720: was not bound, now 1

10> big:bang(1000).

# Note how now only psr 1 is using any CPU and the reduced csw for psr 1 and reduced migr
bash-3.00# mpstat 8
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   325  223  139    0    3    0    0    41    0   0   0  99
   1  519   0    0    27    5   24   20    4    0    0 19089   98   2   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   325  224  139    0    3    0    0    43    0   0   0  99
   1  549   0    0    27    5   24   20    3    0    0 18199   97   3   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   326  224  139    0    3    0    0    40    0   0   0  99
   1  205   0    0    28    5   25   20    3    0    0 10856   99   1   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   328  226  140    0    3    0    0    38    0   0   0  99
   1   80   0    0    28    5   24   20    2    0    0  4007   99   1   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   326  224  138    0    4    0    0    42    0   0   0  99
   1   46   0    0    29    5   27   21    4    0    0  2469  100   0   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   327  225  141    0    3    0    0    45    0   0   0  99
   1   28   0    0    27    5   23   19    3    0    0  2087  100   0   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   326  224  139    0    3    0    0    42    0   0   0 100
   1    2   0    0    28    5   24   21    3    0    0  1391  100   0   0   0

59126403

# assign the running erlang shell process to the FX scheduling class
# and give it an obscenely large time quantum (and fairly high priority)
bash-3.00# priocntl -s -c FX -m 59 -p 59 -t 99999 -i pid 15720

# show us the scheduling classes and details for each OS process
bash-3.00# priocntl -d -i all
...
FIXED PRIORITY PROCESSES:
     PID    FXUPRILIM    FXUPRI      FXTQNTM
       0       60          60       20000
   15720       59          59      100000  <---- yahooo !
bash-3.00# ps -lfp 0
  F S      UID   PID  PPID   C PRI NI     ADDR     SZ    WCHAN    STIME TTY         TIME CMD
  1 T     root     0     0   0  39  0        ?      0            Oct 09 ?           0:40 sched
bash-3.00# ps -lfp 15720
  F S      UID   PID  PPID   C PRI NI     ADDR     SZ    WCHAN    STIME TTY         TIME CMD
  0 S     root 15720  5242   0  40  1        ?  88306        ? 16:45:07 pts/5       4:33 /usr/local/lib/erlang/erts-5.5.1/bi

11> big:bang(1000).

# Note how now csw/icsw/migr counts are further reduced,
# using a true smp machine we should be able to reduce the psr 1 csw/migr count to near zero
bash-3.00# mpstat 8
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   326  224  226    0    2    0    0    52    0   0   0  99
   1  585   0    0    13    5    9    5    1    0    0 19921   97   3   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0   23   330  224  241    0    2    4    0    49    0   1   0  99
   1  561   0    2   112   78   11    9    2    5    0 17652   97   3   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0   17   333  224  240    2    2    1    0    53    0   1   0  99
   1  217   0    4    67   39   12    9    2    1    0 10895   98   2   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    4   326  223  237    0    1    0    0    51    0   0   0  99
   1   80   0    0    24   13    7    5    1    0    0  3947   99   1   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    4   341  224  228    3    1    0    0    50    0   0   0  99
   1   47   0    0    27   13   24    8    1    0    0  2506  100   0   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    2   326  224  237    0    1    0    0    51    0   0   0  99
   1   32   0    0    17    9    6    4    1    0    0  2103  100   0   0   0
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
   0    0   0    0   331  224  241    4    1    0    0    60    0   0   0  99
   1    1   0    0    16    5   14    8    1    0    0  1429  100   0   0   0




More information about the erlang-questions mailing list