Design phase in Erlang

[Marc Feeley]

> }  http://www.sics.se/~joe/ericsson/du98024.html
> 
> Just for fun I tried out this test on an UltraSparc 5 running
> Solaris 8 and R9B-0.  I found that it started paging at about
> 70K processes with 256MB of memory so I upgraded it to 512MB
> and found I could go to 180K processes before it paged.

I ran a similar benchmark in Scheme (compiled using Gambit-C and run
on a 2.4GHz Pentium 4 with 1GBytes of RAM running Linux).  The Scheme
code is attached below.  I ran it with values of N (number of
processes) from 10 to 900 thousand.  Here is a table indicating the
process creation time and the time to send/receive one message in
microseconds):

        process    message
        creation  send/receive
   N    time (us)  time (us)
______    _____      _____
    10                2.36
  1000                2.73
 10000     2.53       2.75
100000     2.69       2.71
200000     2.65       2.74
300000     2.61       2.73
400000     2.60       2.70
500000     2.62       2.74
600000     2.68       2.68
700000     6.91       2.54
800000     6.24       3.01
900000     6.14       2.78

The message passing time stayed at roughly 2.7 microseconds (like
Erlang/OTP with non-shared heap the time does not vary much with N).
However process creation time was around 2.5 microseconds when N <
700000, but above that it jumped to 6 microseconds because the heap
was close to its limit (I gave the system a 1GByte heap so that a very
high number of processes could be created).

My conclusion is that Gambit-C scales very well in the sense that
there is very little variation with N (just like Erlang/OTP with
non-shared heap).  The use of a unified heap allows message passing to
be done in constant time (i.e. no copying necessary).  Moreover
Gambit-C can handle a very high number of processes if there is enough
memory (roughly 1 KByte per process).  It would appear from your
experiment with the UltraSparc that Erlang/OTP is using about 2.7
KBytes per process (i.e. 512 MBytes/190000).  In relative terms (since
the processors aren't the same) the process creation time seems a bit
better in Gambit-C, and the message passing time is a bit worse.

Marc


-------------------------------------------------------------------------------

; file: "ring.scm"

(declare (standard-bindings))

(define (joes-challenge n m)

  (define (create-processes)

    (declare (fixnum) (not safe))

    (define (iota n)
      (iota-aux n '()))

    (define (iota-aux n lst)
      (if (= n 0)
          lst
          (iota-aux (- n 1) (cons n lst))))

    (let* ((1-to-n
            (iota n))
           (channels
            (list->vector
             (map (lambda (i) (open-vector))
                  1-to-n)))
           (processes
            (map (lambda (i)
                   (let ((input (vector-ref channels (modulo (- i 1) n)))
                         (output (vector-ref channels (modulo i n))))
                     (make-thread
                      (lambda ()
                        (let loop ((j m))
                          (if (> j 0)
                              (let ((message (read input)))
                                (write message output)
                                (force-output output)
                                (loop (- j 1)))))))))
                 1-to-n)))
      (write 'go (vector-ref channels 1))
      (force-output (vector-ref channels 1))
      processes))

  (let* ((point1
          (cpu-time))
         (processes
          (create-processes))
         (point2
          (cpu-time)))

    (for-each thread-start! processes)
    (thread-join! (car processes))

    (let ((point3
           (cpu-time)))

      (display n)
      (display " ")
      (display (/ (- point2 point1) n))
      (display " ")
      (display (/ (- point3 point2) (* n m)))
      (newline))))

(define (main)

  (define (usage) (display "usage: ring <nb-processes>\n"))

  (let ((args (cdr (command-line))))
    (if (null? args)
        (usage)
        (let ((n (string->number (car args))))
          (if (not (and (integer? n)
                        (exact? n)
                        (>= n 2)
                        (<= n 1000000)))
              (usage)
              (let ((m (quotient 1000000 n)))
                (joes-challenge n m)))))))

(main)

-------------------------------------------------------------------------------